# Example 3: Block all crawlers except AdsBot (AdsBot crawlers must be named explicitly) User-agent: * # Disallow: / # disallow: /file.asp disallow: /members/roster2.html disallow: /members/roster.html disallow: /members/docs/MembershipRoster2.pdf disallow: /members/docs/MembershipRoster.pdf disallow: /members/docs/MembershipRoster2.xlsx disallow: /members/docs/MembershipRoster.xlsx disallow: /breeders2.html disallow: /breeders.html