# Robots.txt file for http://www.standardsociety.com # # This file is used to allow crawlers to index our site. # # List of all web robots: http://www.robotstxt.org/wc/active/html/index.html # # Check robots.txt at: # http://www.searchengineworld.com/cgi-bin/robotcheck.cgi # # Details about Googlebot available at: http://www.google.com/bot.html User-agent: Googlebot-Image Disallow: * # All other robots will be restricted from accessing the Google-specific index pages User-agent: NPBot Disallow: / User-Agent: sitecheck.internetseer.com Disallow: / User-agent: TurnitinBot Disallow: / User-agent: WebZIP Disallow: / User-agent: WebTrends Disallow: / User-agent: Yahoo-NewsCrawler Disallow: / User-Agent: msnbot Crawl-Delay: 10 User-Agent: Slurp Crawl-Delay: 10 User-agent: * Disallow: /css/ Disallow: /images/ Disallow: /logs/ Disallow: /scripts/