################################################################################################################################################################## # # # Welcome to ZOOMCREATOR.COM! # # Robots.txt File created 2008/3/30 # # # # Beware! Bad Bots will be sent else where!! # # Any unauthorized bot running will result in IP's being banned. Agent spoofing is considered a bot - if it looks like a bot and not from an SE - it is a bot. # # Honey pots are - and have been - running. If your access has been blocked for bot running - please private message me at http://developer.zoomcreator.com # # with a reinclusion request. # # # # # list of known bot user agents robotstxt.org/db.html # ################################################################################################################################################################## ##Some Handy Hints for your robots text file ## Keeping Your Images From Being Indexed User-agent: vscooter Disallow: / User-agent: DittoSpyder Disallow: / User-agent: Googlebot-Image Disallow: / User-agent: psbot Disallow: / ##favicon disallow yahoo User-agent: Slurp Disallow: /favicon.ico ##favicon disallow iconsurf.com User-agent: IconSurf Disallow: /favicon.ico #Stop intrusive spy survey bot #User-agent: SurveyBot #Disallow: / ##Several major crawlers support a Crawl-delay parameter, set to the number of seconds to wait between successive requests to the same server: ##User-agent: * ##Crawl-delay: 10 ##Example ## ask jeeves crawler ##User-agent: Teoma ##Crawl-Delay: 10 ## Preventing your pages being cached from the Internet Archive's spider alexa.com archive.org ##Example ##User-agent: ia_archiver ##Disallow: / ##Some major crawlers support an Allow directive which can counteract a previous Disallow directive.This is useful when you disallow an entire directory but still want some documents in that directory crawled and indexed. ##Example ##User-agent: Googlebot ##Disallow: /folder1/ ##Allow: /folder1/myfile.html ##Sitemap: http ://www.example.com/sitemap.xml.gz User-agent: * Disallow: /webstatsbak/ Disallow: /downloads_new/ Disallow: /jobs/ Disallow: /trap/ Disallow: /test/ Disallow: /survey/ Disallow: /secure/ Disallow: /poll/ Disallow: /newsite/ Disallow: /mobile/ Disallow: /language/ Disallow: /javascript/ Disallow: /images/ Disallow: /frame_cookie/ Disallow: /forum/ Disallow: /faq/ Disallow: /errors/ Disallow: /email/ Disallow: /data/ Disallow: /css/ Disallow: /copyright/ Disallow: /cgi-bin/ Disallow: /bots/ Disallow: /awstats/ Disallow: /advertising/ Disallow: /accessible/ Disallow: /404/ Disallow: /etc/ Disallow: /htpasswd/ Disallow: /logs/ Disallow: /portal/ Sitemap: http://www.zoomcreator.com/sitemap.xml