Create clean, search-engine friendly robots.txt files for your website instantly.
A robots.txt file tells search engine crawlers which pages or folders they can or can't request from your site. Use the form below to customize parameters for individual search bots.
Global Configuration
Search Engine Crawlers Configuration
Restricted Directories (Disallow Paths)
The path is relative to root and must contain a trailing slash "/" (e.g. /cgi-bin/)