# Information for robots how to crawl the web
User-agent: *
Crawl-delay: 5

# For development and test server un-comment following line in order to stop crawling test sites
Disallow: /