Robots.txt

A file that tells search engine crawlers which pages or files the crawler can or can't request from your site.

A file that tells search engine crawlers which pages or files the crawler can or can’t request from your site.

Take the Guesswork Out of Marketing Success
Get clear, actionable strategies delivered to your inbox. Our newsletter gives you the tools you need to grow your business with confidence.