Robots.txt Generator

Take full control over the instructions you give web crawlers about what folders and files to crawl on your site.

Generate Robots.txt file for your website automatically and instantly, don’t code it by yourself anymore

Default - All Robots are:



Search Robots:


Google Image

Google Mobile

MSN Search


Yahoo MM

Yahoo Blogs



DMOZ Checker





MSN PicSearch

Restricted Directories:
The path is relative to root and must contain a trailing slash "/"

Sign up to be the first to know about the next updates on our SERP Simulator Meta Tool

"*" indicates required fields

Your name*

What is robot.txt?

Robots.txt is a file that can be placed in the root folder of your website to help search engines index your site more appropriately. Search engines such as Google use website crawlers, or robots that review all the content on your website. There may be parts of your website that you do not want them to crawl to include in user search results, such as admin page. You can add these pages to the file to be explicitly ignored. Robots.txt files use something called the Robots Exclusion Protocol. This website will easily generate the file for you with inputs of pages to be excluded.

You may be interested our other services.


Organic traffic is the highest quality source of visitors. Be there for those already looking for you. Provide meaningful answers to real questions of internet users.

Read more


A returning visitor is more likely to convert than a first time visitor. Reach your recent visitors, people who are almost reassured to consider your product or service.

Read more


Using the latest targeting techniques to reach your audience, LeanSearch offers GDN design and animation and targets highly qualified audiences.

Read more


Google is now performing just like a shop via its Google Shopping Ads; be there for those already looking for you. Best CTR and conversion rates.

Read more

Schedule a meeting with us now