Robots.txt Generator

Build a robots.txt file to control search engine crawling of your site.

Sitemap URL (optional)
Bot Rules
User-agent
Default
Disallow paths (one per line)
An unhandled error has occurred. Reload 🗙

Rejoining the server...

Rejoin failed... trying again in seconds.

Failed to rejoin.
Please retry or reload the page.

The session has been paused by the server.

Failed to resume the session.
Please retry or reload the page.