Robots.txt Generator
Create a robots.txt file to control how search engines crawl your website.
Quick Presets
Rules
Sitemap
Generated robots.txt
Upload this file as
robots.txt to your website's root directory (e.g. https://example.com/robots.txt).About This Tool
Control how search engine crawlers access your website. A robots.txt file tells bots which pages to crawl and which to skip.
- Quick presets — allow all, block all, block AI bots, or WordPress defaults
- Multiple user-agents — set different rules for different crawlers
- Allow & Disallow — fine-grained path control per bot
- Crawl-delay — optional delay between requests
- Sitemap reference — point crawlers to your sitemap
- Download — save as a robots.txt file ready to upload
- 100% private — everything runs in your browser, no data sent anywhere