# robots.txt & Sitemap > Generate robots.txt and XML sitemaps, plus live checkers to fetch and validate either file from any URL — your starting point for the manual, examples, and tips. Source: https://www.jpkc.com/db/en/tools/robots-sitemap/ ## Build and check crawler control — in one tool Two files decide how search engines and AI systems see your website: the `robots.txt` (what may a crawler fetch?) and the `sitemap.xml` (which pages should be discovered?). [robots.txt & Sitemap](https://www.jpkc.com/tools/robots-sitemap/) helps you with both — and goes a step beyond plain generators: it **builds** the files and also **checks** existing files live from any domain. The tool bundles four functions across seven tabs: a **robots.txt generator**, a **sitemap generator**, and two **live checkers** that fetch and analyze a `robots.txt` or a `sitemap.xml` from any URL. Add to that tabs with ready-made examples and a format reference. Everything runs in the browser — no account, no installation. It's built for everyone working on a site's technical discoverability: **developers** who need a correct `robots.txt` or sitemap fast; **SEO and content people** who want to verify that search engines and AI crawlers see the right pages; and **agencies** that want to take apart an existing configuration on someone else's domain in seconds. ## The four functions at a glance ### robots.txt generator A form that assembles a complete `robots.txt` with live preview. You add as many **user-agent blocks** as you like, give each block `Allow` and `Disallow` rules, optionally set a `Crawl-delay`, and finish with a `Sitemap:` line plus an optional `Host:` directive (Yandex). Blocks can be reordered by drag and drop. An autocomplete field suggests 40-plus common bot names — from `Googlebot` and `Bingbot` to AI crawlers like `GPTBot`, `ClaudeBot`, or `PerplexityBot`. Copy the result, download it as a file, or read an existing `robots.txt` via **Load File** to edit it. ### Sitemap generator Here you build an XML sitemap per the [Sitemaps protocol](https://www.sitemaps.org/protocol.html). You set a **base URL** and add URL rows, each with a path and optional `lastmod` (date), `changefreq` (`always` to `never`), and `priority` (`0.0`–`1.0`). Reorder by drag and drop, copy or download the result — and import an existing `sitemap.xml`. ### Live checker: robots.txt Enter a domain (e.g. `example.com`) or a full URL, and the tool fetches the `robots.txt` and dissects it: a **Per-Bot Access table** shows, for 40-plus known bots, whether each is allowed or blocked and which rule applies; plus all user-agent blocks, the declared sitemaps, and an interactive test of whether a given path is reachable for a given bot. ### Live checker: sitemap The same flow for the `sitemap.xml`: the tool counts the URLs, checks coverage of `lastmod`, `changefreq`, and `priority`, warns when the spec limits are exceeded (50,000 URLs / 50 MB per file), and detects sitemap index files whose child sitemaps you can drill into directly. ## Generators local, checkers via a proxy Important for understanding and for privacy: the two **generators run entirely in your browser** — nothing leaves your machine, the working state is only stored locally. The two **checkers**, by contrast, have to fetch a foreign file, which a browser can't do directly because of CORS. So a **server-side proxy** on the JPKCom server fetches the file; the analysis then runs locally in your browser again. The checked domain therefore sees a request from the JPKCom server, not your IP address. Internal and private addresses are blocked server-side. How the proxy works and what limits apply (size, timeout, rate limit) is covered in the manual. ## Try it now **[→ Open robots.txt & Sitemap](https://www.jpkc.com/tools/robots-sitemap/)** — build a file or check a foreign domain, right in the browser, no account. In the **Examples** tab, ready-made templates (standard website, WordPress, block AI crawlers, blog, shop) load into the generators as a starting point. ## Related JPKCom tools - **[SEO & GEO Analyzer](https://www.jpkc.com/db/en/tools/seo/)** — inspects a whole URL including its own *Robots Analysis* tab; here you build the rules, there you see them in the context of the entire page. - **[Meta Tags Generator](https://www.jpkc.com/db/en/tools/meta-tags/)** — clean titles, descriptions, and Open Graph/Twitter data for the pages your sitemap lists. - **[llms.txt Generator](https://www.jpkc.com/db/en/tools/llms/)** — the AI counterpart to the sitemap: a structured content overview specifically for LLMs. --- There's more on the subpages: the **[manual](https://www.jpkc.com/db/en/tools/robots-sitemap/manual/)** with every function, option, and limit in detail, hands-on **[examples](https://www.jpkc.com/db/en/tools/robots-sitemap/examples/)**, and a collection of **[tips & tricks](https://www.jpkc.com/db/en/tools/robots-sitemap/tips/)**.