Static SEO deployment
robots.txt, XML sitemaps and hreflang: separate SEO contracts
These files help crawlers understand a site, but none proves that a URL exists, is canonical or will be indexed. Verify each contract and its deployed response separately.
Content updated: · Maintainer and corrections
Crawl permission is not access control or noindex
robots.txt is public crawler guidance. Choose the correct user-agent group and inspect the most specific matching path; an equally specific Allow wins in the supported matching dialect. Case, percent encoding, wildcards and end anchors matter. Unsupported directives need separate documentation.
A disallowed URL can still be known to a search engine. Blocking crawling can prevent a crawler from seeing a page's noindex instruction. Protect private resources with actual authentication and authorization, not robots.txt. The offline tester cannot establish the live file's status, location or a crawler's cached copy.
A sitemap lists candidates, not verified live pages
A urlset lists page URLs; a sitemapindex lists sitemap URLs. Use the correct namespace, XML escaping and absolute addresses. A lastmod value describes a real significant modification, not the time a scheduled build happened. Duplicate or syntactically correct entries do not prove canonicalization or availability.
The protocol permits up to 50,000 entries and 50 MiB uncompressed per file. This local inspector intentionally has smaller browser-processing limits; its limit is not a new protocol rule. It does not fetch referenced indexes or pages, inspect gzip, validate every extension schema or replace Search Console.
Localized alternates need a complete, consistent set
Use supported language codes, optional script or region subtags, complete URLs and a deliberate x-default fallback. A region alone is not a language tag. For example, zh-TW and zh-Hant can describe Traditional Chinese targeting, while es-419 is not a Google-supported region code.
Every corresponding page needs its own entry and reciprocal alternates. Copying a generated mapping does not prove remote reciprocity, page language or indexability. Keep each localized page's canonical consistent with its real preferred URL; do not canonicalize all translated content to English merely because it is the default language.
Data provenance