XML Sitemap: Format, Limits and What to Leave Out

An XML sitemap is a file, usually at /sitemap.xml, that lists the URLs you want search engines to crawl and index, with an optional date for when each one last changed. It is a discovery and scheduling aid, not a ranking factor. A good XML sitemap contains only canonical, indexable pages and tells the truth about when they were updated. A bad one lists everything the CMS knows about and stamps every URL with today’s date.

This guide is for anyone who owns a site’s technical setup: marketers checking what their platform generates, and developers building one from scratch. It covers the format as defined on sitemaps.org, the size limits, index files, what Google and Bing actually do with each tag, how to submit, and what to leave out.

On this page

• What an XML sitemap does and doesn’t do
• Do you need an XML sitemap?
• The XML sitemap format, tag by tag
• Sitemap limits and sitemap index files
• How honest does lastmod need to be?
• What to leave out of your XML sitemap
• How do you submit a sitemap to Google and Bing?
• Keeping your XML sitemap honest

What an XML sitemap does and doesn’t do

An XML sitemap tells search engines which URLs exist and which have changed, so crawlers can find pages that internal links don’t surface well. Google’s sitemaps overview is careful about the limits: a sitemap “helps search engines discover URLs on your site, but it doesn’t guarantee that all the items in your sitemap will be crawled and indexed.”

It does three jobs well:

• Discovery. URLs that are deep in the architecture, recently published or thinly linked get found sooner.
• Recrawl scheduling. An accurate lastmod helps engines decide which known URLs to fetch again.
• Canonical hinting. Google treats sitemap inclusion as a weak canonical signal, so listing the preferred URL of a set nudges selection in its favor. The strong signals are covered in our guide to the canonical tag.

It does not make a thin page indexable, override a noindex, or rescue a page that robots.txt blocks.

Do you need an XML sitemap?

Most sites benefit from one, but small, well-linked sites can do without. Google says you probably need a sitemap if your site is large, if it is new with few external links, or if it has a lot of video, images or news content. You may not need one if the site has “about 500 pages or fewer”, is comprehensively linked internally, and doesn’t rely on media or news results.

In practice we add one to every site we build, for two reasons that sit outside Google’s list. Bing’s webmaster team wrote in July 2025 that sitemaps remain a foundational signal for URL coverage in AI-powered search experiences such as Copilot. And a sitemap gives Search Console and Bing Webmaster Tools a list to report indexing against, which makes problems visible.

The XML sitemap format, tag by tag

An XML sitemap is a <urlset> containing one <url> entry per page, and only <loc> is required. The sitemaps.org protocol defines the format: the file must be UTF-8 encoded, special characters such as & must be entity-escaped, and each <loc> must be under 2,048 characters.

<?xml version="1.0" encoding="UTF-8"?>
<urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">
  <url>
    <loc>https://example.com/services/roof-repair/</loc>
    <lastmod>2026-09-14</lastmod>
  </url>
</urlset>

The diagram labels each tag with what the protocol says and what the two largest engines do with it.

Scroll the diagram sideways to see all of it.

Google’s build and submit guide states that it “ignores <priority> and <changefreq> values” and asks for “fully-qualified, absolute URLs”, because it will “attempt to crawl your URLs exactly as listed”. Bing says the same about the two optional hints: they “are ignored by Bing and do not influence how your content is crawled or ranked”. If your plugin sets every page to priority 1.0 and changefreq daily, it isn’t hurting anything, but it isn’t helping either.

Google also accepts RSS, mRSS, Atom 1.0 and plain text files with one URL per line. XML is still the better default because it is the only common format that carries lastmod.

Sitemap limits and sitemap index files

A single sitemap can list up to 50,000 URLs and must be no larger than 50MB uncompressed. The sitemaps.org protocol sets the limit precisely at 52,428,800 bytes, and allows gzip compression for transfer. Past either limit, split the URLs across several files and list them in a sitemap index.

Google’s guide to large sitemaps adds two rules for index files: the referenced sitemaps must be on the same site as the index, and in the same directory or lower. The tree below shows a typical setup for a site with a few content types.

Scroll the diagram sideways to see all of it.

Splitting by content type is not required, but it pays off. Search Console reports indexing per submitted sitemap, so a products file with a low indexed share points you straight at the template with the problem. Bing’s limits are higher at the top end: its July 2025 post says a single index file can reference up to 2.5 billion URLs.

How honest does lastmod need to be?

lastmod should change only when the page’s main content changes, and search engines will stop trusting it if it doesn’t. When Google announced in June 2023 that the sitemaps ping endpoint was going away, it also explained how it treats lastmod: as a signal for scheduling recrawls of known URLs, provided the value “consistently match[es] reality”. Its example was pointed: if a page changed seven years ago but the sitemap says yesterday, “eventually we’re not going to believe you anymore”.

The same post defines what counts. “Last modification” means last significant modification:

• Update lastmod for: changes to the primary text, added or changed structured data, updated links.
• Don’t update it for: a changed footer, a new sidebar widget, a copyright year.
• Leave it out for pages whose date your CMS can’t know, such as a homepage or category page that only aggregates other pages. Google says that is fine.

Bing agrees and adds a format preference. Its July 2025 post calls lastmod “a key signal” for deciding what to recrawl, asks for ISO 8601 dates that include the time, and warns against setting it to the time the sitemap was generated.

The most common failure we see is a static site or headless build that regenerates every page on deploy and writes the build time into every lastmod. The fix is to take the date from the content source, such as the frontmatter or the database’s updated field, not from the file system. There is a related idea in Google’s internal systems: the 2024 leak of Google’s API documentation, first reported by SparkToro, includes a field named PerDocData.lastSignificantUpdateInfo. Google stores this attribute; the leak does not show how much it is weighted. It does show Google tracks significant updates separately from any date a site claims.

What to leave out of your XML sitemap

List only URLs you want to appear in search: canonical, indexable pages that return a 200 status. Google’s build guide says to include “the URLs in your sitemap that you want to see in Google’s search results”. Everything else is noise that dilutes the file’s usefulness as a signal.

Scroll the diagram sideways to see all of it.

A redirected or noindex URL in the sitemap sends two contradictory messages: “please index this” and “don’t”. Google’s crawl budget guidance asks sites to keep sitemaps up to date and to include all the content they want crawled, which is the positive version of the same rule. On large sites, a clean sitemap is one of the cheapest ways to focus crawling, a topic covered in our explainer on crawl budget. If you publish in several languages, the sitemap can also carry your hreflang annotations, as described in our hreflang guide.

How do you submit a sitemap to Google and Bing?

Declare it in robots.txt and submit it in each engine’s webmaster tool. Google lists three methods: the Sitemaps report in Search Console, the Search Console API, and a Sitemap: line anywhere in robots.txt.

Sitemap: https://example.com/sitemap_index.xml

Two things changed in recent years that many guides still miss:

• The ping endpoint is gone. Google deprecated sitemap pings in June 2023, saying most unauthenticated submissions led to spam, and gave six months’ notice before the endpoint stopped working. Old plugins that still ping cause no harm, but they do nothing.
• Bing reads fast and often. Bing says it fetches a sitemap immediately when it is submitted in Bing Webmaster Tools or found in robots.txt, then revisits it “at least once per day”.

For changes that can’t wait for the next sitemap read, use IndexNow. The IndexNow FAQ lists Amazon, Bing, Naver, Seznam.cz, Yandex and Yep as participating engines and allows up to 10,000 URLs per POST request. Google doesn’t participate, so IndexNow supplements a sitemap rather than replacing it. Setup is covered in our guide to IndexNow, and the reporting side in our guides to Bing Webmaster Tools and the Google Search Console guide.

After submitting, check two numbers in each tool: whether the file was read without errors, and how many of its URLs are indexed. A large gap between submitted and indexed URLs is a quality or duplication signal to investigate, not a sitemap bug.

Keeping your XML sitemap honest

A sitemap earns trust the same way a person does: by being right repeatedly. List only the pages you want found, keep lastmod tied to real content changes, split large sites into index files by type, and declare the whole thing in robots.txt. Then read the indexing numbers each tool gives you back, because they are the most direct feedback on page quality you’ll get for free.

Sitemaps are one line item in a longer list, collected in our technical SEO checklist for AI search, and they matter most when URLs move, as in a website migration. If you’d like someone to reconcile your sitemap against a full crawl and both engines’ indexing reports, that comparison is part of our AI visibility audit.