XML Sitemap: How to Create One & Why It Matters | SlapMyWeb
Technical SEO11 min read
XML Sitemap: How to Create One and Why It Matters
Learn how to create, validate, and submit an XML sitemap so Google finds and indexes every page. Best practices, limits, and common mistakes to avoid.
SlapMyWeb Team·
An XML sitemap is a structured file that lists the URLs on your website you want search engines to crawl and index, along with metadata like the last-modified date for each page. It lives at a path like yourdomain.com/sitemap.xml and acts as a direct feed from your server to Googlebot, Bing, and other crawlers. If your site has more than a handful of pages — and especially if you publish or update content regularly — an XML sitemap is one of the cheapest, highest-leverage technical SEO assets you can ship. This guide covers exactly what goes in one, how to create and validate it, and the mistakes that quietly stall your indexing.
What Is an XML Sitemap?
An XML sitemap is a machine-readable file that lists every URL on your website you want search engines to discover, crawl, and index. Instead of hoping the crawler stumbles onto your pages through internal links, you hand it a complete directory — with a small amount of metadata attached to each URL.
The format follows the open protocol published at sitemaps.org. Each entry can include the page URL (<loc>), the last modification date (<lastmod>), a change frequency hint (<changefreq>), and a relative priority (<priority>). Google, Bing, and Yandex all read this format natively.
A few distinctions matter:
XML sitemap vs HTML sitemap — an XML sitemap is for bots and lives at /sitemap.xml; an HTML sitemap is a human-facing navigation page. They solve different problems.
Sitemap vs sitemap index — a single sitemap lists URLs directly; a sitemap index is a parent file that points to multiple sitemaps. You need an index once you cross the per-file limits (more on those below).
Discovery aid, not a ranking lever — a sitemap helps Google find pages faster. It does not make a thin or duplicate page rank. Get that expectation straight before you start optimizing it.
If your site is small and tightly internally linked, Google may eventually find everything anyway. A sitemap just makes that discovery faster, more complete, and observable in Search Console.
Why XML Sitemaps Matter for SEO
The value of a clean sitemap goes well beyond "nice to have." Here is what it actually buys you.
Faster discovery of new and updated pages. Google's own documentation states that a sitemap helps it "learn about pages on your site that we might not otherwise discover." For blogs, e-commerce catalogs, and news sites that ship content constantly, faster discovery directly shortens the gap between publishing and ranking.
Better use of crawl budget. Google allocates a finite crawl budget to every site. A focused sitemap signals which URLs you consider worth crawling, so the bot wastes less time on faceted-navigation URLs, parameter variants, and dead archives. On large sites this can be the difference between full coverage and partial indexing. If crawl budget is a real constraint for you, pair this with crawl-budget forensics in a full technical audit.
A baseline for diagnostics. Google Search Console reports "submitted vs indexed" against your sitemap. Without one, you are flying blind on which pages are indexed, excluded, or erroring. Want to confirm a specific page is even in the index? See how to check if your site is indexed by Google.
Honest about the limits. A sitemap will not fix a noindex tag, a Disallow in robots.txt, a canonical pointing elsewhere, or low-quality content. Those signals override the sitemap every time. The sitemap is an invitation, not a command.
Run a free SlapMyWeb audit to see whether your current sitemap is valid, submitted, and free of dead URLs — alongside the rest of your technical SEO.
SEO specialist reviewing a sitemap coverage report on a desktop monitor in an office
1. Audit Which Pages Belong in the Sitemap
Before you generate anything, decide which URLs to include. Not every page belongs in your XML sitemap, and over-inclusion is one of the most common ways to dilute its value.
Include:
Indexable content pages — blog posts, product pages, service pages, landing pages
Category and tag archives only if they carry unique, useful content
Key media assets you want in image or video search (via the appropriate sitemap extensions)
Internal search-result pages and infinite-filter URLs
Admin, login, cart, and staging URLs
Non-canonical duplicates — handle those with canonical tags instead
Thin or near-empty pages
The litmus test: if you would not want someone landing on the page from a Google search, it does not belong in the sitemap. Treat the file as a curated list of your best, indexable URLs — each one returning a clean 200 status. (Sitemaps full of redirects and errors are a frequent finding; brush up on HTTP status codes for SEO if 301s and 404s are creeping in.)
2. Generate the XML Sitemap
You can write a sitemap by hand for a tiny site, but for anything past a few dozen URLs you want it generated — manually maintaining hundreds of entries is a recipe for stale data and typos.
A quick note on the optional tags: Google has publicly said it largely ignores `<changefreq>` and `<priority>` and treats <lastmod> as a signal only when the date is consistently accurate. So do not agonize over priority values — spend that effort keeping lastmod truthful.
Once you cross 50,000 URLs in a single file, split your URLs into multiple sitemaps and reference them from a sitemap index:
This pattern also makes diagnostics far easier: when Search Console flags coverage problems, a per-section sitemap tells you immediately whether the issue is in your blog, your product catalog, or your docs.
Sitemap protocol limits at a glance
Limit
Value
What to do when you hit it
Max URLs per sitemap file
50,000
Split into multiple files + a sitemap index
Max uncompressed file size
50 MB
Split files; serve gzip-compressed
Max sitemaps in an index
50,000
Nest indexes (rare)
Required tag per entry
<loc> only
lastmod, changefreq, priority are optional
3. Validate the Sitemap Format
A malformed sitemap is worse than no sitemap — it can throw errors in Search Console and cause crawlers to skip URLs. The usual culprits are invalid XML encoding, unescaped characters in URLs (& must be &), missing <loc> tags, and URLs that resolve to redirects or 404s.
Validate before you submit. At minimum, confirm:
The file is well-formed XML and uses UTF-8 encoding
Every <loc> is an absolute URL including the https:// protocol
Every listed URL returns 200 and is genuinely indexable
The file is reachable at its stated path and not blocked by robots.txt
Fix every error before submission. A clean sitemap that you trust is the foundation for trusting Search Console's coverage data later.
Two colleagues at a laptop checking sitemap XML code with the file open on screen
4. Reference the Sitemap in robots.txt
Your robots.txt should point crawlers to your sitemap. This is the discovery path for any bot that does not already know your sitemap from Search Console — including Bing and several AI crawlers.
A few rules: use the full absolute URL with protocol, capitalize the Sitemap: directive, and list your sitemap index (not every individual sitemap) if you use one. You can include multiple Sitemap: lines if you have separate roots. For the complete picture on directives and testing, see the robots.txt guide.
5. Submit to Google Search Console
Log into Google Search Console, open Sitemaps in the left sidebar, paste the path to your sitemap (e.g. sitemap.xml), and click Submit. Google validates the format and begins processing — this can take anywhere from minutes to days.
After submission, watch two things in the report:
Status — it should read "Success." "Couldn't fetch" usually means the file is unreachable or blocked.
Discovered vs indexed — a large gap means Google is finding the URLs but choosing not to index them, which points to quality, duplication, or canonical issues rather than a sitemap problem.
When that gap is wide, the fix lives upstream in your content and architecture, not in the sitemap. A complete SEO audit is the fastest way to find the real cause.
6. Submit to Bing and Enable IndexNow
Do not skip Bing — it powers Yahoo, DuckDuckGo's web results, and feeds several AI search experiences. In Bing Webmaster Tools, open Sitemaps and submit the same URL.
Bing (and Yandex) also support IndexNow, a protocol where you ping the search engine the moment a page is published or updated, prompting a near-immediate crawl. IndexNow complements your sitemap rather than replacing it: the sitemap is the standing inventory, IndexNow is the real-time nudge. As AI answer engines lean more on Bing's index, getting your URLs into it quickly is increasingly worth the setup — relevant context in getting featured in AI answers.
7. Automate Generation for Dynamic Sites
Static sitemaps go stale the moment you publish your next page. If your content changes regularly, automate generation. Most platforms handle this for you — WordPress (via its core feature or an SEO plugin) and Shopify both output sitemaps natively. For custom-built sites, generate the sitemap at build time or on request.
Here is a dynamic sitemap in a Next.js App Router project:
typescript
// app/sitemap.ts — Next.js auto-generates /sitemap.xml
import { MetadataRoute } from 'next';
export default async function sitemap(): Promise<MetadataRoute.Sitemap> {
const baseUrl = 'https://example.com';
// Fetch all published blog posts from your database
const posts = await fetchAllPublishedPosts();
const blogEntries = posts.map((post) => ({
url: `${baseUrl}/blog/${post.slug}`,
lastModified: new Date(post.updatedAt),
changeFrequency: 'weekly' as const,
priority: 0.7,
}));
return [
{ url: baseUrl, lastModified: new Date(), changeFrequency: 'daily', priority: 1.0 },
{ url: `${baseUrl}/pricing`, lastModified: new Date(), changeFrequency: 'monthly', priority: 0.8 },
...blogEntries,
];
}
Because lastModified is driven by your real updatedAt timestamps, the sitemap stays honest automatically — exactly the accuracy Google rewards with reliable recrawls.
XML Sitemap Best Practices
Respect the limits — 50,000 URLs and 50 MB uncompressed per file; split with a sitemap index past either threshold.
Keep `lastmod` truthful — update it only when the page content actually changes. Faking fresh dates to force recrawls trains Google to ignore your dates entirely.
List only canonical, 200-status URLs — no redirects, no 404s, no noindex pages, no www/non-www or trailing-slash inconsistencies.
Use absolute HTTPS URLs — every <loc> needs the full protocol and matches your canonical domain. If you are still mixed-protocol, fix that first; see HTTPS and SEO.
Gzip large sitemaps — serve sitemap.xml.gz to cut bandwidth; all major engines support it.
Use sitemap extensions where they pay off — <image:image> and <video:video> entries help media-heavy sites surface in image and video search.
Common XML Sitemap Mistakes
These errors quietly stall indexing. Check yours against each:
Including `noindex` or robots-blocked pages. This sends conflicting signals: the sitemap says "crawl me," the page says "don't index me." Pick one — and it should never be the sitemap fighting your own directives.
Listing 404s and stale redirects. Dead and redirecting URLs trigger coverage warnings and waste crawls. Prune deleted pages and update URLs after any migration.
Forgetting the robots.txt reference. A perfect sitemap that no crawler is told about is half-wasted. Add the Sitemap: directive.
Skipping a rebuild after migrations. Protocol changes, domain moves, and slug updates all require a fresh sitemap. Old URLs sitting in the file slow down indexing of the new structure.
Using relative URLs. Every <loc> must be absolute. Paths like /blog/post-1 are invalid and will be rejected.
Treating the sitemap as a ranking fix. It is a discovery tool. If pages are indexed but not ranking, the work is in on-page SEO and content quality, not the sitemap.
Web developer at a standing desk editing a Next.js sitemap file on a laptop
Where the Sitemap Fits in Your Technical SEO
A sitemap is one node in a larger crawl-and-index system. It works best alongside clean internal linking, accurate canonicals, a sensible robots.txt, and fast-loading pages. If you are building out the whole foundation, the complete technical SEO guide is the pillar to anchor on — this article is the deep dive on the sitemap piece of it.
Frequently Asked Questions
Do small websites need an XML sitemap?
Yes — even a 10-to-20-page site benefits. A sitemap guarantees Google can discover every page quickly and gives you a coverage report in Search Console to spot indexing problems you would otherwise miss. There is no real downside to having one, and the diagnostic visibility alone makes it worthwhile.
How often should I update my XML sitemap?
Update it whenever you add, remove, or substantially change a page. For dynamic sites, automate generation so the file always mirrors your live content; for static sites, regenerate on every deploy or at least monthly. The key is that the lastmod dates stay accurate, because inaccurate dates erode Google's trust in your sitemap.
Can a bad XML sitemap hurt my SEO?
It will not trigger a direct penalty, but it can waste crawl budget and delay indexing. Including noindex pages, 404s, or non-canonical duplicates sends mixed signals that slow Google's understanding of your site, and a sitemap full of errors makes your Search Console coverage data unreliable.
What is the difference between an XML sitemap and an HTML sitemap?
An XML sitemap is a machine-readable file at /sitemap.xml built for search engine crawlers, following strict formatting rules. An HTML sitemap is a user-facing page that lists your pages for human navigation. Both can be useful, but only the XML version directly affects crawl efficiency and indexing speed.
Where should my XML sitemap be located?
Place it at the root of the domain it covers — typically https://yourdomain.com/sitemap.xml — and reference it in robots.txt. A sitemap can only include URLs on the same host where it lives (cross-host sitemaps require verified ownership in Search Console), so each domain or subdomain generally needs its own.
