Sitemaps and Instant Change Notices for Search Engines

Search engines discover pages by following links and by reading sitemaps. A sitemap is a file that lists a website's public pages, often with when each changed and which images it contains. For a site with a few pages, links may be enough. For a site with hundreds of blog posts, products and events, a good sitemap makes a real difference in how quickly and completely pages are found.

Our platform generates sitemaps automatically for every website. Here is how they work and the rules behind them.

What our sitemaps include

A website's sitemap is organized into sections for different kinds of content:

• Pages built with the visual builder, including the home page
• Blog posts, with their featured images, plus blog categories and tags
• Events that are public and published
• Products that are active and public in the shop

Each entry uses the site's preferred address, so if a site has its own domain, the sitemap lists that domain rather than the platform's address. When a site moves to a new domain, the sitemap follows automatically.

What our sitemaps leave out

A sitemap should list only what the owner wants found. Ours skip:

• Pages marked "do not index"
• Pages whose canonical address points somewhere else
• Drafts, scheduled posts and anything not yet published
• Private, members-only and unlisted content
• Search result pages, which search engines recommend keeping out of their indexes

Listing a page in a sitemap while telling search engines not to index it sends mixed signals, so we keep them consistent.

Images help discovery

For blog posts and pages, sitemaps can include their main images with titles. That helps image search understand which images belong to which pages. Owners can turn image entries off if they prefer, and the setting is part of each site's search settings.

Freshness without constant rebuilding

Generating a large sitemap on every request would be slow, so sitemaps are cached for a period the owner can adjust. When content is published or changed, the relevant cache is cleared, so the next request reflects the change. That keeps sitemaps both fast and current.

Instant change notices

Even with a sitemap, search engines decide for themselves when to check it again. Some search engines accept instant notices through a protocol called IndexNow: when a page is published or updated, the site sends its address so those engines can check it sooner. IndexNow reaches several search engines, including Bing, which also supports Microsoft Copilot answers. Google does not use IndexNow; site owners use Google Search Console instead.

Our platform supports IndexNow as an option in each site's search settings. Owners choose which kinds of content send notices, such as pages, blog posts, events, products and live channels. A private key is created for the site when the feature is turned on.

Timing matters during a move

There is a good reason not to send notices too early. When a site is being rebuilt before moving to its permanent domain, sending its temporary addresses to search engines can create records that will need to be redirected later. We generally turn instant notices on once a site is at its permanent address, while the sitemap itself is always kept accurate.

Search Console and verification

Sitemaps and notices work best alongside the search engines' own tools. Google Search Console and Bing Webmaster Tools let a site owner confirm they own a site, submit its sitemap, see which pages are indexed and learn about crawling problems. Our search settings include fields for the verification codes those tools provide, so owners can prove ownership without editing code. Verification is something only the owner can complete, because it requires access to their own accounts with those services.

A sitemap reflects the whole site

Because the sitemap is built from the platform's own records, it stays complete as a site grows. A new blog post appears in it when it is published. A product appears when it becomes active and public. An event appears when it is published. Owners do not need to remember to update a file, and nothing is left out because someone forgot. That completeness is especially valuable for larger content libraries, such as a blog with hundreds of articles. You can read about how we keep old addresses working when content moves in keeping old links alive with redirect maps.

Robots.txt points the way

Every site's robots.txt file, which tells crawlers what they may visit, points to the sitemap. It also carries any rules the owner set for AI crawlers, which we describe in what we do for AI search visibility. Robots rules and sitemaps work together: one says where crawlers may go, the other says what is worth finding.

Frequently asked questions

Does every website need a sitemap?

Small sites can be found through links alone, but sitemaps help larger sites and new sites get discovered more completely and quickly.

Does Google use IndexNow?

No. IndexNow is used by Bing and several other search engines. Google discovers changes through crawling, sitemaps and Search Console.

Why is a page missing from my sitemap?

It may be marked not to be indexed, point to a different canonical address, be unpublished or be private. Check the page's search settings.