Recently, John Mueller of Google advised an SEO not to create a sitemap for a large website manually. He offered more appropriate solutions.
The exchange happened on a Reddit thread on the r/SEO subreddit. An SEO wanted some advice on sitemap creation for a website that offered over 2 million products.
The thread states:
“I have a client in a very technical, industrial niche. They sell industrial parts that are similar, but very specific. They have 2 Million + SKUs in their inventory.
My challenge here is to create a sitemap. Because there are so many pages, I have to create it manually. I have a list of URLs broken up by products and categories that our dev team pulled through API. Since sitemaps can only contain 50,000 URLs, I have 37 excel sheets with 50,000 URLs on them each…”
The SEO described how they planned to go about tackling this challenge, which is all irrelevant since they don’t “have to” create the sitemap manually as they originally thought.
John Mueller responded by saying, “Don’t create a sitemap manually like that.” Rather than manually creating the website, Meuller says this is what should be done:
“You need to get [the sitemap] from the CMS or the underlying database directly.”
Another Reddit user pointed out that getting a sitemap from the API where the dev team originally pulled the list of URLs would be a good solution.
When creating a sitemap for a large website, you need to consider how it will be created but how it will be kept up to date. If you have a manually created sitemap, it could really mess with your site if the site is this large of a catalog, especially if you’re always adding and removing tens of thousands of products every day.
Manually creating a sitemap isn’t a realistic solution when you could use automation options.
Fortunately, the SEO who asked the question in the Reddit thread came to their senses and agreed that manually creating a sitemap is not the right way to go about it.
“Thanks for the wake-up call guys. I agree, this was the plan of action laid out for us and it didn’t sit right with me from the beginning.”