If you know what a Sitemap is then that’s great, but for those that don’t – well, there are two types of Sitemap for a website, an XML Sitemap and a HTML Sitemap.
Sitemaps allow search-bots to quickly crawl/read, understand the structure and context of where “things live” and the full Information Architecture of your site. You usually will have an XML Sitemap at the root of your site (if you don’t, I cover off later how to get one and add it), but you can also have an additional HTML Sitemap, which I explain more about and the benefits of creating/having one.
A Sitemap in either format is beneficial for SEO – now I know there has been debate over debate for years in the SEO industry, about how beneficial the HTML sitemaps actually are, but for me, its can be a small amount of work that is beneficial for SEO, I do recommend them for our client SEO work, as it can give you the marginal gains to your SEO strategy.
What’s an XML Sitemap?
An XML Sitemap is plain text and stored at the root of the domain – usually https://www.domain.com/sitemap.xml – its a lightweight file which allows search engine crawlers to access quickly and understand any content, page, folder structure on the site… it also checks if there is new content on the site and the priority in which it should crawl the URLs. The output file, also has a frequency range added to each URL, which in all honesty will not have any effect on rankings – its just the format in which the URLs are read.
An XML Sitemap is a really good way for a search engine crawler to crawl your site quickly and easily to understand its structure and of course all the URLs. For example, if you have large site, in excess of say 2000 pages/URLs, then it would take a long time for a search engine to crawl all of your URLs in one go. It would more than likely dip into your site, crawl a bunch of URLs and then dip back out, onto its next website to crawl – this is called Crawl Budget (which I will get to in a different post), and then it would come back at another point in the future to crawl more pages. An XML Sitemap is a quick and easy way to signal and inform search engines about your site and its structure.
To check if you have an XML Sitemap, simply put your website address in the address bar of your browser and then /sitemap.xml after it.
What’s a HTML Sitemap?
A HTML Sitemap is a web page that is built (via HTML obviously) and is more beneficial for users coming to the site to find pages that they may not be able to find through your navigational points – although in this day and age, users are more likely to use your search function to find products/services. If you don’t have a search function, then a HTML Sitemap can really help. HTML Sitemap will include a link to every single page on your site – its a great way to organise, categorise pages on your site for someone to navigate to. When clients ask for an example (and there are many more out there), I seemingly always point them to https://www.boots.com/site-map as an example. You can see how and where each category is built out, with sub folders, pages and so on for Boots.com.
Benefits of Taking Time to Build a Great HTML Sitemap
Every client site I worked on, and sometimes due to my nerdy inner SEO self, I end up checking my favourite ecommerce sites for, always have an XML Sitemap – very few have a HTML version, so I will explain why I think its beneficial to include a HTML Sitemap.
Helping Search Engine Crawlers Understand
Ideally, you want to help a search engine crawler to understand your site, its structure as easily and quickly as possible. I sometimes think of the crawlers as being a bit “rude” – you kind of need to help, hand hold and educate them as much as you can about your site, or they will leap straight out of your site and on to the next site to crawl… so helping them to find your content is a way to keep them on site. Whereas an XML Sitemap is a long list of URLs, search engine crawlers do prefer to read links on your site to find out where content “lives” and how its structured – so having a HTML Sitemap highlights the important pages that you want discovered and crawled. You want to help those search engines out in any way you can.
Target Keywords That Are Important to Your Products and Services
As mentioned above, if you have a HTML Sitemap with links to pages and content on your site, then you can also include some nice juicy relevant keywords (from your SEO Keyword list) to target, include them in your HTML Sitemap, with a link to the page you want a search engine to rank you for. This is all about sending signals and the importance of internal linking structure for your site. As you can see from the Boots.com example, they have their keywords in their for each category of their site and individual page level. This is where marginal gains for SEO can come into play, it won’t sky rocket you to position 1, but it will aide in your overall SEO Strategy.
Organising Larger or Growing Websites
There used to be a time (way back when circa 2008-10 God that makes me feel old), when the more content and pages you had, the better your site may perform organic search – but those were cowboy days (with some black, grey and white hat tactics) and it just isn’t like that anymore, its about more than 200 different elements to get your site to rank well and bring in organic traffic via search engines.
There are a huge list of very large websites with 1000s of pages on them (most are media sites), but even your own site will grow over time as you add more and more content to it. You may have an ecommerce site now, which over the next few years could branch out into more and more categories and products – so having a HTML Sitemap, can help to organise those pages for a user and search engine alike. Its also beneficial for you to understand how and where products, categories should be positioned and everything should be in the right place. Think of it as a long list of products on a page in a catalogue or directory – for someone trying to find what they need.
It Should Improve and Increase Search Engine Visibility
As its been mentioned further up the article, search engines like reading and following internal links on your site – they want to understand where the link points to, what the keyword is that you are using and then form an understanding of the structure, content and context of your site and pages. Where search engines may not index every single page on your site, it helps to have a HTML Sitemap with a nice juicy, keyword rich link to the categories and pages for the search engine to “read” and understand. The search engine may well choose to read and follow a link from your HTML Sitemap to get the understanding of the navigation and structure of your site. The taxonomy you build in your HTML Sitemap will help navigate both users and search engines.
As with the Boots.com HTML Sitemap – there isn’t a limit to how big it is, you can have literally 1000s of links on it to categories, pages, content on your site.
The HTML Sitemap is a Part of Your Internal Linking Strategy
Your main navigational points on your website will link through to the most important parts of your site – I like to call them the “money pages”. Where commercially, you naturally add a link to important pages you want people to visit and help them with their journey to those pages. A HTML Sitemap can help link pages together internally, and help both the user and search engine where those pages “live”. In-turn, this will help drive your website visibility when it comes to search engines discovering what your site is and offers. A HTML Sitemap becomes the backbone to make sure all pages can be discovered and none will become “orphaned” (pages which usually have no internal links to them at all, and can end up becoming very low value – if they ain’t linked to by you, then a search engine will think you don’t think value them, so why should they).
Optimising Your Site Navigation
Websites naturally grow over time, whether that’s more products or services that you offer, or when developing more and more blog content – and sometimes where you once had a navigation that worked, it may now need to be expanded or isn’t fit for purpose.
We had this issue with a client last year, so we ran some live user testing – we asked the users to find categories/products via the navigation and if they were stuck, to then use the HTML Sitemap. We tracked the testers through video feeds and heatmaps and found that the users would quickly hone into areas that they thought (and remember, its how users think and use your site, not how YOU think it should work) had the information for them and their task. It was great to share this with the client during our feedback sessions, and allowed us to develop the navigation for users and put the most important information into a wider and expanded Site Navigation for them.
The results were more pages in sessions for users and a much clearer, easier to follow customer journey – especially when moving users through the research phase into the purchase phase and for those returning to the site too and an increase in conversions £££s. So having a HTML Sitemap gave us a big benefit, not only those SEO marginal gains we’ve mentioned, but actually to improve the site architecture and needs of the users.
Check if you site has an XML Sitemap – type in your website address in a browser address bar and then /sitemap.xml – you should see a long plain text looking file with all your URLs on there (sometimes, some plugins change the format of the name of the file, so it could be sitemap_pages.xml). WordPress sites and plugins like Yoast offer these automatically.
If you don’t have Sitemap.xml file, then one can be easily extracted for your site – using platforms/software like SiteBulb, Screaming Frog (free version upto 500 URLs) and so on, where you can crawl a site and extract a sitemap within no time and then add that file to the root of your site. If you want a hand creating one then get in contact with us and we can sort, run a Technical SEO Audit of your site and make sure its in place.
If you don’t have an HTML sitemap but do use a platform like WordPress, then there are quite a few good sitemap plugins that you can use to add one to your site – although they do automate the HTML file, personally, I prefer to develop one by hand and keep it upto date (https://www.digitalthrive.co.uk/site-map/).
For those sites that are larger then I use SiteBulb or Screaming Frog – they are paid versions, but I think the cost of them is outweighed by the benefit of having one.
As I have mentioned earlier, search engine crawlers love links on your site to its pages, so make sure to add a footer link to your site to link to the HTML Sitemap. After developing the HTML sitemap, don’t forget to put a link on your website that is easy to find.
So, now you know more about the different types of Sitemaps for your site, how they can benefit and help both search engine crawlers and users alike. If you need any help with your SEO efforts, or a stand alone Technical SEO Audit, to better understand how a search engine crawler “sees” and crawls your site, then feel free to get in contact with us today.