We speak to a lot of Clients who don’t realise that it is extremely important for their site navigation, (commonly referred to as internal links or information architecture) to be extremely well considered so that the right pages get indexed easily and regularly by the search engine spiders. Connected to the site architecture is the preference that no one page contains more than 100 links, this keeps the quality score assigned to each link at a respectable level and helps the spiders move through the site properly.
To start, it helps to understand how the spiders prioritise the pages, and then crawl the site.
Spiders will visit popular pages more often, popular pages are defined by the number of back-links and the site architecture should correlate with this. For example:
- Your homepage, and chosen landing pages, should be the most popular with the most back-links
- First and second level category pages should be fairly popular but containing less back-links than the homepage
- At the bottom of the priority are the deepest pages, these will be pages such as news pages, product pages, service price lists etc
The spiders will enter the site via a landing page, this doesn’t need to be the homepage, they will then follow links through each page looking to index the whole site. They don’t like being sent in circles and they don’t like feeling lost in too many links, so it’s important that your site architecture makes it as easy as possible for the spiders to do their job, whilst getting all the pages which need indexing, indexed. Ideally you want the spiders to be able to index everything within three clicks of arriving on the site, regardless if that is your homepage or your deepest category page.
XML Site Maps
XML site maps are seen as the quick fix for architecture issues, and this is what they are. They do not resolve problems in the site architecture and internal navigation, they merely hide the problems so that you are unaware of them.
In an ideal world, you would not add an XML site map until you know the website architecture is sound and secure and most importantly indexing on it’s own. Below are some basic architecture tips to get you started.
Keep Architecture Flat
You want to keep your architecture as flat and easy to navigate as possible, whilst retaining the three click rule (if a spider lands on one of your deeper pages, can they reach the other pages within three links?)
In a brand new website the following structure is a common one used with the 100 links per page being the absolute maximum you should have on each page.
At the top: Homepage with no more than 100 links per page
First Level: Categories – no more than 100 pages (each page has no more than 100 links)
Second Level: Sub-Categories – no more than 10,000 (each page has no more than 100 links)
At the bottom: Detail/Products – no more than 1,000,000 pages
Index and rankings are determined by how much authority each page has, the higher the domain authority of your site the more links you can realistically get away with including on each page. As a rough guide, if your website already holds some domain authority (DA) you can increase the links on each page as follows:
DA 7-10 = 250 links
DA 5-7 = 175 links
DA 3-5 = 125 links
DA 0-3 = 100 links
So, the smaller the number of links the spiders have to follow to index the whole site, the happier they are and the more weight each page will hold.
This is a common and useful aspect of ecommerce sites, which allows you to pick facets of a product which are important to you. For example, you could pick the category of T-Shirts, pick the colour black, and the size Medium, the results you are shown then directly correspond with what you specifically want. In essence the website has ignored anything which doesn’t contain the facets you have chosen.
Setting up faceted navigation can be tricky, and you need to keep in mind that the primary facet pages won’t rank, you want the deeper facet pages to rank as these are the one’s that will help the spiders discover all of the product pages.
When setting up faceted navigation, some of the things to keep in mind are:
You must have a unique URL for each facet level. The URL’s should be clear and not complicated and hard to follow:
Clear URL: www.tshirtdomain.co.uk/tshirts/black/medium
Unclear URL: www.tshirtdomain.co.uk/all/tshirts/all/black/all/medium
You also want to ensure that whatever route somebody takes to reach this facet level the same URL is shown so for example:
Somebody clicks on Tshirts, then Medium, then Black the URL they end up on should still be www.tshirtdomain.co.uk/tshirts/black/medium and not www.tshirtdomain.co.uk/tshirts/medium/black which would result in you creating unnecessary duplicate content issues!
Adding & Removing Facets
You should make it easy for your customers to add or remove additional facets as they see fit.
As they add facets to their search these should be displayed as follows so that any or all facets can be removed by the user:
So that they can easily choose which facets can be automatically generated from the results meta data so it is easy for you to display the number of results within that facet, for example:
Any pages which could be considered as duplicate content should be no-indexed, the spiders will still visit these pages but they won’t index them. To keep a page out of the index you want to add some code to the page as follows:
<meta name = “robots” content = “noindex”> – This will make the page no index
<link rel = “canonical” href = “domainname.co.uk/tshirts/black”> – This will take the spiders back to the correct page.
Filtering & Pagination
Another common aspect of ecommerce sites is filtering results. This is where you can choose a filter which will sort the products in a certain way, for example only showing 10 items per page (creating pagination or multiple pages), or showing lowest priced items first.
The ideal way to deal with pagination in category results is to programme the page to show all results rather than writing each page of results as page 1, page 2, etc.
Plan, Plan, and Plan Again
Don’t under-value the benefit of properly planning your website. Most of our examples have referred to ecommerce sites, but the same principal applies to brochure sites. Plan to succeed and your website will be a spider’s navigational dream and you will be rewarded with good search results and no duplicate content issues.
In summary, the number one rule for you to keep in mind when you are planning your navigation is that you want as few pages as possible to be indexed, whilst allowing for each and every product page to be indexed.