SEO Monday – Understanding Spiderability and PageRank

By Nik


Google’s spiders have it rough – they’re the little guys that do all the work that help Google the giant put together such a comprehensive index of content, and yet they are one of the most feared concepts by SEOs.

Why do so many SEO’s shy away from answering questions about the spiderability of a website? Well, for starters, it’s tough to be right… and yes, these little guys are misunderstood.

I have tips to share with you today, and they’re not coming from a Google manual or other blog posts that I’ve read, but rather my empirical experience in managing the SEO programs of dozens of sites; sites that range in size from mom and pops to super-shops. I’m going to focus on a game-plan for an e-commerce business owner. These are just tips, but should help you gauge the potential hang-ups that exist on your site and what you can do to take advantage of how Google works.

Ok – so my chemistry teacher back in high school, Mr. Shannon, used to stress to me the importance of understanding the basics; that those who made the greatest discoveries in chemistry where the same folks who dedicated a part of the day to re-studying the basics of chemistry. I think it’s the same with all sciences, like SEO.

So what are the basics? Well, spiderability is discussed in this presentation and it refers to how easily a Google spider can find your page. Healthy spiderability typically signifies to Google that somebody finds the page important. This is amplified in its affect as a page with healthy spiderability then points to another page – it is essentially passing along its visibility to the spiders. It’s a lot like networking: a page that is well connected then passes on its connections to another page.

PageRank then quantifies this concept. You can find the Google PageRank of any page on the internet through the Google toolbar or other tools like SEOQuake, which is a handy plug-in for Firefox. Basically, a high PageRank indicates that Google is finding it easy to locate your page: that other well connected pages are recommending your page.

So now that you understand the basics, I’m going to list some common mistakes that I see when it comes to harnessing the power of spiderability. Each one has a lot of merit to why it’s a mistake and you could very easily be hurting from it. So here we go:

  1. Using a sink to fill a lake is the scenario in which a website really does not have a ton of strength in terms of spiderability, acts like it does. These are your PageRank 1 and 2 homepage sites that use their navigation to point to 200 pages on the site, which are then supposed to continue the link-flow into deeper pages. If a page doesn’t have a high metric of spiderability, it doesn’t carry with it the type of clout that a high PageRank page would. In other words, it can only contribute so much towards the pages it points to. If you’re finding that your product pages are not being indexed – you may be in the this boat. Why product pages? Because homepages typically point to category pages and if those pages are spreading thin the worth of the homepage, then there’s very little left to pass along to the product pages.
  2. Long distance travel exasperates this effect. If it takes 2-3 clicks for a user to find a product page from the homepage, then the spiders are suffering equally. If you value product pages and want them to be indexed by Google, make sure they are easy for Google to find. Of course, the higher the PageRank of a homepage, the more liberty you have to create deeper hierarchical sections of your site.
  3. Two-to-four front doors is when a site has multiple homepages. It happens all the time. A site will have the root as a homepage, and also an index.html page, both living independently of one another. This often gets worse when these two pages also live at both the www version of a site and the non-www version of a site. That’s 4 versions of a homepage. If they were all forwarded to one location, then the homepage of your site would be as strong as possible, and then support its internal pages the right way.
  4. The hermit is a site that has a major lack of in-bound links. A site that is not spiderable from other sites will essentially not be found by Google’s spiders. These are the sites will the lowest PageRanks. To be on the map, you must be somewhere on the road often traveled by Google’s spiders.

We help our clients identify their spiderability issues in a project we call the Advanced SEO Foundation Project. Basically, we’re helping lay down the foundation for the SEO success of your website. If you’re interested in learning more or would like an SEO audit to determine your strengths and weaknesses in terms of spiderability, give us a call at 800-504-4324.

Thanks folks and have a wonderful SEO Monday.