Finding Your Webmastertools (WMT) Crawl Errors – Wildcard Friday

By Advanced SEO Team


Want to know how you can easily use Google Webmastertools (WMT) to find your crawl errors?

My name is Rebecca and I ’m a Search Marketing Specialist here at Exclusive Concepts. Today’s post is about understanding your Google Webmaster Tools crawl errors. Back in March, Google updated the Crawl Errors page in WMT to enhance the usability and now reports on even more types of errors to aid you in having the best site possible!

Site Errors

To get to your crawl errors, log in to your Google Webmaster Tools account and click into the diagnostics on the left navigation. Clicking the crawl errors will show you a screen like the one seen here.

I’m going to stop here for a second – if you currently are a client of Exclusive Concepts and don’t have Webmaster Tools set up, contact your SEO specialist to get it up and running!

For most sites, there are no site errors. You’ll see little green check marks next to the 3 sections of the site errors, indicating that Googlebot has no problem accessing your site. To spot check, you can use Fetch as Googlebot to see if Googlebot can currently crawl your site. Typically, this is completed within 24 hours. If Fetch as Googlebot returns the content of your homepage without problems, you can assume that Googlebot is generally able to access your site properly.

URL Errors

URL errors are the more common error type seen in WMT. This list can include errors such as not found (404s), not followed (due to things like java, flash, session IDs, or redirects), access denied (password required), server errors, soft 404 (a non existent page with a friendly message on it), or Google News specific errors (not applicable for e-commerce stores). More information on these error types can be found here:

From a user perspective, 404s are the most important pages to fix. If it was once a major page that had a lot of organic traffic for your site, it’s best to redirect it to a similar page on your site. If the page was only live for a short period of time and gained no links and had little SEO value, having the page deleted is fine – just make sure you’ve removed all links to it on your site. Clicking in to the URL on the errors dashboard will bring up an Error Details screen, which also has a “Linked From” tab showing all URLs on your site linking to that broken page.

Use the filter feature to look at one error type at a time, and mark the broken URLs as fixed once you address the issue on the site – these won’t be updated until Googlebot crawls your site again.