Google Webmaster Tools: Monitoring Indexation

PushON | July 25th 2012

Google have announced a new feature in Webmaster Tools, which allows people to monitor the indexation of a website in more detail.

Found under the Health option within the Webmaster Tools navigation menu, the new feature allows you to see both a basic and an advanced graph, visualising search engine indexation over time. The advanced report shows total number of indexed pages, total number of pages crawled, “not selected”, and “blocked by robots.txt” – which are all quite useful to see when reviewing a website.

Indexation Analysis in Google Webmaster Tools

While this is a good step in the right direction in terms of seeing which pages are within Google’s index, there’s still a number of improvements that could be made.

Here’s some improvements I’d like to see rolled out:

1. Full Analysis

Having an “indexed total” is quite basic; there should be the ability to drill-down into the individual pages which are indexed, and subsequently those which are not. It would be good to see which folders in particular may not be indexed, as it would help webmasters to quickly de-bug issues (i.e. perhaps a section of the site is blocked via robots.txt).

2. Export Data

Having a graph with some numbers is fine, but to analyse the data properly, webmasters should be able to export all of the data.

3. On-the-fly Indexation Checking

Within Webmaster Tools, it would be useful to have a simple tool which allowed you to enter a single URL, or a batch of URLs (i.e. via an XML sitemap) to quickly get a view of which ones are indexed, and which ones are not. There are tools which help to do this, but it has always seemed like something Google should offer as part of their Webmaster Tools package.

Another Issue

One other issue I’ve spotted with this new feature of Webmaster Tools, is that you cannot see this indexation information (graphs, etc) on a profile in which you are monitoring a sub-directory on a domain – it is only available for root domain monitoring.

Subfolder Indexation unavailable via Google Webmaster Tools

This is particularly frustrating for large websites in which the profiles have been split out.

All-in-all, it’s a nice (basic) new feature of Google Webmaster Tools, but it has some way to go before it becomes entirely useful.