WordPress Not Being Indexed

Over the past week we have noticed that WordPress blogs (including this one) have not been indexed by Google. After investigation we decided to deactivate all of our plugins, upgrade the blog and ensure that it was being pinged out to all of the search engines and it seems to have worked!

The steps we took were:

Backup the blog files
Backup the database using WordPress Database Backup
Disable all plugins through the admin
Follow the instructions to upgrade WordPress
Ensure that the blog is secure using WP Security Scan
Enable only essential plugins

The only problem now is that we’re not using Feedburner and several of our other plugins are still disabled. Several people have commented that it may be Feedburner that is causing the blog not to be indexed so we have now installed Joost de Valk’s RSS Footer plugin which links back to the blog and have re-enabled Feedburner.

If you have any more suggestions or have had similar problems then please let us know by leaving a comment.

September 30, 2008

Posted by Sam Rutley to PushON Work

To find out how PushON research & analysis services can give your company a competitive online edge, simply call 0844 967 0565 or contact us via our form.

http://www.goodwebpractices.com David Towers

Interesting that you guys have had a problem with your WordPress blogs… I haven’t heard of anyone else with problems but will post back here if I do. Thanks providing the tip off!
PushON

Thanks David. Our blog posts usually made it to the google index in minutes but we noticed that other sites that were scraping the RSS feed for content were getting credited instead of us. Hopefully the we’ll be getting the credit again from now on.
http://www.goodwebpractices.com David Towers

Ah ok so it was other sites scrapping content – yeah that sucks. Maybe you should consider not publishing your whole articles in the RSS feeds. Only publishing say the first 300 words ensures you get more users clicking on to your site too.

I have problems with people scrapping content too, but I embed a in the article so Google sees where the content comes from.
Ben

I came across your blog while looking for a solution to this problem. I have quite a few blogs and most of the ones I care about haven’t had the posts indexed since mid September. The blogs are still in the index but no new posts are being indexed. I think your theory about a plugin causing the problem looks like it could be true. All my experiments have produced contradictory results.

I have just disabled all the plug-ins but thinking most likely candidates, are All-in-one SEO and XML sitemaps. Would you be able to tell me whether you had any of these and whether you re-enabled them? I do not have feedsmith on this site but I have it on another site which is not suffering from this lack of indexing at all. Thats on a WP 2.5 install though.

I am suspecting a problem with my hosting company. My main site stopped getting indexed for about 2 -3 weeks from the middle of September. Maybe google changed something that only affects certain configurations of webservers with certain configurations of plug-ins. Your disabling of the plug-ins and upgrades could have coincided with a server update which fixed the problem. I had a problem with my web host before blocking googlebot.

Have you got any more theories on this? Can you tell me what versions of WP your upgraded from to please?
http://www.pushon.co.uk Sam Rutley

Hi Ben,

The issue with our blogs seems to have been resolved, we are still unsure as to whether this was due to the upgrade and cleaning of plugins or something to do with Google.

We upgraded from 2.6 to 2.6.2 and disabled all plugins except:
- Akismet
- All in One SEO Pack
- FD Feedburner Plugin
- Google XML Sitemaps
- RSS Footer
- User Photo
- WordPress Database Backup

.. ensuring they are up-to-date.

So yes, all in one and XML sitemaps were both activated before and after the cleaning process.

We’ve discounted bad neighbors theory (see: http://www.myipneighbors.com/) as nothing has changed on our server.

It’s a weird one and all I can suggest is to just make sure everything is up-to-date and non-essential plugins are deactivated.

Please keep us updated!
Ben

Still no joy. I downgraded one of my installs to wordpress 2.5.1 in case it was the WP version but no luck.

I have just updated the All in one seo and XML sitemaps and re-enabled them.

I don’t think that bad neighbourhood theory is true in general. I have about 250 websites hosted on my IP (very cheap and cheerful shared hosting). I can’t see how google can penalise you for sharing hosting with bad neighbourhoods as its so common.

Dave’s suggestion about content scrapers doesn’t seem right. That looks like a symptom rather than the cause.

All other pages seem to index fine using my own CMS. Its something WP specific. I just can’t work out what. The common thing for all mine is the host. Themes, versions and plug ins seem to work on other hosts.

I think I am going to set up a new domain (on a new host), use that for publishing and scrape the content from it for my main blog. As main blog isn’t being indexed there’ll be no duplicate content penalty! Hopefully WP 2.7 will fix this problem.

Have you got any recommendations for a different CMS? Using wordpress for everything is putting all my eggs in 1 basket. I have a Drupal site but too powerful and took a long time to set up. I am thinking maybe movable type.
http://www.pushon.co.uk Sam Rutley

Hi Ben,

We’re actually in the process of evaluating Drupal and Joomla, I’m sure there will be a blog post in a week or so about which one we think is best!

I agree it must either be something specific to your setup as all of our blogs seem to be fine now or it could also be Google changing their algorithms- it is very odd though.

Have you checked Google Webmaster Tools for errors/problems?
Ben

Hi Sam,

Web master tools says nothing. I am going to set up an A-record and host the blog on a sub-domain on a different web host. I want to keep all the trust rank I have. Hopefully that will fix the problem.

I evaluated Joomla and Druapl. I decided on Drupal. The range of modules looked better and theming the sites was easier. This module is class http://drupal.org/project/seo_checklist For my purposes they are both too bloated and WP is so much better for what I do. The closest thing to WP I see is movable type, so I might give that a blast.
Ben

Moved my server (same host). Blog is indexed now but only in the supplementary bit. Makes no sense at all to me. Would have been better off leaving it.

Going with Plan B which is to build another few sites on another host and try get back up in the serps.
http://www.virtuaffinity.com Simon Wharton

Ben
I’m pretty certain bad neighbourhoods have influence. If you are going to try a difierent server, try a different host as well.
Ben

Hi Simon

It depends on how you define bad neighbourhood. I was always under the impression that it meant you linked to a bad site. Merely being hosted on the same site is different (from what I understand). Fair play to Google if you link to a load of spammy sites and get penalised but just sharing a host is harsh. How can I be held accountable for that?

Its just the wordress part of the site which is not being indexed. The rest of the site is fine. Its something wordpress specific. Guess its Drupal for my next site then.
Ben

Built my new site using CMS made simple http://www.cmsmadesimple.org/ Its as simple as wordpress but more powerful. Not quite Joomla or Drupal but it does all I need it to do. You should check it out. An added bonus is that it is a recursive acronym.

Just need to get it into the index now. Any tips on how to get indexed quick?

PushON Ltd

WordPress Not Being Indexed

Get Social With Us:

Legal:

Recommended Reading:

Address:

Email:

Phone: