Lee Odden

Getting Spidered With Your Pants Down

Lee Odden     Online Marketing, SEO

Here’s an example of what happens when your site or blog gets indexed by a search engine when things aren’t working quite right. A search on Yahoo for “SEO Blog” shows Google’s Matt Cutts at number one. Great, but take a look at the title and snippet of text.

The same search on Google shows Matt’s blog at number one as well, with the title and snippet as it should be:

After comparing search results between Yahoo and Google, I wish Yahoo would add the date a site was last indexed and I wish Google would add the “Also try” search phrases at the top like Yahoo does.

Would I be posting this if it wasn’t Matt Cutts’ blog? Maybe, maybe not. It is a good example though, of what can happen if your site has hosting issues or if your web site/blog software is having issues and a search engine spider comes to visit.

PoorSo SoOKGoodAwesome (No Ratings Yet)

Lee Odden About Lee Odden

@LeeOdden is the CEO of TopRank Marketing and editor of Online Marketing Blog. Cited for his expertise by The Economist, Forbes and the Wall Street Journal, he's the author of the book Optimize and presents internationally on B2B marketing topics including content, search, social media and influencer marketing. When not at conferences, consulting, or working with his talented team, he's likely running, traveling or cooking up something new.


  1. Avatar Thomas McMahon says

    FYI: I think Matt’s title its taken from DMOZ and if his site throws an error, Google won’t show it in the search results.

  2. Avatar Aaron Pratt says

    I have spoken with Matt about his blog bugging out, he is in need of upgrading WordPress which he says he will get around to sometime.

    He also recently used the NOODP header tag to show how it works to his minions, interesting, yahoo?

    BUT search engines pretty much repair these issues pretty quick, one thing I do not like is when you have 3 domains on the same IP with the main one down, it shows the addon domains subfolders in the search. This happens in Google and it is real ugly for a day or two depending on page rank and crawl speed.

  3. Hey Aaron, I doubt Matt relies in any way, on search engine referrals to his blog. But I thought it interesting all the same.

    Are you 301 redirecting those multiple domains?

  4. Avatar Aaron Pratt says

    Nah I just always buy my own IP’s (to avoid being on the same shared IP with spam sites) and Godaddy allows one main domain and two addons. Not sure how they do it but is is via something server side I do not have access to and have not takin’ the time to understand yet.

    I do 301 the non “www” to “www” which can lead sites with few backlinks astray via canonical issues if not added to .htaccesss of header.


  1. Starked SF » Blog Archive » Talk of the town: Friday July 28 says:

    […] Getting Spidered with Your Pants Down. […]