SEO Experts; your robots revealed!

We've just caught Google, Yahoo and Bing's spiders in the FindMeByIP.com web... and the results are quite intriguing...

This article was posted on 1st April.
Don't take it too seriously!

When Google started crawling CSS files some years back there was a lot of noise amongst SEO experts about how much rendering was going on in their data-centres.  But without working at Google ( or Yahoo or Microsoft ) it's very difficult for any of us to tell quite what's going on... until now.

FindMeByIP.com

fmbip.com is a service set up for the web design community to help understand what CSS3 & HTML5 capabilities their clients browsers have.  Basically you send your client to a page and we send you a table showing you exactly what browser they're using and what CSS3 & HTML5 capabilities their browser supports.

"Googlebot", "MSNBot" & "Yahoo! Slurp"

Like many of us it seems that Google, Yahoo and Bing have all been along to take a look at findmebyip.com and I was surprised to find this morning that we've caught their spiders submitting their own results for what CSS3 and HTML5 capabilities those spiders support.

Fingerprinting a Spider

While Googlebot appears to have broad support for HTML5 & CSS3 I was surprised to find that it's not quite at the level of Chrome 4.  Microsoft Bing's MSNBot, even more remarkably, has quite good support for HTML5 & CSS3 where their IE browser, well, doesn't! 

So we ran a fingerprint comparison of the Googlebot, Bing and Yahoo results against our full set of tens of thousands of "in-the-wild" results and found a startling result: it looks very much like all of their web spiders are running Firefox!!. 

  • Googlebot 2.1 : Firefox 3.6
  • MSNBot 1.1 : Firefox 3.6
  • Yahoo! Slurp : Firefox 3.0
Sharing the Wealth

We like to share here at Deep Blue Sky so we thought we'd make the information available to everyone.  So for all you SEO experts who need to know exactly what these bots are capable of here's a table of the three biggest 'bots and their capabilities:

 

Grow your business

Find out how Deep Blue Sky can grow your business.

  1. Digital benchmark
  2. Digital roadmap
  3. Digital engineering

Write a comment.

Responses. (1)

  1. C L

    C

    but when?

    What's the date of these findings?