The Google Bigdaddy Update: What Happened?

What happens when you do everything right and Google drops all of your web site's best pages from its search results? Many webmasters from weekend tinkerers to corporate site managers were asking themselves this question recently. On February 21, 2006 the Google datacenter update known as "Bigdaddy" began killing high ranking spam sites from the index. Not a bad thing for searchers and for most webmasters. But soon after Bigdaddy began rolling out, many webmasters managing sites full of useful content for searchers couldn't find their pages listed in the Google index. In most cases, the main page still appeared, but in others, sometimes tens of thousands of pages were "supped." Supplemental results are always viewed negatively by webmasters and search engine optimizers because it indicates that Google has assigned those pages a lower crawl frequency, perhaps because the pages are viewed as duplicate or inefficient for the Googlebot spider to crawl. Pages that go into the much feared supplemental results are not known for coming back to life.

Bigaddy caused pain in the hearts of many webmasters and search engine optimizers as they sat helplessly watching other pages take their place in rankings for competitive keyword phrases that earn high revenue. This pain that could be measured in dollars caused members of the "supplemental club" (coined by someone at Webmaster World forums) to frantically begin analyzing the various Google datacenters for the slightest changes. Some tried requesting reinclusion in the Google index and received at best word that the sites that they'd worked so hard to build had not been penalized.

By March, most webmaster forums had Whats-Going-on-with-Google-Bigdaddy threads. So what happened? Enter Matt Cutts, Google Engineer and frequent poster at Webmaster World Forums. As late as November 2005, Mr. Cutts had given webmsters a heads up via his blog and later warned, explained and generously offered an opportunity to give feedback on the new Google datacenter technology. Yet, no one seemed to expect a gitch of this magnitude. Matt Cutts, known pseudonymously on Webmaster World Forums as "GoogleGuy," assured webmasters that he was aware of the problem and would check with the crawl/indexing team to resolve it stating that he did "expect these pages to come back to the main index." He asked for examples. Some waited. Some complained. In at least one case, a forum member stated that he had prepared lists of employees to lay off.

Finally, on March 11, many Google datacenters had begun bringing back to life the previously supplemental pages. As the recovery rolled out and pages started appearing correctly in the Google rankings, a chorus of celebratory thank you's to Google erupted from webmasterdom. Happy to escape another Google update, many webmasters began thinking about the direction of organic search. The big question in the back of many minds must surely be "What if this happens again?" Those who rely on organic search as their primary source of revenue might consider adapting their business strategies. Would this be a good time to start reading up on paid search? Perhaps Google wouldn't mind that idea too much.

--Emory Rowland