Google Fellow Jeff Dean, in a keynote talk at WSDM 2009, shared some numbers about Google’s impressive growth run from 1999 to 2009. According to Dean, while both search queries and processing power have gone up by a factor of 1000, latency has gone down from around 1000ms to 200ms. Crawler updates now take minutes compared to months in 1999.
Another significant change was the switch to holding the complete search index in memory, resulting in the use of 1000 machines to handle a single query compared to just 12 previously.
This revelation may be a bit embarrassing for Google, which has defended its ecological record in the past, claiming that a single Google query takes just 0.0003KWh of energy and that the Google datacenters are “the world’s most efficient.”
It’s quite crazy to think that with how many searches are taking place each day, the amount of effort and work it takes to generate a query from Google. A thousand computers generating an answer in under a half second is beyond impressive. It will be interesting to see how Google improves this even further over the next few years.