TechHui

Hawaii's Tech Community

Daniel Leuck

Google Crunches 20 Petabytes a Day

According to a recently published whitepaper, Google is currently processing 20 petabytes of data every day. I find it amazing that their search engine and applications like Google Docs still have nearly instantaneous response times.

3 Comments

Clifton Royston Comment by Clifton Royston on 11 January 2008 at 10:32am
Interesting - this paper first came out a year or two ago - it was originally written for OSDI 2004 - and I've been pushing it ever since I stumbled on it. (You can also find it at http://labs.google.com/papers/mapreduce.html.)

Anybody involved in computer software - and I mean anybody at all - should carefully read this whitepaper, which should help understand how Google gets to where they are.
Daniel Leuck Comment by Daniel Leuck on 11 January 2008 at 11:39am
> Interesting - this paper first came out a year or two
> ago

I read the original version as well along with their whitepaper on GFS with great interest. I was noting the updated figures in this version which runs through September 2007. The volumn of jobs has increased 13X since March 2006.

> Anybody involved in computer software - and I
> mean anybody at all - should carefully read this
> whitepaper, which should help understand how
> Google gets to where they are.

Agreed! It is a fascinating read.
Cameron Souza Comment by Cameron Souza on 12 January 2008 at 11:57pm
Google is buying up land for data centers all over middle America and other areas where land and power are cheap and tax incentives are proffered. Apparently each new data center costs about $600M to open and employs approximately 200 people. To keep up with Google Microsoft is building $1B dollar data centers, and yet they appear to be far behind in terms of overall processing power.

Add a Comment

You need to be a member of TechHui to add comments!

Join this network

RSS

© 2008   Created by Daniel Leuck

Report an Issue  |  Feedback  |  Privacy  |  Terms of Service