My friend Ken Berkun recently sent me a link to a NYT article that was more amusing than anything I've seen in The Onion. Mr. Glanz inquires, "Is Big Data an Economic Big Dud?" I'm not sure where to start with this article.
Mr. Glanz: There is just one tiny problem: the economy is, at best, in the doldrums and has stayed there during the latest surge in Web traffic. The rate of productivity growth, whose steady rise from the 1970s well into the 2000s has been credited to earlier phases in the computer and Internet revolutions, has actually fallen. The overall economic trends are complex, but an argument could be made that the slowdown began around 2005 — just when Big Data began to make its appearance.
So...you are looking for an economic impact on the day Big Data arrived? I agree that Big Data had zero impact zero days after it made its "big appearance." What you are missing is that highly sophisticated NLP (natural language processing) techniques and classification engines coupled with a massive increase in computing power only recently enabled us to do significantly useful things with this volume of data in an automated manner. 90% of data transmitted over the internet has been produced in the past two years. On a daily basis we now produce about 2.5 quintillion bytes of data. Why is this relevant? Because the vast majority of data we have was collected very recently, and only in the past few years could we query and classify it with a high degree of accuracy.
In some cases, online companies like Amazon and eBay are fighting among themselves for customers. But in others — here is where the cannibals enter — the companies are eating up traditional advertising, media, music and retailing businesses, said Joel Waldfogel, an economist at the University of Minnesota who has studied the phenomenon.
You mean the "guess how many impressions this print ad actually had" and the "I think 50% of my advertising is working but I'm not sure which 50%" traditional advertising? I think its OK to cannibalize that with interactive, trackable advertising targeted and analyzed by Big Data systems.
Joel Waldfogel (cited professor): “One falls, one rises — it’s pretty clear the digital kind is a substitute to the physical kind,” he said. “So it would be crazy to count the whole rise in digital as a net addition to the economy.”
No one who has ever worked in advertising would agree with this. The ability to target, measure, analyze and optimize online advertising has been a huge net positive. It took an industry built on assumptions and fuzzy measurements and made it evidence based. The ability to target with rich contextual knowledge, achieve conversion and measure results with interactive advertising is vastly greater than traditional media. How do you think Google built a $180B advertising business? To imply online advertising is simply a replacement for traditional advertising is...well...an adjective escapes me. Lets just say it brought about a radical disruption, reinvention and massive expansion of the industry.
Some economists argue that it is often difficult to estimate the true value of new technologies, and that Big Data may already be delivering benefits that are uncounted in official economic statistics. Cat videos and television programs on Hulu, for example, produce pleasure for Web surfers — so shouldn’t economists find a way to value such intangible activity, whether or not it moves the needle of the gross domestic product?
I don't follow the relevance of cat videos, but the value of Big Data analytics is by its very nature much easier to quantify than the impact of traditional advertising. The data is recorded, organized and analyzed. The amount of context and information available for each consumer is much greater.
Organizations ranging from the largest VCs and banks to intelligence agencies and healthcare companies are banking on Big Data having a very significant impact on everything from marketing and product design to managing public health and reducing crime. They are pouring billions of dollars into Big Data analytics companies. A quick browse around CrunchBase will paint the picture. Startups like Palantir have quietly taken in hundreds of millions in funding each. There are dozens of other similarly funded companies working on Big Data solutions for everything from tracking the evolution of pandemics to helping companies make huge gains in sales by doing better ad targeting, up-selling and cross-selling. Tell companies like eBay, Amazon, Apple and the travel giants that Big Data isn't having a large economic impact. They will laugh you out of the room. At a previous employer I hired people from the personalization, CEM and VoC analytics groups at those companies. I assure you, people who make those types of salaries are generating revenue for their employers. They know the question of whether or not Big Data analytics works (when done correctly) has been answered.
And then there is Walmart. They have increased revenue by an astonishing $1B with their Big Data based inventory control, ad targeting, cross-selling and up-selling. There have been very visible missteps, but have no doubt, this is the future of Big Data. Consider the existing revenue boosts the aforementioned companies are already achieving, the savings governments can gain by understanding the evolution of pandemics using Google searches, and the thousand other applications of Big Data analytics that will be enabled by tomorrow's NLP and classification engines.