TechHui

Hawaiʻi's Technology and New Media Community

NYT on Big Data Being an Economic Dud

My friend Ken Berkun recently sent me a link to a NYT article that was more amusing than anything I've seen in The Onion. Mr. Glanz inquires, "Is Big Data an Economic Big Dud?" I'm not sure where to start with this article.

Mr. Glanz: There is just one tiny problem: the economy is, at best, in the doldrums and has stayed there during the latest surge in Web traffic. The rate of productivity growth, whose steady rise from the 1970s well into the 2000s has been credited to earlier phases in the computer and Internet revolutions, has actually fallen. The overall economic trends are complex, but an argument could be made that the slowdown began around 2005 — just when Big Data began to make its appearance.

So...you are looking for an economic impact on the day Big Data arrived? I agree that Big Data had zero impact zero days after it made its "big appearance." What you are missing is that highly sophisticated NLP (natural language processing) techniques and classification engines coupled with a massive increase in computing power only recently enabled us to do significantly useful things with this volume of data in an automated manner. 90% of data transmitted over the internet has been produced in the past two years. On a daily basis we now produce about 2.5 quintillion bytes of data. Why is this relevant? Because the vast majority of data we have was collected very recently, and only in the past few years could we query and classify it with a high degree of accuracy.

In some cases, online companies like Amazon and eBay are fighting among themselves for customers. But in others — here is where the cannibals enter — the companies are eating up traditional advertising, media, music and retailing businesses, said Joel Waldfogel, an economist at the University of Minnesota who has studied the phenomenon.

You mean the "guess how many impressions this print ad actually had" and the "I think 50% of my advertising is working but I'm not sure which 50%" traditional advertising? I think its OK to cannibalize that with interactive, trackable advertising targeted and analyzed by Big Data systems.

Joel Waldfogel (cited professor): “One falls, one rises — it’s pretty clear the digital kind is a substitute to the physical kind,” he said. “So it would be crazy to count the whole rise in digital as a net addition to the economy.”

No one who has ever worked in advertising would agree with this. The ability to target, measure, analyze and optimize online advertising has been a huge net positive. It took an industry built on assumptions and fuzzy measurements and made it evidence based. The ability to target with rich contextual knowledge, achieve conversion and measure results with interactive advertising is vastly greater than traditional media. How do you think Google built a $180B advertising business? To imply online advertising is simply a replacement for traditional advertising is...well...an adjective escapes me. Lets just say it brought about a radical disruption, reinvention and massive expansion of the industry.

Some economists argue that it is often difficult to estimate the true value of new technologies, and that Big Data may already be delivering benefits that are uncounted in official economic statistics. Cat videos and television programs on Hulu, for example, produce pleasure for Web surfers — so shouldn’t economists find a way to value such intangible activity, whether or not it moves the needle of the gross domestic product?

I don't follow the relevance of cat videos, but the value of Big Data analytics is by its very nature much easier to quantify than the impact of traditional advertising. The data is recorded, organized and analyzed. The amount of context and information available for each consumer is much greater.

Organizations ranging from the largest VCs and banks to intelligence agencies and healthcare companies are banking on Big Data having a very significant impact on everything from marketing and product design to managing public health and reducing crime. They are pouring billions of dollars into Big Data analytics companies. A quick browse around CrunchBase will paint the picture. Startups like Palantir have quietly taken in hundreds of millions in funding each. There are dozens of other similarly funded companies working on Big Data solutions for everything from tracking the evolution of pandemics to helping companies make huge gains in sales by doing better ad targeting, up-selling and cross-selling. Tell companies like eBay, Amazon, Apple and the travel giants that Big Data isn't having a large economic impact. They will laugh you out of the room. At a previous employer I hired people from the personalization, CEM and VoC analytics groups at those companies. I assure you, people who make those types of salaries are generating revenue for their employers. They know the question of whether or not Big Data analytics works (when done correctly) has been answered.

And then there is Walmart. They have increased revenue by an astonishing $1B with their Big Data based inventory control, ad targeting, cross-selling and up-selling. There have been very visible missteps, but have no doubt, this is the future of Big Data. Consider the existing revenue boosts the aforementioned companies are already achieving, the savings governments can gain by understanding the evolution of pandemics using Google searches, and the thousand other applications of Big Data analytics that will be enabled by tomorrow's NLP and classification engines.

Views: 251

Tags: big data, data analytics, data mining, data science

Comment

You need to be a member of TechHui to add comments!

Join TechHui

Comment by Dan Starr on August 29, 2013 at 7:39am
I agree it is an exciting time to be entering the big data and analytics space.  Just understanding how to leverage your data by asking the right questions, augmenting it with often free data resources, and performing advanced analytics can allow new companies to gain an edge in long standing business verticals.
I'm also seeing a lot more public and commercially available datasets that supersede resources traditionally provided by state and local governments.  Working in the analytics space, most companies I encounter have no problem finding these datasets, and I would be interested in knowing what data is hard to come by and which only the Hawaii State government can provide.  
I see a lot of potential in Hawaii for companies that wish to tackle hyperlocal or highly personalized consumer / tourist / service industry problems.  I feel solutions provided by startups like uber, taskrabbit, stowthat, homejoy, etc... are very applicable to Hawaii's economy, but miss truly solving the hyperlocal problem due to poor modeling of and customization for customers and service providers.  These are problems which may be solved with better use of data (eg: reputation, modeling preferences/aptitudes, scheduling).
Comment by Brian on August 26, 2013 at 8:41pm
Free download and free public cloud to play on. Highly recommend you look at it.
Comment by Daniel Leuck on August 24, 2013 at 6:58am

@Brian - I've read about Tableau but I haven't played with their software.

Comment by Brian on August 23, 2013 at 8:53pm
Sorta unrelated, but have you played with tableau?
Comment by Daniel Leuck on August 22, 2013 at 2:02pm

Thank you for the feedback and the book recommendation. Who Owns the Future? sounds like an interesting read.

Its good Hawaii finally has a CIO, but the battle isn't over. Sonny and his team need better funding and plenty of support to overcome the state's enormous IT issues.

Comment by David B. Fisher on August 22, 2013 at 12:18pm

Good review and analysis of a subject we should be thinking, talking and doing more about.  

My particular path in exploring this world started with the very first TED talk that I saw in '09 with Hans Rosling visualizing world health data over time , and shortly after Tim Berners-Lee on Linked Data   Unfortunately back then, there was not much interest (we even have a Semantic Web group here on Tech Hui that has not gained any traction).  Of course that has changed, at least in the government sector with the hiring of Sonny Bhagowalia to lead the State's Office of Info Management and Technology. and the Open Data law that just passed although not adequately financed.  

I am working on learning more about how we might use these approaches and resources to develop local businesses and would love to get to know others who are also working on this.  It feels to me a lot like the early 90's when we were just learning about the WWW and websites.

BTW, another interesting public policy critique with more constructive suggestions for where we could go is Jaron Lanier's "Who Owns The Future?".

Sponsors

web design, web development, localization

© 2014   Created by Daniel Leuck.

Badges  |  Report an Issue  |  Terms of Service