Commentary
Who Will Win In Emerging Big-Data World?
- by Dave Morgan , Featured Contributor, February 7, 2013
It seems like everyone in our business today is talking about Big Data, at conferences and in articles in the trades, including this one. Big data even pops up in every sort of job description for new roles in media, marketing and advertising. And it is even invading earnings calls for holding companies.
Why has Big Data become so important in our industry? The explosion and invasion of connected computing devices in all aspects of consumer and business fields over the past 10 years -- from home computers to smartphones, from store checkout scanners to those laser-like theater or Amtrak ticket readers -- has created a volume of consumer and business “data exhaust” that exponentially overwhelms any notions of data we’ve dealt with before.
Our potential to capture, synthesize and act on data -- even from disparate, disconnected sources -- is now virtually limitless. We can no longer compare the size of databases to multiples of the amount of information in the Library of Congress. That reference point is too small to even matter anymore.
advertisement
advertisement
Meanwhile, the cost and complexity of capturing, storing and synthesizing data has been significantly reduced. Six years ago it cost $10-$15 million and took nine months to build and configure a robust 30-terabyte data-mart with systems from Oracle and IBM. Today, you can build a robust 300-terabyte data-mart with $500,000 of off-the-rack Dell server storage arrays running free, open-source software (Hadoop, MySQL, PostreSQL) -- or you can use a credit card for cloud-based data warehouses from companies like MongoDB.
Big Data elsewhere means rapid progress in the accuracy of weather forecasting or tracking the spread of communicable diseases. But for marketers, Big Data means that you can analyze a marketing activity like the delivery of banner ad impressions or TV spots and determine “relatable” purchases made by tens of millions of exposed and non-exposed consumers with a level of precision that dramatically outstrips any marketing metric we’ve ever seen before. You can begin to see how every piece of marketing material or advertising affects upper-funnel activity like consideration, along with actual purchase and customer loyalty.
Big Data is not just revolutionizing data management and data mining -- it’s also democratizing it. If your data infrastructure investments are already about six years old, you are most likely operating at a 300-1 cost disadvantage against competitors with a modern Big Data "systems." But you no longer need to be Walmart to have the world’s biggest and best retail transaction database.
How will this trend affect those of us working in media and marketing? At the very least, it means that more and more of what we do every day will need to be empirically justified, either before, during, or after we recommend or take an action -- or, most probably, at all three points. If you are not armed with data, you can be sure that someone else in your organization will be, particularly those who want either your job, your budget or your industry standing.
Marketing data analysts will become both more valuable and less in control. First, those who are good with making data relevant for business decisions will do well. But those who have relied on being the only data-enabled executive in the room will now find lots of competition. Big Data democratization means more competition for those who used to rule the data analysis “roost” in organizations.
Who within marketing and media companies will win in the emerging Big Data world? I believe that the spoils will go to those who can move past basic data analysis and convert it into predictable business outcomes. Big Data means that we’re no longer limited to looking in the rear-view mirror. The future will reward those who can convert data into foresight and take bold, market-changing actions. What do you think?
Such an important article for communication professionals who need to know that the data they are working with is current, accurate and easy to get. But I found that working with big data does not have to be complex, time consuming, or expensive. Check out a company called Chiliad and its product Discovery/Alert 7.0 --- www.chiliad.com and its Iterative Discovery research method where they do not require you to have a data scientist on board to grab the data you need. The user asks questions using conversational queries and instantly explores relevant concepts with a mouse click. Discovery/Alert 7.0 supports real time, ad hoc queries across multiple data sources, including structured and unstructured data, without special preparation. And the other key thing is that their Virtual Consolidation technology means analysis across all your data locations with no consolidation expense. Sorry if I sound like an advertisement. Just excited to see the that this process does not have to be expensive or complex and can work within a company's existing workflow systems.
"You bought the wrong candy bar with your mobile devise even after there was a coupon for our brand. Since we have all your data, your next purchase will be charging you more. We are in control. Your complaints will fall into the black hole of complaints since our parent company has so much invested with your credit card/finance company in which they cannot afford to split."1984
Cathy,
You sound like an ad because Chiliad is your PR firm's client.
It would be nice to disclose the relationship in your gushing remarks about their most amazing spectacular solution.
Peter
good article Dave :-)
Dave, not at all my area of expertise, but in re:
Six years ago it cost $10-$15 million and took nine months to build and configure a robust 30-terabyte data-mart with systems from Oracle and IBM. Today, you can build a robust 300-terabyte data-mart with $500,000 of off-the-rack Dell server storage arrays running free, open-source software (Hadoop, MySQL, PostreSQL) -- or you can use a credit card for cloud-based data warehouses from companies like MongoDB.
I'd like to ask just how many company IT shops, especially in big corps, have shown any willingness to give up on the existing infrastructure they know and love and spent years introducing and FINALLY getting to work (mostly) OK, are willing to make the move you suggest. My experience with IT tells me that the average CIO would pnly say, Over my dead body.
Kenneth, you raise a key issue. Most corporate IT folks, particularly those in leadership roles, have almost no familiarity or comfort with new, open source data management technologies like Hadoop. Thus, their companies are in a tough spot if they wan't to keep pace with the market and start-ups nipping at their heels.
Data: it has red skin, white seeds and an acidic pulp. Information: it's a tomato. Insight: it is actually a fruit. Wisdom: don't use it in fruit salad. --- From the point-of-view of the agronomist, the data is the really interesting thing about the tomato. But, the rest of us are more interested in the insight and the wisdom. In the advertising and marketing industry, we have a large number of brilliant people who see data like the agronomist. They think the information is an insight. I don't yet see big data yielding wisdom. But, I'd settle for big data being used to empower senior marketing executives so that they can layer on some wisdom instead of using "big" data to overwhelm marketers. For every leading edge member of the digital industry who tells me that I now know exactly how my media mix is working to move individual consumers through the funnel, I have a senior marketing executive who is feeling overwhelmed instead of empowered.