Commentary
It's Time For Industry To Change The Way TV Is Measured
- by Dave Morgan , Featured Contributor, November 20, 2014
In a post in MediaDailyNews, Megan Clarken, Nielsen’s EVP, Global Watch Product Leadership, delineated the failures of existing measurement protocols to keep pace with consumer changes. She concluded with a call for new measurements:
“We believe that the fundamental changes occurring in today’s viewing landscape call for the industry to adopt a new set of ratings standards:
1. Total Audience, which combines the total audience for a program or content regardless of the mode of access, including SVOD.
2. Total Commercial, which includes ratings for the ad campaign regardless of where and how it’s consumed, providing flexibility for dynamic ad insertion.”
advertisement
advertisement
Nielsen executives have long talked about the fact that they are the only custodians of measurements (TV’s C3/C& Ratings) owned and controlled by the TV media industry that transacts on them. So it was quite surprising to see Nielsen step up so publicly with a call to media owners to change the ratings. However, given everything that is going on in the broader video/media business today -- from massive audience fragmentation, to the rise of subscriber online services, to mobile and tablet viewing -- it’s clear that change is needed.
Personally, I like the holistic approach of Nielsen’s proposed framework and hope that the industry takes steps to move it forward. However, I also hope that the industry goes further. As I’ve written before, I believe that it is critical for the media industry to transact not only on “media outputs,” but to also measure and deliver against specific “business outcomes.”
For media to survive and thrive in the commercial communication mix, it needs to be self-evidently valuable not only to advertisers’ heads of media, but to their chief marketing officers, CFOs, COOs, and CEOs, as well as boards and shareholders. Those folks don’t care about GRPs and sex/age demos and “viewable impressions.” They care about real business outcomes, like sales.
While Clarken is 100% correct that our industry needs to bring new, holistic, modern approaches to creating ratings and measuring commercial exposures, we also need to link those media impressions to their ultimate business impacts. I hope that we move both reach and reaction metrics forward in parallel. What do you think?
Dave, several comments on your interesting post. Regarding how TV should be measured so it provides the kind of ROI-related information you advocate, I am interested in your opinions, even on a theoretical basis, of how this might be accomplished. At present there are 116 million TV homes and something like 300 million working sets---maybe more. Would all of these be hooked up in some manner, so the effect of ad exposures---or, more properly, potential ad exposures----could be tracked home by home or set by set? And, if so, how would the advertiser know what the impact of his campaign was unless real purchase data was available from each home---or a large subset of these homes? Or would such correlations be inferred via the oft discussed merging of "big data" from third party sources---in which case you don't need to track the viewing of every home or set but can get by with a Nielsen-like panel after all. Of course, you would want a much larger TV rating pane than Nielsen now has----but whose going to pay for that? My second point is that a great deal of TV time is bought with "other" factors in mind, as well as the cause and effect relationship of ad exposures on sales. Many advertisers select TV program genres---sports, news, primetime entertainment shows on the broadcast networks, etc.--- as a matter of image shaping associations, merchandising, corporate egos, etc. which, in their minds justify premium CPMs. Obviously, it's going to be pretty hard to quantify the value of such decisions, in my opinion. My experience has been that while advertisers claim to want more refined data, especially about the ROI of their TV ad spending, in practice few of them can come up with a workable definition of ROI, nor to they practice what they preach. Until advertisers get serious and involve themselves in a truly zero-based and objective approach to media selection, one which integrates the media and "creative" functions, there is little hope for a major improvement in TV audience measurement.
Ed, thanks for the comments. While I don't think that it's realistic to expect a full census, directly measured TV world anytime soon (or ever), you can certainly capture de-identified viewing data on many millions of TV & video users and match it to all of the different kinds of first and third party data that captures advertisers' desired business outcomes - sales, web site visits, search, etc. Also, I agree, that making ROI available as part of TV media doesn't mean that all of the advertisers are going to know what to do with it or what they want from it. That will take a while for sure.
I totally agree Dave. But with one caveat. That we continue with "Average Minute Audience" as the most core and basic measure. We do not want to go down the 'count the streams' route of the interactive world. Sure that data can be utilised, but it needs to be average minute. Buyers and advertisers, not knowing when a broadcast ad will occur, rely on the average as a 'fair bet' of what audience their ad can get (if only their creative was as good as the programme). We also can't rely on 'machine-based' metrics - we can use them but not rely on them - to provide viewing data.
I'm with you John. Average minute audience is critical for true comparability, and server-based measurement has lots of issues if it's the only measure. We still need to have gold standard panel data, no matter how much direct measurement we utilize.
One more factor we have to remember is the distinction between a campaign and any given commercial exposure in the campaign. Most advertisers try to evaluate the effectiveness of their total ad campaigns as they garner awareness, build momentum and then level off or wear out. Invariably, any per-exposure measurement will show a diminishing effect over time, as regards the ad's ability to build awareness and selling power, even though the campaign, in its entirety, is performing as expected----mainly by reinforcing the convictions of those it has motivated. This being the case, I think that any new audience measurement that attempts to improve upon what we have now--- primitive demos----should focus on the media targeting aspect, more so than on ROI, as the latter is very difficult to handle on a per-exposure basis. Among the new metrics I would regard as critical, the most important, aside from product usage/interest, would be the mindset of the viewer. Is the viewer into quality, ecology, health/fitness, convenience, self image concerns, etc. or is price the primary motivator? Mindsets such as these are frequently the basis of product positioning and creative treatment ploys in TV campaigns and these cut across demos, in many cases. It would be a fairly simple matter to define the mindsets of any TV rating panel, then produce ratings for each show or channel by mindset grouping as well as product user and demo segments. MRI and Simmons both have such indicators in their studies, but these are rarely used in TV time buying as they are dated and the information for many shows is lacking.
That's a very nice idea Ed. Are you aware of anywhere it is in use. Being pragmatic, how do you see the data being collected? I assume a self-completion questionnaire at time of recruitment. As we are talking about a panel rather than a sample, the questions (well actually the panellists responses) would need temporal stability. In my experience this tends to make the questions 'less sharp. I work on another currency where we have gone down the 'slimmed down' route of doing that (using a sample and not a panel) and the questionnaire has blown out to 90 minutes already - though that is a blend of mindset and product usage. What sort of questionnaire length do you think would be needed. If it is long. I would be concerned about co-operation rates and maybe introducing propensity bias (i.e. you may end up with too many 'light' consumers). We have clients who want/demand customised product usage recency questions in return for financial co-operation - in a TV panel that data quickly becomes irrelevant if not wrong and misleading. Despite this potential misgivings ... a really nice idea!
John, I think that it would be quite easy to obtain the kinds of mindset information I am thinking of from an existing panel of cooperators, as well as an indication of their product usage for a fairly large number of categories---say once a year in a 25-35 page questionnaire. In my experience, once a person---or household---agrees to participate in a panel, the "respondents" tend to be quite cooperative as most of those who are disinclined, or too busy, were weeded out in the original recruitment process. Regarding the product use information, I believe that this could be handled in a fairly simple manner, using one or two lines per category---without trying to obtain brand by brand detail. The idea could certainly be tested to determine what the cooperation rate was for various lengths of questionnaire, types of questions, etc.
John, Ed, great stuff. I think that it would be great to back-up the declared data with lots of matched, but de-identified actual purchase behavior. Finally, there are lots of set of credit card and shopper card data available, and in near real time. Plus, lots of retailers and brands are finally working with first party purchase data and digital interaction data in a systematic way. With big sets of privacy/safe matchable set-top box data, you can now truly tie attribution off at the household impression level. Exciting times are ahead.
Dear Dave,
I have great personal and professional respect for you, especially after we met at the Memorial for Erwin Ephron that you so generously organized and underwrote in a collaborative manner for the benefit of the entire industry just one year ago.
However, I regret not seeing your commentary sooner. Could it be because MediaPost Publications filed it under "Online SPIN: Controversy Served Fresh Daily"? (Seems narrow-minded and inappropriate to me. Must your thoughts be spread like peanut butter? Surely, your words could have been positioned better!)
As I respect the Sabbath in all traditions and observe the Sabbath within my own tradition, I shall not post a full opinion. That said, I need to put a stake in the intellectual, professional and commercial common ground upon which your argument is built.
It is too easy to say that "It's Time For Industry To Change The Way TV Is Measured." First and foremost, It is time to make certain that the existing measurement of record (i.e., Nielsen NTI/NPM/GTAM/STPM) is executed in a manner that results in a methodology that is technically accurate, reliable and useful and is E&Y audited and receives unqualified MRC accreditation.
I am also reserving the right to review the comments of other respondents to your commentary.
I do not consider you to be a "spin doctor" but unfortunately the underlying premise of the framework into which your comments and the comments of respondents has been put is that of "Spin."
We know that a "spin doctor" is defined as "a person (such as a political aide) whose job involves trying to control the way something (such as an important event) is described to the public in order to influence what people think about it."
I do not want survey research science, television audience measurement or my clients business currency to be in the hands of ignorant charlatans or the unjustifiably self-impressed, self-declared prophets who espouse facile, false judgments in the interest of convenience or greed. The "illusion of technology" has done more damage than most of our professional colleagues are willing to acknowledge. En masse, I have not seen any "Big Data" that really made decision-making better or easier for the vast majority of those who depend on research and measurement?
Until I compose my assessment of the important positions you and others have taken, I appreciate your respect and patience.
Sincerely,
Nick (Nicholas P. Schiavone)
Cc MediaPost Publications
Onwards & upwards
Nick, I assure that you "spinning" is not my strategy here. I believe that TV measurement must change, and that a hybrid of a highly curated panel(s) and massive direct measurement is the answer. I also believe that while survey are helpful, matching actual ad viewership to purchase data at the household level across massive sets of consumers is the best way to know the impact that the ads had on actual business outcomes. That is the future of the TV business, I am certain of it.
Dear Dave,
I regret that you seem to have misunderstood my intention and my comments, especially where you are concerned.
The "spinning" to which I referred was the only the "spin" that MediaPost sadly seems to invite in their "OnlineSPIN" blog. There can be little ambiguity about what is welcome in this column given its title, albeit inadvertently. And if they want controversy -- not your "certainty" -- they shall have it. Let us not forget, that spin has long been the dysfunctional fashion in politics and other human pursuits.
In fact, the point of my earlier opening remakes was to express my confidence that you are not a "spinner" personally or professionally.
While I respect your sense of certainty, I think we are all mistaken to embrace unwarranted certitude, as appears to be the case here. Over a year ago, Johnathon Keats expressed this notion better than I could. He wrote "The Biggest Limitation on Human Thought: Certainty" In essence, he said, "I think that human beings are most limited in their tendency to pursue certainty and to think that answers are somehow absolute or even beneficial.
The degree to which we can go about the process of living as humans in increasingly interesting ways, in increasingly productive ways, is one in which we need to make our society operate, but once we get things working this can enslave us at a certain level. This can entrap us.
This can make us work according to other peoples' terms and they can put us on the track of trying to improve them in terms of finding better and better answers at the level of making things work more and more efficiently.
What we need is to step outside of that. We need to instead think about what those mechanisms are, what those technologies are, in which we can pursue questions that are bigger than those technologies, bigger than those routines, and therefore, lead to a more fulfilling life in which other opportunities for thinking and other opportunities for engaging our world come about every day."
At this point, it would appear that all one can say is that the future of research is open and that we need better measurement. My reading of the latest MRC audits indicates that massive direct measurement and its integration with other forms of measurement is not only far from optimal. It is far from "certain" as well.
Alas, I have written far more than I intended.
Let us give thanks, especially this week in the United States, for the gift of dialogue afforded by MediaPost Publications, even if it too is in need of enhancement like audience and advertising research.
Peace,
Nick
Nicholas P. Schiavone
Onwards & Upwards!
I'd like to add some thoughts to this discussion. Since the Media Post format does not allow for paragraphs, I'll deal with this in several posts. First, as to Nick's points, above, it is not likely that you will obtain a 100% perfect match if you try to simulate or "blend" information from an independent "Big Data" source with Nielsen's ratings. You have to allow for slippage in such a process. How much slippage? This can be looked at by taking a source that already measures both TV viewing claims on a show by show basis, as well as product usage/purchase from the same respondents. Say one takes a two-year sample from MRI and ascribes product usage/purchase data on to its viewing data using whatever simulation process seems most appropriate. Then you can compare the resulting "ascribed" indices with the actual results from the same respondents for every show and product category. Instead of theories, this kind of analysis will tell us a good deal about the potential validity of this concept, if applied to Nielsen and, equally important, how big the differences really are relative to more "primitive" indexing by demographics.
Now, I'd like to add another point or two about the ways advertisers look at "linear TV". It is obvious to any of us who study this business, that a substantial amount of ad dollars are allocated to sports, news, "specials", children's shows and others, irrespective of targeting indices. It matters not what the index of heavy beer consumption or sports car ownership is for the NFL, NBA and MLB games, and many other sports attractions ----the beer and sports car advertisers ( and many others ) are still going to buy time in and "sponsor" such fare. In like manner, it's enough for most pharmaceutical advertisers to go where the older viewers are----namely TV news. Even if one news show indexes five points higher than its rivals for some malady, the advertiser will probably buy time in most or all of these shows to attain a particular reach level. The same issue is evident in many dayparts----especially for the broadcast TV networks. My sense is that most of the time the differences between shows of a similar genre---day talk shows, game shows, courtroom shows, late night talk variety shows, etc.-----are fairly small. Even in primetime,where there are larger differences because program formats, even within genres, are more diversified, the networks aren't going to let you pick and choose what programs you want to buy time in. They are going to bundle up all of their showsand sell these at "package" prices, not show by show prices. In certain cases, the sellers may allow "favored" or very large spenders to have greater representation in a number of "desirable" programs, but only to the extent of having a few more announcements scheduled in such shows. And, inevitably, the sellers will raise their CPMs in such cases, thereby nullifying any advantage gained by improving targeting by a small amount. My basic point is that we should take a long, hard look at the realities of TV buying and selling decisions, as well as the likely results----whether it will make a significant improvement----before embracing any major change in the way TV is measured. We should also take into account how the players----buyers and sellers actually operate, their respective needs, etc. It may well be that some important changes are required. But let's start doing some homework, including an examination of actual data that is available, before we go overboard. As a final thought, I have seen a number of syndicated audience studies go belly up when they were stretched too far, too fast. In this regard, I share Nick's concerns about Nielsen's attempts to measure TV everywhere. A lot of questions arise about this ambitious plan----but, frankly, I don't see them being raised publicly----except here.