Facebook Admits To More Ad Measurement Problems
- by Gavin O'Malley @mp_gavin, November 16, 2016
Testing the patience of brand partners, Facebook reported some additional ad-metric inaccuracies on Wednesday.
On one of its Pages dashboards, for instance, a summary number showing
7-day or 28-day organic page reach was miscalculated as a simple sum of daily reach instead of de-duplicating repeat visitors over those periods, Facebook admitted.
“However, the vast majority of reach data in the Page Insights dashboard … was unaffected,” the social giant insists in a new blog post.
The re-duplicated 7-day summary in the overview dashboard will be 33% lower on average and 28-day will be 55% lower, according to Facebook. The “bug” has been live since May, it said, adding: “It does not affect paid reach.”
Facebook also said it is making an “improvement” to Page organic reach to match what it has already done for paid reach. “Reach counts will now be based on viewable impressions,” the company said.
For paid ads reports, the social net is also moving to a stricter definition that only counts reach once the post enters the person's screen, i.e., “viewable impressions.” With the “stricter” definition, the company estimates that reported reach will be 20% lower on average.
The news comes about two months after Facebook copped to inflating the average time it told brand partners that users were viewing video ads. Going forward, Facebook is vowing to provide a more accurate picture of brand’s campaign performance.
That effort includes the creation of a new internal review process, and Metrics FYI -- a new blog with which the social giant promises to keep marketers abreast of ad measurement updates and changes.
Facebook is also exploring additional third-party reviews to validate its present ad reporting, the company said.
“We’re also launching the ability to verify display impression data through our third-party viewability verification partners, including Moat, IAS, and comScore,” according to a company statement.
Facebook is also partnering with Nielsen to include Facebook video and Facebook Live viewership in Nielsen's Digital Content Ratings (DCR).
This should give publishers access to third-party verification for video metrics and allow for comparable digital and TV metrics in Nielsen's Total Audience Measurement, according to Facebook.
Even before these most recent revelations, marketers were smarting over Facebook’s restatement of average video views. In particular, the Association of National Advertisers called the bloated estimates a big deal.
“The recent disclosures by Facebook that they overestimated video viewing for two years is troubling,” ANA President and CEO Bob Liodice asserted in a blog post. “While ANA recognizes that ‘mistakes do happen,’ we also recognize that Facebook has not yet achieved the level of measurement transparency that marketers need and require.”
In light of the fact that Facebook’s metrics had yet to be accredited by the Media Rating Council, Liodice called for an official audit.
“With more than $6 billion of marketers’ media being directed to Facebook, we believe that it is time for them … to be audited and accredited,” Liodice asserted.
Not all industry leaders have been so critical of Facebook, however. At an Advertising Week event, Interpublic Group CEO Michael Roth said the matter was “not that big a deal.” That’s because average time viewed is “not a key measurement for buying on Facebook,” Roth said.
Jeff Hinz, managing partner and digital director Mediacom, agreed. “From a position perspective with my clients, we don’t buy on those standards from Facebook … so it doesn’t affect us,” Hinz said during a recent panel discussion at OMMA Video.
“However, what it does speak to is … transparency and accountability,” Hinz noted. It’s a “true issue,” he said.
Ok, I get to say "I told you so" on this...seriously. MediaPost was running stories from Rakuten saying that all the third party software was wrong, just two weeks ago! http://www.mediapost.com/publications/article/288023/facebook-sued-over-inflated-video-ad-metrics.html
What forced them to finally admit it? An "internal audit"? Really? I think there's more to this story, and I think the inflation is FAR GREATER than they're admitting.
I should add: I would like to see what their audit of "link clicks" and "all clicks" leads to - in our experience, the metrics FB is giving us looks completely crazy. I think that they're counting anyone who uses their fingers on the ad to scroll past it as an engagement, so essentially their entire mobile user metrics are BS.
Why any advertiser would accept data supplied by the digital ad seller as "proof of performance" is beyond me. In "legacy media" such practices were banned almost at the outset ---- many decades ago ---and independent as well as fully audited or accredited surveys have always taken precedence.
Some things in this article just don't seem to feel right or pass "The Man On The Clapham Omnibus" test.
The error is blamed on summing the seven (or 28) individual daily reach figures.
First, it is terrifying that someone thought that was the right way of calculating reach, but even more terrifying is that the data was publicly released.
Second, reducing the 7-day reach by 33% means that two-thirds of people (i.e. the large majority) only use Facebook once a week. Put another twice as many people use Facebook on just one day a week, than use it more than once a week. That simply doesn't gel with the Facebook observations I have made. I know a statistician should never use the old sample of n=1 trick (I hope the Research Society is not reading this), but does ANYONE think that is right - I'd love to hear from them.
Third, looking at the 28-day data, the reach drops to 45% of what it was, so in round terms, just less that half the active Facebook population use it one day a month. Does that also feel right?
Fourth, what is the 'reach threshold'? Is it like the video viewing threshold of 1-second? If so, this could have major repercussion for media planners and buyers. If the 28-day reach is only half what was originally thought, and that could have been as low as just one second in the 28-days, then what is the chance my ad would be seen within my campaign - very slim.
Fifth, how come it is only wrong in Organic Reach, but miraculously isn't in Paid Reach? Why do they pass through different reach calculation algorithms? That is an amatuer-hour error.
John & Ed:
Terrific analysis of the specious numbers as usual. Acceptance of data from any ad seller without the most thorough evaluation of the methodology is indeed irresponsible of the agencies Mr. Roth.
While Facebook is a very sophisticated Tech Company, it is also a Publisher (whatever Zuckerberg believes - Bob Garfield nailed this!) as well as a Research Company. In other words a complex media & marketing chameleon. Bob Liodice is absolutely correct! All Facebook's metrics and methodologies should be MRC audited. However this raises an on-going fundamental MRC issue. When is MRC going to be sufficiently funded and consequently staffed to take on accreditations of this immense scope in a timely manner? We can be sure many more are coming.