Sign in Subscribe
Display

CMA reveals significant revenue decline in Privacy Sandbox testing results

Competition authority publishes comprehensive analysis showing up to 60% publisher revenue drop.

Luis Rijo

• 7 min read
CMA
CMA

The UK's Competition and Markets Authority on 13 June 2025 published test results demonstrating that publishers could face substantial revenue declines even with Privacy Sandbox tools replacing third-party cookies. According to the CMA, "publisher revenue on each impression was around 30% lower without TPCs even accounting for the availability of the Privacy Sandbox at the time of testing."

PPC Land Newsletter

Get the PPC Land newsletter ✉️ for more like this

Subscribe

The testing framework evaluated three main Privacy Sandbox APIs between November 2023 and June 2024: the Protected Audience API (PA API), Attribution Reporting API (ARA), and Topics API. Google initially planned to phase out third-party cookies and replace them with these privacy-preserving alternatives before abandoning the plan on 22 April 2025.

Summary

Who: The UK's Competition and Markets Authority conducted oversight of Google and 24 third-party companies testing Privacy Sandbox APIs, with participants including ad tech platforms, publishers, and advertisers across the digital advertising ecosystem.

What: Comprehensive quantitative testing of Privacy Sandbox tools (Protected Audience API, Attribution Reporting API, and Topics API) revealed significant performance limitations, with publisher revenue declining around 30% even with privacy-preserving alternatives to third-party cookies enabled.

When: Testing occurred between November 2023 and June 2024, with results published by the CMA on 13 June 2025, following Google's decision on 22 April 2025 to abandon third-party cookie deprecation plans entirely.

Where: Testing took place across Chrome browser traffic representing approximately 10% of users, with experimental infrastructure provided by Google and testing conducted by companies primarily in the UK, US, and European markets.

Why: The testing aimed to evaluate whether Privacy Sandbox tools could adequately replace third-party cookie functionality without creating competitive disadvantages, as part of regulatory oversight of Google's original plans to phase out cross-site tracking technologies in Chrome.

Testing reveals mixed performance across advertising sectors

The CMA worked with 25 businesses, including Google and 24 third parties, to conduct quantitative assessments of Privacy Sandbox effectiveness. Testing used an A/B framework comparing outcomes with and without the APIs across similar traffic volumes. Google provided infrastructure within Chrome to enable testing across approximately 10% of traffic.

According to the testing results, Privacy Sandbox tools provided only partial mitigation of third-party cookie restrictions. The CMA found that "the Privacy Sandbox tools would mitigate some of the impacts of restricting TPCs. However, the test results show that even with the Privacy Sandbox tools as available at that moment in time, outcomes such as publisher revenue would remain materially below those seen on traffic with TPCs."

Google's own testing showed advertiser spend falling 14% on its DV360 platform and 11% on Google Ads when using Privacy Sandbox tools instead of third-party cookies. Publisher revenue per impression declined 27% on Google Ad Manager and 15% on AdSense platforms. The testing demonstrated that remarketing campaigns, which rely heavily on cross-site tracking, experienced particularly severe impacts.

Third-party testers report more significant challenges

Independent testing by third-party companies revealed more substantial performance degradations than Google's internal results. According to the CMA, ten third-party testers measuring revenue per impression "all saw decreases when using PA API in place of TPCs." The median tester observed 30% reduced publisher revenues, with results ranging between 20% and 50% reductions.

Third-party advertiser spend declined by magnitudes of 42%, 60%, and 67% across different testers, more severe than Google's 14% DV360 estimate. Click-through rates fell by 12%, 24%, 30%, and 67% respectively across four third-party testers. The CMA noted that "retargeting-focussed businesses appeared to experience greater impacts, though some of these businesses also identified steps to mitigate these impacts."

Latency emerged as a significant technical challenge during testing. Third-party experiments found latency increases of 19%, 115% to 152%, and around 200% respectively when using Privacy Sandbox APIs. Google did not provide latency comparisons in its testing report.

Recovery rates demonstrate limited effectiveness

The testing measured how effectively Privacy Sandbox tools could recover value lost from restricting third-party cookies. According to the CMA analysis, third-party testers found that Privacy Sandbox tools "did not close more than half of the degradation in aggregate outcome metrics (such as overall publisher revenue) created by restricting TPCs."

One academic study found PA API recovery rates of 46% and 43% based on clicks and click-through conversions respectively. Another tester highlighted recovery rates of only 17% based on spend, though this figure included advertising inventory not yet enabled for Privacy Sandbox testing.

Publishers using alternative identifiers alongside Privacy Sandbox tools saw improved outcomes. One publisher comparing treatment groups with and without alternative identifiers found a median 9 percentage point improvement in revenue when alternative identifiers were enabled.

Attribution measurement faces substantial degradation

The Attribution Reporting API, designed to measure conversion events without third-party cookies, showed significant measurement challenges during testing. Google's open display testing confirmed that "ad tech and ad inventory suppliers would have a reduced ability to measure conversions using ARA compared to using TPCs."

For click-through conversion campaigns, over two-thirds experienced Absolute Percentage Error rates above 20% for two of three Google products tested. View-through conversion measurements faced even greater degradation, with over 95% of campaigns experiencing error rates above 20%.

Testing revealed that campaigns more dependent on third-party cookies for measurement experienced particularly high mismeasurement rates. According to the CMA, "for campaigns most dependent on TPCs, the vast majority of campaigns experienced very high error rates (above 60%)."

Testing methodology faces multiple validity challenges

The CMA identified several factors that could have influenced testing results beyond the Privacy Sandbox APIs themselves. These included imbalanced participation across experimental groups, sample attrition from technical issues, and displacement effects where advertisers shifted budgets between test groups.

According to the testing analysis, "experimental traffic interacted with non-experimental traffic, in a way that is unrepresentative of how advertising will work outside the experimental setting." Some testers implemented strategies to control for these effects, though these approaches introduced their own potential biases.

The CMA placed varying weight on different experiments based on how effectively they addressed methodological limitations. Testing results from Criteo and academic researchers received higher weighting due to more robust experimental designs that minimized sample attrition and other biases.

Google's owned properties show differential impacts

Testing revealed variations in how Privacy Sandbox restrictions would affect different parts of Google's advertising business. The CMA found "some evidence that Google's owned and operated advertising businesses would be less affected by these changes than its open display businesses."

YouTube experienced more severe impacts than Google Search, with revenue effects falling within ranges observed for open display advertising. Google's Search advertising showed the smallest revenue impacts among tested properties. The CMA analysis suggested this reflected different levels of third-party cookie dependence across Google's advertising platforms.

Market share effects during testing showed Google's supply-side platforms gaining impressions when third-party cookies were restricted. According to the CMA, "GAM realised greater impressions of around 9% on traffic without TPCs but with the Privacy Sandbox tools, relative to that with TPCs."

Topics API demonstrates limited utility

Testing of the Topics API, designed for interest-based advertising without cross-site tracking, revealed minimal effectiveness across multiple testers. According to the CMA findings, seven buy-side testers provided evidence on Topics utility, with four testing outcomes compared to third-party cookies and finding "lower performance."

Three testers evaluating whether Topics augmented Privacy Sandbox performance or contributed value absent third-party cookies concluded that "Topics contributed minimal improvements in outcomes." The limited utility appeared linked to Topics being present on only a minority of advertising requests during the testing period.

Publishers testing Topics compared to contextual targeting found revenue per impression increases of around 4% when Topics was present, though this evidence carried limitations due to potential selection effects in which requests included Topics signals.

Industry context shapes testing interpretation

The CMA acknowledged that testing occurred during a period when Privacy Sandbox adoption remained limited across the advertising ecosystem. According to the analysis, "only a limited section of the ad tech and ad inventory ecosystem engaged with the testing – and many testers suggested that greater uptake of the Privacy Sandbox tools as TPC restrictions come into force would improve outcomes."

Testing participants operated within constraints including limited Privacy Sandbox data for training bidding algorithms and incomplete ecosystem participation. The CMA noted that Google had already implemented some Privacy Sandbox improvements after testing concluded, with potential for further enhancements over time.

The regulatory context shifted significantly after testing completion. Google announced on 22 April 2025 that it would no longer implement third-party cookie restrictions, maintaining the current approach to cookie support in Chrome. This means Privacy Sandbox tools will function alongside rather than replace third-party cookies.

Regulatory implications and next steps

The CMA testing results contributed to its assessment of Google's Privacy Sandbox commitments, originally designed to ensure the tools didn't disadvantage competitors. With Google abandoning third-party cookie deprecation, the CMA published a Notice of Intention to Release Commitments on 13 June 2025.

According to the notice, "the CMA believes the commitments are no longer necessary and is now consulting before it takes a decision on whether to release them later this year." The regulatory framework had been designed for a scenario where Privacy Sandbox tools replaced third-party cookies rather than complementing them.

The CMA's assessment of Privacy Sandbox found ongoing competition concerns despite Google's strategy revision. Industry testing revealed significant technical and performance challenges that likely influenced Google's decision to maintain third-party cookie support.

The testing evidence provides valuable insights for marketers and publishers preparing for potential future privacy initiatives. While Privacy Sandbox tools demonstrated some effectiveness in preserving advertising outcomes, the results highlighted substantial gaps compared to current third-party cookie functionality.

Timeline

Stories like this, in your inbox

Enter your email
Subscribe