Bullshit Metrics In Retail Media

My latest skit video for the 'Basketcase' podcast from The Drum & QSIC features a brand CMO with main-character energy who is quite pleased with herself for inventing a new performance metric called purchase intent velocity: velocity of intent to purchase at the moment of consideration. Of course, there's no methodology behind it and no engagement with the retailers, but we're sure they'll figure it out before the next JBP! (If they really want that brand budget, that is).

It's a funny take on bullshit metrics in the industry. But like any joke, there's true pain and weirdness behind it.

Bullshit is in the eye of the beholder

The thing about bullshit metrics is they only look like bullshit to the other side. Many measurement folks I speak with at retailers truly believe they are providing the absolute best and most holistic type of metric within their capbility, while preserving their customer data. There's also missing pieces the retailer can't account for. For example, Albertson's incremental MTA measurement framework pulls in large swaths of 'walled garden' touchpoints from Meta and DV360, but not Google search. They recognize there's inevitable gaps, but retailers might view their efforts as getting a 96% on a test.

Brands zero in on the gaps like a type-A parent. "What about that last 4% then, son?" And the whole thing gets discounted.

From the retailer's POV, brands are asking for too much. They ask competing retailers to setup data clean rooms to isolate audiences – a scenario where a challenger retailer has much more to gain than the dominant one. They demand custom dashboards that never get used.

Your advertisers can’t wait weeks for audiences.

GrowthLoop’s Composable Commerce Media solution helps media teams turn first-party data into high-value audiences, launch campaigns faster, and prove what’s working across every channel.

Learn how retail media leaders at Costco, Fanatics, and Gopuff use GrowthLoop to create highly segmented audiences and deliver stronger results for their brand partners.

Learn more

We can't even agree on the basics

Albertson's and Ovative group's recent iROAS Demystified white paper found that methodological choices alone can swing incremental ROAS by 6.5x.

When I spoke with Liz Roche, vice president of media and measurement at Albertsons Media Collective for that piece, she described what genuine curiosity looks like in practice: getting data scientists from both sides of the table to sit down and, as she put it (seemingly unironically), "duke it out for about six hours" until they reach agreement on a methodology. I said in my write-up that that requires vulnerability from the retailer, whose methods are being scrutinized, and from the brand, which might learn that their first conclusions were misplaced.

Mutual benefit over bullshit

This is an area where retail media has a unique and profound benefit: both parties share the same ultimate goal: sales lift. Brand wants to sell more stuff, retailer wants to sell more stuff.

At the Ascendant Bootcamp event in September, amid a discussion about measurement and some degree of hand-wringing about how we're still facing down major gaps, an RMN leader stood up and said, "We do have standardized measurement. It's called sales lift."

A brand-side media buyer agreed: "My goal is to shift pallets. That's it."

And this week I spoke with GoPuff retail media GM Michael Peroutka for an upcoming profile piece. He told me that he's been calling ROI and ROAS bullshit metrics for years, and that moving product is the only metric that cannot be gamed.

Now, this all makes a lot of intuitive sense. But as I wrote last week, the tricky part is the planning part. What combination of ads, when to serve them, where to serve them – these are all questions that are upstream of that end goal of sales lift. If I'm spending budget as a brand, I want some early clues. I want to isolate which tactic contributed more than the others. And that's what retailers are at least attempting to solve for with recent MTA launches – whether the market sees that as the right solution or not.

Why ROAS refuses to die

As a brand, I also have a report card – one that hopefully identifies my own fingerprints on the outcome of growing sales. Because there's likely a whole lot of other people internally who'd also like to claim credit for growing those sales. That's one reason why ROAS persists.

I published Why ROAS refuses to die in February this year, discussing why brands still rely on ROAS, why it's lacking, but also some fairly rational reasons as to why it persists. Here's some comments that came through on LinkedIn when I published that piece:

Austin Leonard, GM of Dollar General Media Network: "ROAS as a standalone, no context KPI should not be the standard, but it does provide some basic currency across retailers (when used with context) and IS an informative supporting metric. Brands need to decide how/if they incorporate ROAS in performance analysis across short term reporting and long term models like MMM. But even emerging brands should use it while also factoring in targeting parameters, new brand buyers, LTV, media efficiency (CPM/CPC/unique reach), etc. as part of the story of how their marketing is performing. And yes, getting the details on basic ROAS closed loop methodology should always be a first step in the decision to use or discard it as a relevant metric."

Kirsten Gilbert, now VP sales & growth at digital signage company Navori Labs says that half the decks she sees on in-store campaign success still focus almost entirely on ROAS.

"Brands want physical retail media to behave like digital ads, but the store is really a mix of media, merchandising and environment — and increasingly a place for softer narrative brand moments as well. The most promising work I’m seeing isn’t about replacing ROAS, but layering it with signals like basket lift, dwell and category impact to understand the full effect of in-store media."

Corey Buller, senior director, commerce media at Dentsu, said that ROAS isn't an inherently bad metric. It just needs a lot of context to derive insights. "Your benchmarks have to be very specific to the situation: what's in the measurement set (Featured v Halo ROAS), Brand v Non-Brand targeting, heck even individual placement benchmarks depending on the attribution model. And even then it should inform your learning agenda, not campaign success. ROAS is red herring city."

This was echoed by Fernando L. Oliviero, CSM at retail media aggregator Unlimitail, who said that ROAS without context is just a number. Without clarity on incrementality, margin structure, and baseline demand, ROAS will be inefficient. Until incrementality frameworks are standardized across retailers and advertisers, ROAS will be king."

See the full range of comments on the post on LinkedIn

What next

So, we end up where we started. I hear from people who are brave enough to proclaim that they are actually solving measurement for good, for real this time. (Let's hear it then!) But as I so often find when writing this newsletter, it comes down to incentives. Everyone is behaving rationally in their head, even if it looks like bullshit to someone else.

Blurred lines create a little bit of shadow. Just enough shadow to hide your demons and sandbag some budget for next year, when you might really need it.