Conversation Episode 1 Measurement · Brand Lift

Measuring creative persuasion before a campaign beats measuring it after.

Interviewed by Justin Cooke

Published

Portrait of James Slezak, Co-founder & Chief Executive, Swayable

James Slezak is co-founder and chief executive of Swayable, a persuasion measurement platform used by some of the world's largest brands, agencies, and strategy consultancies to test whether marketing creative will move demand before it goes to market. Swayable is a public benefit corporation, a Y Combinator graduate, and works with Paramount Pictures, Grupo Bimbo, Meta, and a list of technology vendors including Intel, Square, Intuit, and Amazon. In this conversation he sets out why the traditional brand lift study delivers data too late to act on, what 10 million respondents on the platform have taught Swayable about the relationship between baseline demand and incremental lift, why public benefit incorporation shapes who the company will and will not work with, and where generative AI fits inside a measurement business that still believes only real consumers can validate a story.

The measurement gap: why brand lift data arrives too late

Start with what Swayable does, and the problem you set out to solve.

Swayable measures persuasion. It's a data platform used by many of the world's largest brands, their agencies, and some of the strategy consultancies they work with, and it tells you whether your creative will cause lift. The important thing is it tells you that in a timeframe that lets you act on it. People are familiar with the Brand Lift Study. The idea is that some creative is like a coupon, designed to be assessed by whether someone buys at the checkout counter. But most of the work happening in a community like this one in New York is about big ideas that get someone to want the product in the first place, love the brand, or trust the equities the brand carries. The problem is people haven't had much data on whether that work is doing its job. You can't point at click-through rates and view counts and engagement and treat them as a proxy. It's really not a proxy. Those metrics can lead you toward decisions that don't cause lift.

And the timing problem with the existing studies.

The Brand Lift Study is a good thing to do. We encourage them. But the limitation is you get the answer too late. You've already run the campaign. Maybe you feed the result into next year's plan. But you're taking decisions every day, every week, every month. If you can inform those with the best available evidence on whether the creative works, you're in a much better position.

The case files: Paramount, Thomas English Muffins, and the Gen Z trap

Bring it to life. A case study.

Hollywood is a clear one. Paramount has been one of our big users for several years. Until a recent release that I won't name, Top Gun Maverick was their largest-grossing picture. The use case is straightforward: drive intent to see a film in theatres. There's enormous investment in making the films and marketing them, and it all comes to a head on opening weekend. You need the audience to get in the car, buy the tickets, turn up. Reach on a digital ad isn't enough. You need to know you are causing them to turn out. Paramount makes its creative decisions using Swayable data. We don't take credit for the blockbuster year, but we're part of it.

Another category, another story.

We work a lot with Grupo Bimbo, a group out of Mexico. A very large fraction of all American baked goods come from this one company. Brands you might think were American, Sara Lee and Thomas English Muffins, sit inside their portfolio. They engaged us at the start of a wider effort with Bain & Company to rebuild their marketing tech stack and put more weight into upper-funnel marketing. Bain's strategy teams know that to gain share you have to go beyond coupons and conversion campaigns and tell the story.

The setup phase is fast. With Paramount we did it in under a day. With the baked goods business we had a few weeks. The question is: what are the goals and what are the metrics? For Thomas English Muffins it was I love these muffins, I'm considering purchasing them, I intend to purchase them. The marketing works if those numbers move and it doesn't if they don't.

The Gen Z question came up here.

Their consumers age out. Many companies have the same problem. The team had a genuine debate. Some thought the answer was to age down the language, use more emojis, mirror how Gen Z talks. Others thought you should stick with the traditional tone. They produced ads exemplifying each option and ran them on Swayable. The traditional tone won. The lift was about twice what it would have been if they'd gone with the seemingly natural choice. Being authentic turned out to work best. Who would have known? That's the point. You don't have to guess.

From baseline demand to lift: rethinking who the target really is

Most brands pick their target audience by who already wants the product. You're arguing that's the wrong question.

Marketing is about increasing demand. Many teams start with the people who already have it. They run a survey asking who's interested in our product, who has the most demand, and assume that's their consumer. Maybe. But your marketing is designed to generate more demand. We have a case with Intuit where they had multiple potential target segments. The one with the highest baseline demand wasn't the one with the highest lift. The segment with close to the highest baseline, not quite the highest, produced a much larger increase in demand when the message was delivered to them. If you target the people who are already in market and not moving anywhere else, you're wasting the effort.

You mentioned the test-and-learn habit. What separates teams that get a lot from it from teams that don’t?

When teams don't iterate, the spread of lift across creative variants is wide enough that a 2x improvement is fairly easy to find. When teams do iterate, when they take what worked and feed that into what they create next week, next month, next quarter, we've seen lifts as high as 10x. That sounds implausible. We have the case study references. Grupo Bimbo has built learning labs around it. Bain has been good at helping major consumer brands set up the test-and-learn infrastructure. Because we're not an agency and we don't create the creative, our role is to provide the measurement that supports the development of that internal capability.

Pre-launch testing and the agency-brand tension it dissolves

There's a lot of tension between brand, agency, creative, and media teams over what to run and why. Where does measurement sit in that?

A lot of that tension comes from getting insight too late to act on it. At that point you're just grading people's homework and no one wants to get an F. They're going to want to have a fight about it. There's a difference between telling somebody that the campaign they ran two months ago wasted millions of dollars because it didn't generate lift, and telling them that the thing they're about to launch next week would be better if they went with option two rather than option five. That's a different conversation entirely. We've seen that tension reduce dramatically. We've seen people who were sceptical about quantitative evidence in this kind of decision become its strongest advocates. We try to see our role as supporting the people doing the creative work, not grading them.

A practical question on the technology. Swayable is platform-agnostic. Does it work the same way across formats?

The consumer sees the ad or treatment in our first-party environment, which makes us platform-agnostic. You can test Facebook ads, Google ads, billboards. We've done large out-of-home work with Airbnb. The platform delivery isn't what we're measuring; you've usually made the creative decision before you get to that point. We're focused on whether the creative is the right thing to give to that platform. The delivery vehicle is one question. The active ingredient is another, and that's what we measure.

Where AI fits, and where Swayable insists it doesn't

You're a measurement business in a market that's becoming an AI-creative business. How are the two intersecting?

We saw this revolution coming earlier than most. In our Y Combinator pitch deck we had a slide on machines making creative. People laughed. They said it was thirty years away. It has been mind-blowing to see how quickly that's arrived. The question is how to use these models effectively. Our position is that they should support human creativity and the validation of real consumers, not replace consumer judgement. Ultimately the person has to go buy the muffins. The machine isn't going to do that.

Where AI is most useful for us is in drawing more insight from real consumers, not replacing them. We've been developing population modelling since the beginning. You build a sophisticated view of the consumer base, who there are more of, who holds which kinds of opinions. The richer the model, the greater the precision when you run a test, because the platform can identify patterns and outliers across demographics automatically. The models are trained and retrained on the fly. We've had more than ten million responses through the platform, which gives us a set of priors that makes each new test more precise than it would otherwise be.

Public benefit, ethics, and where Swayable won't take the money

You incorporated as a public benefit corporation. Why?

We launched just before the Cambridge Analytica scandals broke. The question of corporate structure mattered then and matters now. A public benefit corporation has a mission baked into its structure beyond maximising returns. Ours is to help people tell the truth more effectively. We've never taken the position that we'll work with anybody for any purpose. We are careful about who uses the platform and what the work is doing in the world. We try to assess whether it's on balance more harmful than not.

And consumer data.

The Cambridge Analytica moment forced everyone to take seriously what you do when consumers give you their information. You can't just present a click-wrap contract, get everyone to click agree, and use the data in ways the person never expected. Where we can avoid collecting personally identifiable information, we do. We certainly don't keep it. Our work is about big marketing campaigns reaching thousands or hundreds of thousands or millions of people. The fact that a specific individual at a specific address thinks something isn't useful at that scale. You need to verify the respondents are real people. Beyond that, we're not trying to retarget anyone.

The question for the board

If creative persuasion can be measured before launch, what share of our pre-test investment validates the message versus the channel?