What Is Incrementality Testing? The Most Reliable Way to Measure Marketing Impact

Marketing measurement has a trust problem.

Meta, Google Ads and your analytics platform all report conversions. In many cases, multiple platforms claim credit for the same customer journey. That's the problem incrementality testing was designed to solve.

What Is Incrementality Testing?

Incrementality testing is a controlled experiment that measures the additional business impact created by a marketing activity by comparing outcomes in an exposed treatment group with outcomes in a comparable unexposed control group.

The key word is additional.

Many marketing channels receive credit for conversions that may have happened anyway. Incrementality testing helps separate those conversions from the ones genuinely influenced by a campaign.

It gives marketers a clearer picture of what's actually driving growth. That's especially valuable as privacy changes and tracking limitations make traditional attribution models harder to trust.

It's no surprise, then, that 71% of retail advertisers now rank incrementality as the most important KPI for their media investments, according to the Association of National Advertisers.

Why Traditional Attribution Often Falls Short

Attribution and incrementality answer different questions.

• Attribution attempts to identify which touchpoints contributed to a conversion.
• Incrementality testing asks whether the conversion would have happened at all.

Imagine a customer who already intends to buy from your company. They click a paid ad and complete a purchase.

An attribution model will likely credit the ad. But would that customer have converted anyway?

Attribution can't answer that question confidently because it measures correlation, not causation. This often leads to attribution inflation, where multiple channels claim credit for conversions they didn't meaningfully influence.

That’s why 82% of marketers aren’t fully confident in their attribution data.

Attribution models also understate the impact of upper-funnel channels that shape buying decisions long before a conversion occurs.

How Marketers Measure Incremental Impact

This additional impact is known as incremental lift, the core metric used in incrementality testing. It measures the increase in conversions, revenue, or another target outcome generated by a campaign compared to what would have happened naturally.

For example, imagine two similar audience groups:

• The treatment group sees your campaign
• The control group does not

If the treatment group converts at a higher rate than the control group, the difference represents incremental lift.

Let's say the treatment group converts at 6% while the control group converts at 5%.

The result is a 20% incremental lift, meaning the campaign generated 20% more conversions than would have occurred without advertising.

Types of Incrementality Tests

There is no single way to measure incrementality.

The right testing methodology depends on your channels, audience size, available data, and business goals.

The most common approaches include:

1. User-level holdout tests to measure campaign incrementality
2. Conversion lift studies to validate advertising effectiveness
3. Geo lift tests to measure market-level impact
4. Matched market tests to evaluate channel investment decisions

1. User-Level Holdout Tests to Measure Campaign Incrementality

A portion of your target audience is intentionally excluded from a campaign, allowing marketers to compare outcomes between exposed and unexposed users.

This approach works best for digital advertising channels with strong audience targeting capabilities, such as Meta, LinkedIn, and display advertising.

Example: Uber questioned whether Meta ads were generating truly incremental customer acquisition. After running a large-scale holdout test, the company found no measurable business impact from the channel and reallocated roughly $35 million in annual spend.

Reflecting on the project, Sundar Swaminathan, former Head of Marketing Science at Uber, said it remained one of the most rewarding analyses of his career because it demonstrated what a high-functioning, data-driven culture looks like in practice.

He added that it also delivered tangible business impact, which is ultimately what every analyst hopes to demonstrate.

2. Conversion Lift Studies to Validate Advertising Effectiveness

Conversion lift studies are platform-native incrementality tests offered by platforms like Meta and Google.

While they use the same treatment-and-control principles as traditional holdout tests, the platform handles audience assignment, measurement, and analysis to make it easier to validate incremental business impact beyond standard attribution reporting.

Example: Home goods retailer Biedronka Home used a Meta Conversion Lift study with search lift methodology and found that its campaigns drove a 24% lift in incremental paid traffic, and 20% in organic traffic. That helped to validate business impact beyond what standard attribution reporting captured.

3. Geo Lift Tests to Measure Market-Level Impact

Geo lift testing compares performance across regions with different levels of advertising exposure. Because it doesn't rely on user-level tracking, it's often used when privacy restrictions or platform limitations make audience-based testing difficult.

A closely related methodology, matched market testing, applies the same principle while using carefully selected treatment and control markets to improve the reliability of the comparison.

Example: Google developed its Geo Experiments framework to measure the incremental impact of advertising by varying media exposure across geographic regions. By comparing performance between test and control markets, advertisers can estimate true business impact without relying on user-level attribution.

4. Matched Market Tests to Evaluate Channel Investment Decisions

Matched market tests are a more controlled variation of geo lift testing. Rather than comparing any two regions, marketers select markets with similar historical performance, demographics, and competitive conditions before introducing different levels of advertising exposure.

This helps isolate the incremental impact of a channel and supports more confident budget allocation decisions.

Example: A retail advertiser used a matched-market test to evaluate the incremental impact of Google's Demand Gen campaigns after platform reporting suggested the channel was underperforming.

The team turned Demand Gen off in a group of test markets while keeping it active in carefully matched control markets, then compared revenue outcomes between the two groups.

The results showed Demand Gen generated 328% more revenue than platform reporting showed and delivered an incremental ROAS of 6.9.

As the Silverback Strategies analyst behind the experiment observed, "A channel that the platform said to cut was one of the strongest performers in the account once the real contribution was measured."

ROAS vs. iROAS: Understanding the Difference

One of the biggest reasons marketers run incrementality tests is to understand the difference between ROAS and iROAS.

• ROAS, or Return on Ad Spend, measures attributed revenue relative to advertising costs.
• iROAS, or Incremental Return on Ad Spend, measures only the revenue directly caused by the campaign.

Since a campaign can appear highly successful in platform reporting while generating relatively little incremental value, sophisticated marketing teams use iROAS when evaluating channel performance and allocating budget.

Which Incrementality Test Should You Use?

The right approach depends on your goals, available data, audience size, and advertising channels. A B2B company evaluating LinkedIn campaigns may need a different methodology than an eCommerce brand testing Meta ads across multiple markets.

First, it's important to understand when incrementality testing makes sense and when it doesn't.

When Incrementality Testing Isn't the Right Choice

Incrementality testing can provide valuable insights, but it's not always the best measurement approach. Consider alternative methods if:

• You don't have enough traffic or conversions. Low conversion volume can make it difficult to reach statistical significance and draw reliable conclusions.
• You're running a major seasonal campaign. Testing during Black Friday, holiday promotions, or product launches can introduce unnecessary risk and distort results.
• You're changing multiple variables at once. If you're simultaneously testing new audiences, creative, offers, and landing pages, it's difficult to isolate what's driving performance.
• Your treatment and control groups overlap. If control group users are still exposed to your marketing through other channels, the test results become less reliable.
• You need answers immediately. Incrementality testing typically requires several weeks of data collection and analysis, making it better suited to strategic decisions than day-to-day optimization.

Choosing the Right Test Type

If incrementality testing is a good fit for your situation, the next step is selecting the appropriate methodology.

How to Interpret Incrementality Test Results

Running an incrementality test is only half the job. The real value comes from understanding what the results mean and how they should influence future marketing decisions.

So, what is a good incremental lift?

The answer depends on the channel, campaign objective, customer acquisition cost, and overall profitability. A 5% lift may be disappointing for one campaign and highly valuable for another.

Instead of focusing on arbitrary benchmarks, evaluate incremental lift in the context of business outcomes.

Ask:

• Did the campaign generate profitable growth?
• Did incremental revenue exceed advertising costs?
• Did the results justify continued investment?
• How does the lift compare with other channels?

It's also important to remember that lift doesn't tell the whole story on its own.

How to Evaluate Your Results

As a general rule:

• High lift and strong iROAS suggest the campaign is generating meaningful business impact and may justify additional investment.
• High lift but weak reported ROAS may indicate attribution is under-crediting the channel. Before cutting budget, investigate whether the campaign is influencing conversions that attribution models aren't capturing.
• Low lift but strong reported ROAS can be a sign of over-attribution. The campaign appears effective in platform reporting but may not be driving as much incremental value as expected.
• Low lift and low ROAS often indicate limited business impact. In these cases, it may be worth revisiting the targeting, creative, offer, or channel strategy before investing further.

How To Make Results Useful for Budget Decisions

Incrementality results are most valuable when they can stand up in a budget conversation. That means going beyond reported ROAS and showing:

• What was measured: Incremental revenue, conversions, or another business outcome.
• How it was tested: Treatment group, control group, duration, and confidence level.
• What changed: Whether the result supports increasing, reducing, or reallocating spend.

This is where iROAS is especially useful. It gives finance teams a clearer basis for investment decisions because it measures revenue caused by advertising, not just revenue attributed to it.

What Is Incrementality Testing: Final Words

Instead of more dashboards, marketing teams need more confidence in the decisions those dashboards support. Incrementality testing helps provide that confidence by moving the focus from attribution to causation.

No measurement approach is perfect, but those looking to understand what is genuinely driving growth, incrementality testing offers one of the clearest paths to more informed marketing decisions.

Our team ranks agencies worldwide to help you find a qualified partner. Visit our Agency Directory to find top-rated digital marketing companies, as well as:

1. Content Marketing Agencies
2. Conversion Rate Optimization Services
3. Affiliate Marketing Companies
4. Direct Marketing Companies
5. Top Digital Marketing Agencies In Portland

What is Incrementality Testing: FAQs

1. What are some common mistakes that undermine incrementality tests?

Common mistakes include using sample sizes that are too small, allowing treatment and control groups to overlap, testing during unusual seasonal periods, and changing multiple variables at once.

Any of these issues can make it difficult to isolate the true impact of a campaign and reduce confidence in the results.

2. How is incrementality testing different from A/B testing?

A/B testing compares two versions of a campaign, ad, or experience to see which performs better, while incrementality testing measures whether marketing activity generated additional outcomes compared to no exposure at all.

3. How long should an incrementality test run?

Most incrementality tests run for several weeks, but the ideal duration depends on traffic volume, conversion rates, and how much lift you're trying to detect.

4. What are the limitations of incrementality testing?

Incrementality testing requires sufficient data, careful experimental design, and a willingness to withhold marketing exposure from a portion of your audience, which can increase both complexity and opportunity cost.

5. How can incrementality testing help identify ad fraud?

Incrementality testing can help uncover ad fraud by revealing when a channel appears successful in attribution reports but generates little or no incremental business impact.

If a campaign receives a high volume of attributed conversions but fails to produce measurable lift in a controlled experiment, one possible explanation is fraudulent traffic, attribution manipulation, or other forms of wasted spend.

This is especially relevant in display, affiliate, and performance marketing channels.