Incrementality Test Design Checklist

Stop Guessing Whether Your Ad Spend Actually Works

---

Your platform says 5x ROAS. But how much of that spend is creating revenue that wouldn't exist without it?

This checklist walks you through designing your first incrementality test - from choosing the right test type to reading the results. Based on audits across 20+ e-commerce brands and data from 322 published incrementality studies.

Use it before you spend another month making budget decisions on numbers you can't verify.

---

Part 1: Test Readiness Assessment

Before you design a test, make sure you have the prerequisites.

Minimum Requirements

• Monthly ad spend above $10K (below this, opportunity cost of testing outweighs the value)
• 100+ weekly conversions across the account (below this, most tests won't reach statistical significance)
• Clean conversion tracking verified in the last 30 days (broken tracking invalidates everything)
• No major campaign restructuring planned during the test window
• Stakeholder agreement on what you'll do with the results (pre-commit to decision rules)

Data You Need Before Starting

• 90 days of performance data by geographic region (DMA or state)
• Conversion lag distribution: what percentage of conversions arrive at 1, 3, 7, 14, and 30 days
• Current brand search spend as a percentage of total spend
• Organic brand traffic baseline (from Google Search Console)
• Your breakeven ROAS threshold

---

Part 2: Choose Your Test Type

Brand Search Quick Test (Start Here)

Best for: First-time testers. Answers "is brand search incremental?" in 7-10 days.

• Verify organic rank is #1 for your top 10 brand terms
• Screenshot current Auction Insights for brand campaigns
• Record 30-day baseline: brand search clicks, organic brand traffic, total site traffic, total revenue
• Set up daily monitoring: organic brand clicks (Search Console), total site sessions, total revenue
• Pause all brand search campaigns
• Monitor for 7 days during a "normal" week (no sales, promotions, or major email campaigns)
• Calculate traffic delta: (Pre-test traffic - Test period traffic) / Pre-test traffic
• Calculate organic recovery rate: Increase in organic brand clicks / Decrease in paid brand clicks
• Calculate incremental CPA: Monthly brand search cost / (Monthly conversions x (1 - organic recovery rate))

Decision thresholds:

• Organic recovery above 90% AND traffic loss below 5%: Strong evidence brand search is not incremental. Consider cutting.
• Traffic drops more than 10%: Re-enable immediately. Plan a geo-split for more precise data.

Geo-Split Test (Gold Standard)

Best for: Brands with 250+ weekly conversions wanting high-confidence answers.

• Choose geographic units (DMAs recommended for most tests)
• Pull 90 days of performance data by geo unit
• Match treatment and control markets on:
  • Historical conversion rate (similar rates over past 90 days)
  • Revenue per visitor / AOV
  • Population and traffic volume
  • Demographics (age, income, urban/rural mix)
  • Historical revenue trend direction
  • Seasonal patterns
• Size the holdout at 15-20% of total market volume
• Verify no geographically adjacent DMAs are split between groups
• Set up geo-targeting for ALL campaigns to match test geography (not just the campaign being tested)
• Run 1-2 week baseline with both groups under identical conditions
• Verify treatment and control perform within 5% of each other during baseline
• Execute campaign changes in holdout markets only
• Freeze all other campaign changes during test period
• Monitor daily: conversions, revenue, organic traffic by test geography
• Run for minimum duration based on conversion lag (typically 3-4 weeks)
• Add observation window after test ends (equal to your longest conversion lag)

---

Part 3: Contamination Prevention

Every item here can invalidate your test results.

• No adjacent DMAs split between treatment and control groups
• ALL campaigns geo-targeted to match test geography (not just the campaign under test)
• No geo-targeted promotions running in only treatment or only control markets
• National promotions documented but allowed (they affect both groups equally)
• No new campaigns launched during the test window
• No bid or budget changes during the test window
• No major creative refreshes during the test window
• Competitor activity monitored for unusual changes

---

Part 4: Results Calculation

Core Metrics

Incremental Revenue:

Treatment market revenue: $_______ Control market revenue (raw): $_______ Control market population share: % Expected control revenue at treatment scale: Control revenue x (Treatment share / Control share) = $ Incremental revenue: Treatment revenue - Expected control revenue = $_______

Incremental ROAS (iROAS):

iROAS = Incremental Revenue / Test Ad Spend = _______x

Platform ROAS (for comparison):

Platform ROAS = Platform-Reported Revenue / Ad Spend = _______x

Incrementality Factor:

Factor = iROAS / Platform ROAS = _______

This is the calibration number you apply to ongoing reporting.

What Your Factor Means

• Factor above 0.80: Campaign is highly incremental. Scale with confidence.
• Factor 0.50-0.80: Meaningful credit-taking. Adjust reporting. Consider optimization.
• Factor 0.30-0.50: Significant overstatement. Review campaign structure.
• Factor below 0.30: Most reported value is credit-taking. Reduce spend or restructure entirely.

---

Part 5: Decision Rules (Fill In Before Testing)

Pre-committing to decisions prevents reinterpretation of uncomfortable results.

If lift is below _____%, we will: ________________________________

If lift is between _____% and _____%, we will: ________________________________

If lift is above _____%, we will: ________________________________

If results are inconclusive, we will: ________________________________

---

Part 6: The 0.7x Quick Reference

When you can't run a formal test, apply these conservative multipliers to platform ROAS:

These are conservative estimates based on published test data. Your actual factor will differ. Run the test to find your number.

---

Part 7: Testing Calendar

Refresh triggers (run a new test when any of these occur):

• Campaign structure changed significantly
• New major competitor entered or exited
• 90+ days since last measurement
• Business model changed (new products, pricing, markets)

---

Next Step

This checklist gives you the test design framework. The Google Ads AI Agentic System ($4,997+) includes:

• Complete geo-split test design with market matching methodology
• Brand search decision framework with published case study data
• PMax isolation protocols (3-cell methodology)
• YouTube/Demand Gen geo holdout protocol for measuring halo effects
• Statistical tools for reading results without a data science degree
• Quarterly testing calendar with refresh triggers
• Ready-to-use test design template, geo-split calculator, and results dashboard

The system is built for e-commerce brands spending $10K-$100K/month who want to stop making six-figure budget decisions on data they can't trust.