Skip to content
← Articles & field notesField note2 min readRevised

Design feed experiments without losing item identity

Keep valid product identifiers and isolate the change before interpreting campaign performance.

A supplemental source or new campaign does not by itself create a clean experiment. Preserve valid item identity and compare supported exposures rather than assume learning is isolated.

A feed test needs a specific hypothesis and an eligible exposure design. Supplemental data can help manage product attributes, but it does not create a control and variant by itself.

Start with one factual change: a missing brand, a supported product attribute or clearer wording around an actual buying question. “Benefit-focused” and “spec-focused” titles are candidate approaches, not permission to invent a product claim.

Record the existing item identifiers, primary source, supplemental attributes, campaign scope and destination. A supplemental source can override an attribute for the same item; that does not mean several versions of that item are independently randomized into separate experiments.

Use a platform-supported design that can isolate the proposed change. Preserve valid product identity and avoid duplicate offers created solely for test cells. If only a before-and-after comparison is possible, retain its confounders rather than call it a controlled A/B test.

Choose eligible products from the actual question and available signal. The top twenty percent by revenue is a possible prioritization heuristic, not a guarantee of statistical power.

Measure compatible exposure, relevant clicks, conversions and contribution over the same window. Document product availability, price, bidding and other concurrent changes. Statistical confidence depends on the design and observations; it does not automatically arrive in week five.

A title or description should communicate the product accurately and follow the current content rules. Do not add shipping promotions or pricing language indiscriminately to product text.

Keep the original valid source, settings and baseline so the change can be recovered. Export the observed results and record what the comparison can and cannot establish before extending it to the rest of the catalog.

Take the idea further

Plan a valid feed test

Give your growth
an operating team.

Google Ads, Meta Ads and Klaviyo Email/SMS. Buying, creative, copy, pages, testing and optimization, handled by the team that built the AI systems.

Typically live within 48 hours. The clock starts when we have account access and the assets we need.

Discuss your growth goalCompare services and fees

Keeping execution in your team?
Book a paid working session or explore licensed systems.