Optimize · Pricing / Offer Tests

Build an A/B Test Plan for Two Offer Structures

Build a rigorous A/B test plan comparing two agency offer structures with sample size, randomization, and decision rules.

foundermanagerAdvanced⏱ 4-6 hours of test design work
When to use
Use when you have two distinct offer structures (e.g. fixed retainer vs sprint-based, or scope-locked vs flexible hours) and want to run a clean head-to-head instead of guessing. Best for agencies with 15+ pitches per month so you can hit sample size in a quarter.
The prompt
You are an agency monetization strategist who runs pricing tests responsibly (no race to the bottom). You design experiments with explicit hypotheses, clear primary metrics, and pre-committed kill criteria so the agency learns regardless of outcome.
Agency: [AGENCY_NAME] — [SERVICES] | Offer A: [OFFER_A] @ [PRICE_A] | Offer B: [OFFER_B] @ [PRICE_B] | Current win rate: [WIN_RATE] | Avg deal: [AVG_DEAL_SIZE] | Test audience: [TEST_AUDIENCE] | Hypothesis: [HYPOTHESIS] | Monthly pitch volume: [PITCH_VOLUME]
Build an A/B test plan to compare Offer A vs Offer B head-to-head, including randomization method, required sample size, primary/guardrail metrics, and kill criteria.

- Specify randomization unit (lead, demo, week) and why
- Calculate minimum sample size per arm to detect a 5pt close-rate lift at 80% power
- Define one primary metric and 2 guardrail metrics (e.g. ACV, churn risk)
- Include explicit kill criteria and a max test duration
- Flag confounds (seasonality, rep skill mix, lead source)

Sections: (1) Test Design Summary, (2) Variants table with hypothesis, (3) Sample Size & Duration, (4) Primary + Guardrail Metrics with Decision Rules, (5) Confound Mitigation, (6) Go/No-Go Checklist.
Variables
  • [AGENCY_NAME] — Your agency name
  • [SERVICES] — Core services
  • [OFFER_A] — First offer structure (control)
  • [PRICE_A] — Offer A price
  • [OFFER_B] — Second offer structure (variant)
  • [PRICE_B] — Offer B price
  • [WIN_RATE] — Current close rate %
  • [AVG_DEAL_SIZE] — Current ACV
  • [TEST_AUDIENCE] — Segment to test against
  • [HYPOTHESIS] — What you expect to happen
  • [PITCH_VOLUME] — Pitches per month
Example input
Agency: Lumen Studio — web design + Webflow dev | Offer A: 12-week fixed-scope build @ $38k | Offer B: 2-week design sprints @ $9k each, min 4 | Current win rate: 28% | Avg deal: $36k | Test audience: Series A-B founders | Hypothesis: sprint model wins more deals because of lower entry commitment | Monthly pitch volume: 18
Example output
1) Test Design Summary: 90-day A/B by lead week.

2) Variants:
| Variant | Structure | Price | Hypothesis |
|---|---|---|---|
| A | Fixed-scope build | $38k | Baseline |
| B | Sprint-based | $9k × 4+ | Lower entry lifts close +8pts |

3) Sample Size: 64 pitches/arm to detect 8pt lift at 80% power → ~7 weeks at 18/mo.

4) Metrics:
- Primary: close rate (decision: ship B if +5pts sustained)
- Guardrail 1: 6-mo realized revenue per pitch (kill B if -15% vs A)
- Guardrail 2: sprint-to-sprint renewal rate (kill if <70%)

5) Confounds: alternate by lead week to absorb seasonality; balance rep assignments; tag lead source.

6) Go/No-Go: pricing committee sign-off, deck variants locked, CRM stage map updated, weekly readout cadence set.
Pro tips
  • Randomize by week, not by lead — it prevents reps from cherry-picking which variant a 'hot' lead gets
  • Always set guardrail metrics on realized revenue and retention, not just close rate
  • If your pitch volume is under 12/month, run sequentially (A for 90 days, then B) instead of true A/B
Works with
ClaudeChatGPTGemini
Done with prompts? Time to install the system
Book a STAOS call
Related prompts