A/B Testing

Run controlled experiments on your Swft Checkout — template variations, accent colours, typography, copy — and let real conversion data tell you which design wins.

What you can test

• Template — minimal vs split vs fullscreen vs centered vs express.
• Accent colour — your primary action button colour.
• Typography — font family for body and headings.
• (Coming soon) — full custom CSS variants.

Each test has a control (your current design) and one or more variants (changes to test against the control).

Setting up a test

Swft Dashboard → A/B Testing → Create test.

1. Name the test (e.g. “Split vs Minimal for mobile”).
2. Add 2-5 variants. For each:
   • Pick the template
   • Pick the accent colour
   • Pick the font(s)
3. Set the traffic split. Defaults to 50/50; you can weight asymmetrically (e.g. 10% to a risky variant, 90% to control).
4. Click Start test.

The test runs against incoming sessions — each shopper is assigned a variant via a cookie and stays on it for the rest of their session. Returning shoppers see the same variant they saw before (we cookie for 30 days).

Reading the results

The test page shows:

Metric	Per variant
Sessions	Total checkout sessions on this variant
Completed orders	Successful payments on this variant
Conversion rate	Completed ÷ Sessions
Revenue	Total order value on this variant
AOV	Average order value
Statistical significance	p-value vs control (when sample is large enough)

A green badge appears when one variant beats control with p < 0.05 and at least 250 sessions per variant.

Stopping & rolling out

When you’re satisfied a variant has won:

1. Click Stop test.
2. Pick the winning variant.
3. Click Roll out to 100%.

The winning variant becomes your new default in the Checkout Editor. All future sessions see the winning design.

Tips for good tests

• Test one thing at a time. Don’t change template + colour + font + copy all in one variant — you won’t know which change moved the needle.
• Be patient. You need ~250 sessions per variant for results to be statistically meaningful. For most stores that’s a week of traffic. Resist stopping early because variant B is “up 12%” with 30 sessions.
• Variants should be meaningfully different. Tweaking the accent from #D4EF3B to #D3EF3C won’t move conversion. Test changes you’d actually consider deploying.
• Watch for seasonality. A test during Black Friday is not representative of normal traffic.
• Don’t stop at the first significant result. Wait until the sample size feels solid. Tests can flip-flop with small samples.

Gotchas

• Returning shoppers stick to their variant. This is by design (otherwise they’d see different designs across sessions, which is jarring). But it means switching tests rapidly can leave some shoppers on stale variants for up to 30 days. Wait for cookies to expire if you’re sensitive to that.
• Mobile vs desktop responds differently. A test that wins overall may lose on mobile. We surface a device breakdown once your sample is big enough.
• You can run only one active test at a time. Multi-variant testing across two dimensions is on the roadmap.
• Stripe Connect orders only. The traffic split system attaches the variant cookie before the gateway choice, so all gateways participate — but reporting is most reliable for Stripe orders because we have the richest per-order data.