What Are A/B Tests?

Alpha Insights A/B Testing lets you test changes on your store and measure which version actually converts. Visitors are split between a control and one or more treatments. Each person keeps the same variant on later visits, and results are tied to your Alpha Insights analytics and orders — so you can see conversion rate, lift, and (in Pro) revenue and significance, not just clicks.

Open the feature from Alpha Insights → A/B Test. Analytics must be enabled in General Settings before a test can run.

What you can test

You can also branch theme or plugin code with the helpers wpdai_in_experiment() and wpdai_get_experiment_variant(). See Developer Helpers.

How a test runs

  1. You create a draft with a primary goal (Free: Purchases; Pro: add to cart, page view, custom event, purchase value, or average order value), page rules, and variants.
  2. You start the test. Targeted URLs miss full-page cache so the correct variant can render on first paint.
  3. On a matching page view, Alpha Insights assigns the visitor to an arm (or holdout, in Pro) and records eligibility, assignment, and exposure events.
  4. That visitor keeps the same variant via a sticky visitor cookie until you pause or complete the test.
  5. The A/B Test list and the A/B Tests report show exposures, funnel steps, conversions, conversion rate, and lift versus control.

Statuses

Status What it means
Draft Saved but not assigning traffic. Safe to edit.
Running Eligible visitors are assigned. Targeted URLs bypass page cache.
Paused No new assignments or variant code. Existing assignment history is kept. Cache returns to those URLs.
Completed The test is finished. History stays available in the report.

Free vs Pro at a glance

Free includes one running A/B test, 100% of eligible traffic, CSS/JS, and a Purchases goal. Pro adds A/B/n variants, holdout %, PHP snippets, extra page and audience targeting, scheduled start/end dates, extra primary goals (including purchase value and average order value), and significance / revenue on the report. Full comparison: Free vs Pro.

Related