A/B Test Significance Calculator

Enter visitors and conversions for variant A and B. Get each conversion rate, the relative lift, a two-proportion z-test p-value, and a clear yes/no on significance at 95%.

Answers as you type · Formula shown, not hidden · Free, no sign-up

01 Your numbers
Updates as you type

Control group sample size.

Orders, sign-ups or clicks on A.

Treatment group sample size.

Orders, sign-ups or clicks on B.

Two-proportion z-test on pooled conversion rates. Significant at 95% means the two-tailed p-value is under 0.05 — a difference this large would appear by chance less than one time in twenty.
02 A/B significance
Significant
At 95% confidence
Variant A rate
10.00%
Variant B rate
15.00%
Relative lift
50.00%
p-value
< 0.001

p = < 0.001. The lift of 50.00% clears the 95% bar — safe to ship B on conversion rate alone.

Want these numbers checked against your live accounts?

The calculator gives you the number instantly. Leave your email and we will look at your actual accounts and send back what we would change first.

A person reads your numbers and replies. No drip sequence.

01A/B Test Significance Calculator

How it is calculated

Each variant’s conversion rate is conversions ÷ visitors. Relative lift is (rate B − rate A) ÷ rate A. Significance uses a two-proportion z-test: the rates are pooled, a standard error is formed from that pool, and z = (rate B − rate A) ÷ SE. The two-tailed p-value comes from the standard normal distribution; p under 0.05 is significant at 95%.

The test assumes independent visitors, one conversion per visitor, and that you decided the sample size before peeking. Sequential peeking or stopping early inflates false positives — run the planned sample, then read the p-value once.

02A/B Test Significance Calculator

When a “significant” lift still is not worth shipping

Statistical significance is not practical significance. A 2% relative lift on a tiny baseline can clear p < 0.05 with enough traffic and still be too small to pay for the engineering or creative change. Pair this calculator with the sample-size calculator so you only power tests for lifts that matter.

03A/B Test Significance Calculator

A worked example

Control: 100 conversions from 1,000 visitors (10%). Variant: 150 from 1,000 (15%). Relative lift is 50%. The pooled rate is 12.5%, the z-score is about 3.3, and p is well under 0.001 — significant at 95%, and at 99%.

Pull your real numbers from your ad account

Skip the typing. Connect Meta, Google, TikTok or LinkedIn and Sprites reads your actual spend, impressions, clicks and conversions, then shows where they sit against the ranges above.

Connect your account

Embed this calculator

Put the A/B Test Significance Calculator on your own site or blog. Paste this HTML where you want it to appear. It runs in the reader's browser and needs no account.

<iframe src="https://www.sprites.ai/embed/ab-test-significance-calculator" title="A/B Test Significance Calculator" width="100%" height="620" style="border:0;max-width:720px" loading="lazy"></iframe>
<p style="font-size:13px"><a href="https://www.sprites.ai/tools/ab-test-significance-calculator">A/B Test Significance Calculator</a> by Sprites</p>

Questions, answered

What people ask about the a/b test significance calculator.

Runs on the official partner APIs
GoogleMetaLinkedInTikTokRedditMicrosoft Ads
Still deciding?

See it on your own ad accounts in a short demo, or try every part free for 7 days.

How do you test A/B significance for conversion rate?

Use a two-proportion z-test. Pool the conversion rates, compute the standard error, divide the rate difference by that error to get z, then convert |z| to a two-tailed p-value. p under 0.05 is significant at 95% confidence.

What does significant at 95% mean?

It means that if there were truly no difference, a gap at least this large would show up less than 5% of the time by chance. It is not a 95% chance that B is better.

What is relative lift?

The percent change from A to B: (rate B − rate A) ÷ rate A. Going from 10% to 15% is a 50% relative lift, or a 5 percentage-point absolute lift.

Can I peek at the test early?

Repeated looks without a sequential method inflate the false-positive rate. Decide the sample size first with a sample-size calculator, then evaluate significance once that sample is in.

What if one variant has zero conversions?

The rate is still defined (0%), but relative lift from a zero baseline is undefined, and a pooled rate of 0% or 100% leaves no variance for a z-test. Wait for more data.

More free tools

Your best ads are
one brief away

Research, creatives and media buying, run by one agent.

7-day trial · cancel anytime · or book a demo
Whatnot
Attio
Plaud
Framer
Gushwork