A solid hypothesis, a power analysis, and statistical significance: the holy trinity of A/B testing.

To test for statistical significance you need to calculate the p-value, which you can do above. However, when you are exporting data from Google Analytics or Mixpanel it becomes very repetitive manually entering the numbers, for multiple calculations. So I made an Excel sheet that calculates the two-tailed p-value. You can download this for free below. It has a template based on GA/Mixpanel exports with a single cell formula that you can copy into your experiment analysis Excel sheet. The Jargon Buster explains why you should use a two-tailed calculation, and not one-tailed.

Read more about statistical significance, the p-value, and the Excel sheet calculation in my Medium post.

Experimentation Hub was created by Rik Higham, who is a Senior Product Manager at Skyscanner.

Read Rik's Medium posts on experimentation and Product Management here.

Copyright © Rik Higham 2016 - 2017