All Latest 650 A/B Tests

Screen

Device

Metric

Tag

Powered for

Only Significant

p-value < 0.05

MOST RECENT TESTS

Test #256 by Alex James Aug 23, 2019 Desktop Mobile Signup X.X% Signups

Alex Tested Pattern #109: Required Field Labels

Pattern #109: Required Field Labels - Variant A

Pattern #109: Required Field Labels - Variant B

The original had no fields marked as required. The variant had all fields marked as required with an asterisk (and a reference note).

Total Sample: 116,470 • Statistical Power at 10% MDE: 99.9%

Which A Or B Actually Wins? Find Out Before You Test.

Members see every test result — the winners, the flat ones, and the losers — along with exact effects and sample sizes. Use it to estimate your tests and prioritize by probability, not gut feel. Start every experiment with the odds on your side.

Get Access to All Test Results

Test #40 on Adoramapix.com by Herman Klein Aug 13, 2019 Desktop Product X.X% Sales

Herman Tested Pattern #85: Benefit Button On Adoramapix.com

In this experiment only the button label changed. The control had a more immediate (next step-oriented) button label of "Upload Your Photos Now". The variation tried a more benefit-oriented button of "Order Now" (hinting at paying and obtaining the product).

Total Sample: 13,454 • Statistical Power at 10% MDE: 94.7%

Test #252 on Thomasnet.com by Julian Gaviria Jul 30, 2019 Desktop Mobile Content X.X% Engagement

Julian Tested Pattern #107: Contrast Links & Buttons On Thomasnet.com

Pattern #107: Contrast Links & Buttons - Variant A

Pattern #107: Contrast Links & Buttons - Variant B

In this experiment a light blue bold link was tested against a darker blue bold link.

Total Sample: 142,102 • Statistical Power at 10% MDE: 99.9%

Test #250 on Volders.de by Alexander Krieger Jul 25, 2019 Desktop Mobile Signup X.X% Signups

Alexander Tested Pattern #106: Back Buttons On Volders.de

In this experiment, a version without a back button was tested against a one where it was positioned in the upper left corner. This test occured on a second step of a contract cancellation service.

Total Sample: 16,894 • Statistical Power at 10% MDE: 99.9%

Test #39 on Over-blog.com by Tael Pinault Jul 02, 2019 Desktop Signup X.X% Signups

Tael Tested Pattern #83: Progressive Fields On Over-blog.com

Pattern #83: Progressive Fields - Variant A

Pattern #83: Progressive Fields - Variant B

This test explored a progressive disclosure interaction in variation B. When a user started typing text into the first field, two more fields expanded into view.

Total Sample: 10,464 • Statistical Power at 10% MDE: 90.6%

Test #246 on Thomasnet.com by Julian Gaviria Jun 12, 2019 Desktop Mobile Home & Landing X.X% Progression

Julian Tested Pattern #88: Action Button On Thomasnet.com

In this experiment, the navigation label was changed from "Network" to "Find Suppliers". The idea was to make use of an action label with a clearer benefit.

Total Sample: 956,112 • Statistical Power at 10% MDE: 96.5%

Test #245 on Kenhub.com by Niels Hapke Jun 11, 2019 Desktop Signup X.X% Signups

Niels Tested Pattern #19: Benefit Testimonials On Kenhub.com

Pattern #19: Benefit Testimonials - Variant A

Pattern #19: Benefit Testimonials - Variant B

In this experiment a more elaborate and authentic testimonial was used instead of three more generic ones from social media sites.

Total Sample: 130,872 • Statistical Power at 10% MDE: 99.9%

Test #242 by Alex James May 27, 2019 Desktop Mobile Signup X.X% Signups

Alex Tested Pattern #7: Social Counts

Total Sample: 79,012 • Statistical Power at 10% MDE: 98.8%

Test #241 on Kenhub.com by Niels Hapke May 24, 2019 Desktop Mobile Signup X.X% Signups

Niels Tested Pattern #105: Lead Magnets On Kenhub.com

In this experiment, a free study guide ebook was promoted on a registration page.

Total Sample: 63,631 • Statistical Power at 10% MDE: 99.9%

Test #240 on Thomasnet.com by Julian Gaviria May 16, 2019 Desktop Listing X.X% Engagement

Julian Tested Pattern #13: Centered Forms & Buttons On Thomasnet.com

Pattern #13: Centered Forms & Buttons - Variant A

Pattern #13: Centered Forms & Buttons - Variant B

In this experiment, two different positions of the View Catalog button were compared. In version A the button was smaller and on the right. In version B the button was wider and more central. (The actual test was inverted before publishing to match the A-B of the pattern.)

Total Sample: 207,844 • Statistical Power at 10% MDE: 83.2%

Test #236 by Alex James Apr 04, 2019 Desktop Signup X.X% Signups

Alex Tested Pattern #9: Multiple Steps

In this experiment a single screen signup process was broken into 2 separate steps: account creation & details.

Total Sample: 282,316 • Statistical Power at 10% MDE: 99.9%

Test #233 on Annonce.cz by Ondřej Ilinčev Apr 01, 2019 Desktop Signup X.X% Sales

Ondřej Tested Pattern #100: Postponed Registration On Annonce.cz

Pattern #100: Postponed Registration - Variant A

Pattern #100: Postponed Registration - Variant B

In this experiment, the old flow had a login / register / guest checkout as a second step. The variant B flow removed this step and treated everyone as a guest (and later offered to login / register). In the B version guests had a voluntary registration on the thank you page and in a confirmation email.

Total Sample: 12,146 • Statistical Power at 10% MDE: 99.9%

Test #234 by Alex James Apr 01, 2019 Desktop Listing X.X% Leads

Alex Tested Pattern #101: Search Keyword Highlighting

Pattern #101: Search Keyword Highlighting - Variant A

Pattern #101: Search Keyword Highlighting - Variant B

In this experiment, a different style for keyword highlighting was used.

Total Sample: 738,295 • Statistical Power at 10% MDE: 99.9%

Test #232 on Yoast.com by Sjardo Janssen Mar 15, 2019 Desktop Mobile Checkout X.X% Sales

Sjardo Tested Pattern #6: Customer Star Ratings On Yoast.com

Pattern #6: Customer Star Ratings - Variant A

Pattern #6: Customer Star Ratings - Variant B

In this experiment, the Yoast team replaced one of the bulleted reassurances on their checkout page ("4.9 out of 5 stars" vs "24/7 Free support"). Raising the question - is free support or high reviews valued more? - Thanks Sjardo & Meike for sharing!

Total Sample: 20,661 • Statistical Power at 10% MDE: 81.7%

Test #230 on Goodui.org by Jakub Linowski Mar 09, 2019 Desktop Mobile Listing X.X% Engagement

Jakub Tested Pattern #56: Hover Button On Goodui.org

In this test we tested onhover buttons (variant) versus more traditional always exposed and visible ones.

Total Sample: 30,699 • Statistical Power at 10% MDE: 99.9%

Test #225 by Devesh Khanal Feb 17, 2019 Desktop Product X.X% Sales

Devesh Tested Pattern #41: Sticky Call To Action

Pattern #41: Sticky Call To Action - Variant A

Pattern #41: Sticky Call To Action - Variant B

Total Sample: 53,908 • Statistical Power at 10% MDE: 88.4%

Test #224 by Alex James Feb 11, 2019 Desktop Home & Landing X.X% Progression

Alex Tested Pattern #3: Fewer Form Fields

Pattern #3: Fewer Form Fields - Variant A

Pattern #3: Fewer Form Fields - Variant B

This experiment reduced the search form by removing the distance field.

Total Sample: 97,074 • Statistical Power at 10% MDE: 99.9%

Test #222 on Thomasnet.com by Julian Gaviria Feb 01, 2019 Desktop Listing X.X% Leads

Julian Tested Pattern #7: Social Counts On Thomasnet.com

In this variation, a number of social proof references were added to a signup modal.

Total Sample: 34,676 • Statistical Power at 10% MDE: 96.2%

Test #220 by Alex James Jan 18, 2019 Desktop Listing X.X% Leads

Alex Tested Pattern #34: Open In A New Tab

Pattern #34: Open In A New Tab - Variant A

Pattern #34: Open In A New Tab - Variant B

This experiment measured the effect of opening new listing (job applications) in a new tab, against opening them in the same window. The experiment A-B was inversed to match the pattern (in reality, the original already opened the tabs in a new window).

Total Sample: 55,703 • Statistical Power at 10% MDE: 99.9%

Test #215 on Vivareal.com.br by Vinicius Barros Peixoto Dec 21, 2018 Mobile Listing X.X% Leads