All Latest 644 A/B Tests

MOST RECENT TESTS

Test #302 on Volders.de by Michal FiechMichal Fiech Jun 09, 2020 Desktop Mobile Signup X.X% Sales

Michal Tested Pattern #83: Progressive Fields On Volders.de

Pattern #83: Progressive Fields - Variant A
Pattern #83: Progressive Fields - Variant B

In this experiment a long form (A) was replaced with a progressive form interaction (B). Most of the form fields would appear in a grey-disabled style, until the prerequioste fields were first filled out.

Total Sample: 35,884 • Statistical Power at 5% MDE: 51.2%

Which A Or B Actually Wins? Find Out Before You Test.

Members see every test result — the winners, the flat ones, and the losers — along with exact effects and sample sizes. Use it to estimate your tests and prioritize by probability, not gut feel. Start every experiment with the odds on your side.

Test #301 on Zapimoveis.com.br by Vinicius Barros PeixotoVinicius Barros Peixoto May 31, 2020 Desktop Mobile Product X.X% Leads

Vinicius Tested Pattern #21: What It's Worth On Zapimoveis.com.br

Pattern #21: What It's Worth - Variant A
Pattern #21: What It's Worth - Variant B

In this experiment, the B variation property prices were framed using higher and crossed out price points from 12 months ago - achieving a relative discount. A tooltip was also shown which explained the higher price point on hover. The example in the screenshot translates to "2% less compared to 12 months ago". This high-power experiment measured the number of leads that were generated on property (product) screens.

Total Sample: 136,682 • Statistical Power at 5% MDE: 97.6%

Test #300 on Volders.de by Michal FiechMichal Fiech May 25, 2020 Desktop Mobile Signup X.X% Progression

Michal Tested Pattern #3: Fewer Form Fields On Volders.de

Pattern #3: Fewer Form Fields - Variant A
Pattern #3: Fewer Form Fields - Variant B

In this experiment, a password field was removed on a contract cancellation form (Volders).

In the control version, users were required to enter their email address and a password. If the email address was associated with an existing account, then the password was used to authenticate the user (and validated). When users entered a new email address, then the password field was used to create a new account. 

In the variation, the password field was removed, as the authentication happened after the conversion itself using other backend mechanisms.

 

Total Sample: 23,356 • Statistical Power at 5% MDE: 83.8%

Test #299 on Backstage.com by Stanley ZuoStanley Zuo May 22, 2020 Desktop Mobile Listing X.X% Sales

Stanley Tested Pattern #60: Repeated Bottom Call To Action On Backstage.com

Pattern #60: Repeated Bottom Call To Action - Variant A
Pattern #60: Repeated Bottom Call To Action - Variant B

In this experiment, at the bottom of a search results screen, a membership join button was added along with 3 encouraging reasons. The experiment measured membership funnel starts, as well as paid membership transactions (sales).

Total Sample: 106,791 • Statistical Power at 5% MDE: 12.2%

Test #298 on Zapimoveis.com.br by Vinicius Barros PeixotoVinicius Barros Peixoto May 14, 2020 Desktop Mobile Listing X.X% Leads

Vinicius Tested Pattern #36: Fewer Or More Results On Zapimoveis.com.br

Pattern #36: Fewer Or More Results - Variant A
Pattern #36: Fewer Or More Results - Variant B

In this experiment on a listing page, the search was expanded to show more listings (variation B). Conditionally, if there were fewer than 36 results, set basic filters such as number bedrooms and bathrooms were expanded and appended to the results. Hence if someone chose 2 bedrooms and 2 bathrooms in variation A, they would only see listing with that filter. In variation B however they would first see the filtered results, and later they would also see results with 3 or more of each. 

Total Sample: 1,907,053 • Statistical Power at 5% MDE: 99.9%

Test #297 on Trydesignlab.com by Daniel ShapiroDaniel Shapiro May 04, 2020 Desktop Home & Landing X.X% Sales

Daniel Tested Pattern #41: Sticky Call To Action On Trydesignlab.com

Pattern #41: Sticky Call To Action - Variant A
Pattern #41: Sticky Call To Action - Variant B

In this experiment, a sticky "Enroll" button was shown on a course landing page. The button lead to a payment funnel to allow enrolling/paying for a course. The exeperiment measured inital progression into this funnel as well as the deeper completed sales metric.

Total Sample: 19,954 • Statistical Power at 5% MDE: 9.2%

Test #296 on Backstage.com by Stanley ZuoStanley Zuo Apr 30, 2020 Mobile Content X.X% Sales

Stanley Tested Pattern #23: Inline Link Nudge On Backstage.com

Pattern #23: Inline Link Nudge - Variant A
Pattern #23: Inline Link Nudge - Variant B

In this simple experiment, a text link to a join page was injected on an article page. The hypothesis was that more users would signup as a result of this subtle trigger.

Total Sample: 411,288 • Statistical Power at 5% MDE: 11%

Test #295 on Thomasnet.com by Julian GaviriaJulian Gaviria Apr 29, 2020 Desktop Mobile Content X.X% Engagement

Julian Tested Pattern #25: Nagging Results On Thomasnet.com

Pattern #25: Nagging Results - Variant A
Pattern #25: Nagging Results - Variant B

In this experiment, blog article pages were hidden behind a registration wall - requing a signup to access. The registration wall would appear after the first paragraph using gradual opacity to cover the rest of the article. We have published the effects of this change on registrations (signups) and on engagement (users viewing other more important company detail pages).

Total Sample: 131,642 • Statistical Power at 5% MDE: 26.7%

Test #294 on Umbraco.com by Lars Skjold IversenLars Skjold Iversen Apr 23, 2020 Desktop Pricing X.X% Progression

Lars Tested Pattern #115: Pricing Comparison Table On Umbraco.com

Pattern #115: Pricing Comparison Table - Variant A
Pattern #115: Pricing Comparison Table - Variant B

In this experiment, plan properties on a pricing page were horizontally aligned (for easier comparison). More so, labels and values were also broken on separate lines.

Total Sample: 18,623 • Statistical Power at 5% MDE: 5.3%

Test #293 on Backstage.com by Stanley ZuoStanley Zuo Apr 14, 2020 Desktop Mobile Product X.X% Sales

Stanley Tested Pattern #114: Less Or More Visible Prices On Backstage.com

Pattern #114: Less Or More Visible Prices - Variant A
Pattern #114: Less Or More Visible Prices - Variant B

In this experiment on a casting call site, pricing information was shown beside the application button. This change shows the effect of setting a price expectation and being more clear that the application process is not free.

Total Sample: 115,061 • Statistical Power at 5% MDE: 9.3%

Test #292 on Backstage.com by Stanley ZuoStanley Zuo Apr 13, 2020 Desktop Mobile Listing X.X% Sales

Stanley Tested Pattern #24: Visible Availability On Backstage.com

Pattern #24: Visible Availability - Variant A
Pattern #24: Visible Availability - Variant B

The core hypothesis of this experiment was that by showing clear availability (in green text) beside each casting call, more users would apply and become premium members. The experiment reports on two metrics: application starts (the first progression metric), and premium membership sales (measured a few steps further in the funnel).

Total Sample: 180,098 • Statistical Power at 5% MDE: 16.8%

Test #291 on Elevate App by Jesse GerminarioJesse Germinario Mar 30, 2020 Mobile Signup X.X% Signups

Jesse Tested Pattern #91: Forced Action

Pattern #91: Forced Action - Variant A
Pattern #91: Forced Action - Variant B

This experiment was ran on the initial onboarding screens of the Elevate App - right after installing and launching the app for the first time. The change was the removal of subtle "skip" links that fast tracked users to the signup/login screen (Get Started). Hence in the variation, all users had to scroll through the 4 introductory messages before being asked to create an account.

Total Sample: 135,631 • Statistical Power at 5% MDE: 96.6%

Test #289 on Prepagent.com by Arthur SparksArthur Sparks Mar 23, 2020 Desktop Pricing X.X% Revenue

Arthur Tested Pattern #17: Least Or Most Expensive First On Prepagent.com

Pattern #17: Least Or Most Expensive First - Variant A
Pattern #17: Least Or Most Expensive First - Variant B

In this experiment, the order of pricing plans was rearranged as to show the most expensive one first.

Test #290 on Prepagent.com by Arthur SparksArthur Sparks Mar 23, 2020 Desktop Pricing X.X% Sales

Arthur Tested Pattern #14: Exposed Menu Options On Prepagent.com

Pattern #14: Exposed Menu Options - Variant A
Pattern #14: Exposed Menu Options - Variant B

In this experiment, a simple pulldown menu (for US state selection) was replaced with all state options shown as selectable buttons. The states were also abbreviated.

Total Sample: 26,029 • Statistical Power at 5% MDE: 14.2%

Test #288 on Kenhub.com by Niels HapkeNiels Hapke Mar 05, 2020 Desktop Mobile Home & Landing X.X% Signups

Niels Tested Pattern #117: Company Logos On Kenhub.com

Pattern #117: Company Logos - Variant A
Pattern #117: Company Logos - Variant B

In this experiment, customer logos (of universities attended by students using Kenhub) were placed on a homepage. The experiment tested for the effect on registration visits, and premium subscription starts.

Total Sample: 36,420 • Statistical Power at 5% MDE: 57.2%

Test #287 on Goodui.org by Jakub LinowskiJakub Linowski Mar 04, 2020 Desktop Mobile Pricing X.X% Sales

Jakub Tested Pattern #117: Company Logos On Goodui.org

Pattern #117: Company Logos - Variant A
Pattern #117: Company Logos - Variant B

In this experiment, a handful of customers and contributors from GoodUI were added on a pricing page to test the effect on sales.

Total Sample: 6,021 • Statistical Power at 5% MDE: 4.4%

Test #286 on Volders.de by Alexander KriegerAlexander Krieger Feb 28, 2020 Desktop Mobile Home & Landing X.X% Sales

Alexander Tested Pattern #9: Multiple Steps On Volders.de

Pattern #9: Multiple Steps - Variant A
Pattern #9: Multiple Steps - Variant B

In this experiment, a long contract cancellation landing page (control) was broken down into 4 steps with 1 final summary step (variation).

Total Sample: 67,999 • Statistical Power at 5% MDE: 75.6%

Test #285 on Ibood.com by Lukas JorissenLukas Jorissen Feb 27, 2020 Desktop Product X.X% Sales

Lukas Tested Pattern #7: Social Counts On Ibood.com

Pattern #7: Social Counts - Variant A
Pattern #7: Social Counts - Variant B

In this experiment, realtime social proof information has been added below an add-to-cart button. The variation shows how many users that have viewed, or placed a product into their basket. Translated to "4 visitors have this product in their shopping cart."

Total Sample: 137,190 • Statistical Power at 5% MDE: 36.7%

Test #284 on Thomasnet.com by Julian GaviriaJulian Gaviria Feb 19, 2020 Desktop Mobile Listing X.X% Leads

Julian Tested Pattern #78: Tags, Badges And Structured Information On Thomasnet.com

Pattern #78: Tags, Badges And Structured Information - Variant A
Pattern #78: Tags, Badges And Structured Information - Variant B

In this experiment, structured data tags were displayed on a listing page to help potential buyers make better decisions. The additional information about the listed companies included: annual revenue, employee count, and year of establishment.

Total Sample: 241,428 • Statistical Power at 5% MDE: 13%

Test #283 on Kenhub.com by Niels HapkeNiels Hapke Feb 08, 2020 Desktop Mobile Global X.X% Sales

Niels Tested Pattern #42: Countdown Timer On Kenhub.com

Pattern #42: Countdown Timer - Variant A
Pattern #42: Countdown Timer - Variant B

In this experiment, registered trial users were shown a 65 minute counter on multiple pages (dashboard, listing, quiz, articles) encouraging them to get a full subscription and study all content. Both A and B experiences offered the same limited content for trial users. After the 65 minutes came to an end, the B variation showed an additional "Go Premium" button on all pages, but continued to offer the same limited content.

Total Sample: 100,433 • Statistical Power at 5% MDE: 56.9%