All Latest 633 A/B Tests

MOST RECENT TESTS

Test #519 on Volders.de by Katharina LayKatharina Lay Feb 23, 2024 Desktop Mobile Checkout X.X% Sales

Katharina Tested Pattern #134: Optional or Confident Recommendation On Volders.de

 - Variant A
 - Variant B

In this experiment, copy around an upsell was changed from using "optional" to "our recommendation for you". The idea was to recommend two upsells with more confidence. Impact on the two upsells (secure, and express) as well as overall transactions was measured.

Total Sample: 58,597 Statistical Power at 5% MDE: 99.9%

Which A Or B Actually Wins? Find Out Before You Test.

Members see every test result — the winners, the flat ones, and the losers — along with exact effects and sample sizes. Use it to estimate your tests and prioritize by probability, not gut feel. Start every experiment with the odds on your side.

Test #518 on by Jakub LinowskiJakub Linowski Feb 14, 2024 Mobile Checkout X.X% Sales

Jakub Tested Pattern #64: Tunnel

 - Variant A
 - Variant B

This was an experiment on the first checkout step where users would enter shipping information. The control had a longer footer with more additional sitewide links and a call to action to a newsletter. The variation removed these elements and kept the focus on the shipping information task. Impact on progression to next step and sales was measured.

Total Sample: 10,900 Statistical Power at 5% MDE: 85.8%

Test #517 on Aboalarm.de by Daria KurchinskaiaDaria Kurchinskaia Feb 12, 2024 Desktop Checkout X.X% Sales

Daria Tested Pattern #78: Tags, Badges And Structured Information On Aboalarm.de

 - Variant A
 - Variant B

In this experiment, a "most popular" badge ("am beliebsten") was appended to the middle plan of a contract cancellation service. Impact on overall purchases and revenue was measured.

Total Sample: 127,712 Statistical Power at 5% MDE: 94.9%

Test #483 on Menufy.com by Aleksandr ElesevAleksandr Elesev Jul 17, 2023 Desktop Checkout X.X% Sales

Aleksandr Tested Pattern #124: Confirmed Selection On Menufy.com

 - Variant A
 - Variant B

In this experiment, a thanking confirmation message was appended at the top of the checkout screen of a local food delivery service. Impact on completed transactions was measured.

Total Sample: 8,626 Statistical Power at 5% MDE: 99.9%

Test #481 on Backstage.com by Stanley ZuoStanley Zuo Jul 14, 2023 Desktop Mobile Checkout X.X% Sales

Stanley Tested Pattern #15: Bulleted Reassurances On Backstage.com

 - Variant A
 - Variant B

In this experiment, selling points and benefits of a subscription were placed as bullets at the top of a checkout page. The benefits highlighted things such as: unlimited applications, access to vetted jobs and the ability to cancel anytime. Impact on sales was measured.

Total Sample: 5,616 Statistical Power at 5% MDE: 24.3%

Test #482 on by Jakub LinowskiJakub Linowski Jul 13, 2023 Desktop Mobile Checkout X.X% Sales

Jakub Tested Pattern #124: Confirmed Selection

 - Variant A
 - Variant B

In this experiment, the choice of adding a product to cart was confirmed with a further positive message in the header of the next step (on the add to cart page). Once users left the product detail page, instead of simply stating the product name, the title was rephrased as "Product [X] Makes a Great Gift. They'll Love It!". I view this as a higher "intensity" experiment, given that the add-to-cart page was in some way already confirming the choice. Impact on sales was measured.

Total Sample: 3,379 Statistical Power at 5% MDE: 44%

Test #480 on Aboalarm.de by Daria KurchinskaiaDaria Kurchinskaia Jun 15, 2023 Desktop Mobile Checkout X.X% Sales

Daria Tested Pattern #15: Bulleted Reassurances On Aboalarm.de

 - Variant A
 - Variant B

In this experiment, a list of benefits were shown for each of the 3 plans on the last step of a contract cancelation service. The lowest plan only had one benefit, whereas the highest plan had 3. Impact on transactions was measured. 

Total Sample: 104,902 Statistical Power at 5% MDE: 93%

Test #479 on Aboalarm.de by Daria KurchinskaiaDaria Kurchinskaia Jun 15, 2023 Desktop Mobile Checkout X.X% Sales

Daria Tested Pattern #15: Bulleted Reassurances On Aboalarm.de

 - Variant A
 - Variant B

In this experiment, a list of benefits were shown for each of the 3 plans on the last step of a contract cancelation service. Benefit items not included in the lower plans were also shown with grayed out styles (and an "x"). Clearly the higher paid plan had all the benefits listed. Impact on transactions was measured. 

Total Sample: 104,863 Statistical Power at 5% MDE: 93%

Test #463 on Volders.de by Daria KurchinskaiaDaria Kurchinskaia Mar 25, 2023 Desktop Checkout X.X% Sales

Daria Tested Pattern #115: Pricing Comparison Table On Volders.de

 - Variant A
 - Variant B

This experiment explored a pricing layout that enabled more feature comparisons. It also conveyed more clearly which features were missing between plans. The test has been inspired by this Netflix experiment. Impact on sales was measured.

Total Sample: 23,336 Statistical Power at 5% MDE: 84.8%

Test #458 on Volders.de by Daria KurchinskaiaDaria Kurchinskaia Feb 27, 2023 Desktop Mobile Checkout X.X% Sales

Daria Tested Pattern #103: Money Back Guarantee On Volders.de

 - Variant A
 - Variant B

In this experiment, a cancelation guarantee was added believing it would make users feel safer while canceling their contracts with Volders (the paid service being offered). The variation appended a Guarantee in the headline as a hyperlink with an explanatory tooltip shown on hover. This variation change was added to multiple screens throughout the checkout flow (a 5 step process).

Total Sample: 21,648 Statistical Power at 5% MDE: 31.6%

Test #452 on Volders.de by Daria KurchinskaiaDaria Kurchinskaia Jan 30, 2023 Desktop Mobile Checkout X.X% Sales

Daria Tested Pattern #62: Urgent Next Day Delivery On Volders.de

 - Variant A
 - Variant B

In this experiment, a count down timer was added near the top of a checkout page. The timer was only shown before 1pm and clarified that the serivce (contract cancellation) will be initiated on the same day if users act before a cut off time. Impact on completed payments was measured.

Total Sample: 40,999 Statistical Power at 5% MDE: 99%

Test #435 on Volders.de by Daria KurchinskaiaDaria Kurchinskaia Oct 17, 2022 Desktop Mobile Checkout X.X% Sales

Daria Tested Pattern #9: Multiple Steps On Volders.de

 - Variant A
 - Variant B

In this experiment, a question about a customer's reason for purchase was broken out into a separate step and moved earlier in the process. In the control, this question was asked in the final checkout step along with a plan selection (Step 4 of 4). In the variation, this question was shifted as a standalone first step (Step 1 of 5). Impact on completed purchases was measured (for a contract cancellation service in this case).

Total Sample: 139,608 Statistical Power at 5% MDE: 98.2%

Test #427 on Designlab.com by Daniel ShapiroDaniel Shapiro Aug 10, 2022 Desktop Mobile Checkout X.X% Sales

Daniel Tested Pattern #28: Easiest Fields First On Designlab.com

 - Variant A
 - Variant B

In this experiment, the course enrollment start date was moved from step 2 to step 1 of an enrollment / checkout flow. The test was run by Designlab - that offers design courses and education with a strong element of mentorship. Impact on progression to next step and completed transactions were measured.

Total Sample: 12,185 Statistical Power at 5% MDE: 15.3%

Test #415 on Learnwithhomer.com by Stanley ZuoStanley Zuo Jun 09, 2022 Mobile Checkout X.X% Signups

Stanley Tested Pattern #3: Fewer Form Fields On Learnwithhomer.com

 - Variant A
 - Variant B

Do fewer confirmation form fields matter? In this experiment, redundant password and email confirmation fields were removed during a signup / checkout funnel. Impact on signups was measured.

Total Sample: 2,286 Statistical Power at 5% MDE: 5.2%

Test #414 on Volders.de by Frederik FröhleFrederik Fröhle May 31, 2022 Desktop Mobile Checkout X.X% Sales

Frederik Tested Pattern #98: Auto Suggest On Volders.de

 - Variant A
 - Variant B

Does adding Google's address auto complete functionality to an address field help with higher form completions? This auto fill feature has been tested in the variation of a contract cancellation funnel. After selecting an auto completed address from a pulldown menu, the following fields were preselected: house number, zip code, city and country (potentially lowering friction?). Impact on successful form completions (contract cancellations) has been measured. Notice how the form also expanded progressively upon selecting the complete address in the variation.  

Total Sample: 128,199 Statistical Power at 5% MDE: 97.4%

Test #412 on Volders.de by Frederik FröhleFrederik Fröhle May 16, 2022 Desktop Mobile Checkout X.X% Sales

Frederik Tested Pattern #15: Bulleted Reassurances On Volders.de

 - Variant A
 - Variant B

The Volders experimentation team assumed that adding information about how long it might take to get a cancellation confirmation near a CTA Button would result in higher cancellation requests (the paid service being offered).

The variation contained additional copy translated from German to: "Most of our users receive their cancellation confirmation from <vendor> within 14 days by email or letter."

Total Sample: 53,120 Statistical Power at 5% MDE: 99.3%

Test #403 on by Jakub LinowskiJakub Linowski Mar 29, 2022 Desktop Mobile Checkout X.X% Sales

Jakub Tested Pattern #42: Countdown Timer

 - Variant A
 - Variant B

In this experiment, an urgency related message with a dynamic countdown timer was added on the final checkout screen. Impact on sales was measured.

Total Sample: 7,466 Statistical Power at 5% MDE: 99.9%

Test #399 on by Jakub LinowskiJakub Linowski Feb 27, 2022 Desktop Mobile Checkout X.X% Progression

Jakub Tested Pattern #35: Floating Labels

 - Variant A
 - Variant B

In this experiment, top-aligned field labels were tested against floating labels (with labels floating inside the form field itself).

Total Sample: 2,073 Statistical Power at 5% MDE: 29.2%

Test #390 on Snocks.com by Melina HessMelina Hess Dec 21, 2021 Desktop Mobile Checkout X.X% Sales

Melina Tested Pattern #106: Back Buttons On Snocks.com

 - Variant A
 - Variant B

In this experiment, the variation has a "Back To Shopping Cart" link right underneath the checkout button. Impact on sales was measured.

Total Sample: 22,852 Statistical Power at 5% MDE: 97.5%

Test #389 on Svsound.com by Keenan DavisKeenan Davis Dec 16, 2021 Desktop Mobile Checkout X.X% Sales

Keenan Tested Pattern #1: Remove Coupon Fields On Svsound.com

 - Variant A
 - Variant B

In this simple experiment, a highly visible coupon field was replaced with a less visible (but clickable) link in the variation. Clicking on the link would show the coupon field. Impact on sales and revenue was measured.

Total Sample: 20,321 Statistical Power at 5% MDE: 37.2%