All Latest 633 A/B Tests

MOST RECENT TESTS

Test #237 on Goodui.org by Jakub LinowskiJakub Linowski Apr 29, 2019 Desktop Mobile Listing X.X% Sales

Jakub Tested Pattern #103: Money Back Guarantee On Goodui.org

- Variant A
- Variant B
Total Sample: 21,096 • Statistical Power at 5% MDE: 6%

Which A Or B Actually Wins? Find Out Before You Test.

Members see every test result — the winners, the flat ones, and the losers — along with exact effects and sample sizes. Use it to estimate your tests and prioritize by probability, not gut feel. Start every experiment with the odds on your side.

Get Access to All Test Results

Test #234 on by Alex JamesAlex James Apr 01, 2019 Desktop Listing X.X% Leads

Alex Tested Pattern #101: Search Keyword Highlighting

- Variant A
- Variant B

In this experiment, a different style for keyword highlighting was used.
Total Sample: 738,295 • Statistical Power at 5% MDE: 99.9%

Test #230 on Goodui.org by Jakub LinowskiJakub Linowski Mar 09, 2019 Desktop Mobile Listing X.X% Engagement

Jakub Tested Pattern #56: Hover Button On Goodui.org

- Variant A
- Variant B

In this test we tested onhover buttons (variant) versus more traditional always exposed and visible ones.
Total Sample: 30,699 • Statistical Power at 5% MDE: 98.1%

Test #222 on Thomasnet.com by Julian GaviriaJulian Gaviria Feb 01, 2019 Desktop Listing X.X% Leads

Julian Tested Pattern #7: Social Counts On Thomasnet.com

- Variant A
- Variant B

In this variation, a number of social proof references were added to a signup modal.
Total Sample: 34,676 • Statistical Power at 5% MDE: 47.1%

Test #220 on by Alex JamesAlex James Jan 18, 2019 Desktop Listing X.X% Leads

Alex Tested Pattern #34: Open In A New Tab

- Variant A
- Variant B

This experiment measured the effect of opening new listing (job applications) in a new tab, against opening them in the same window. The experiment A-B was inversed to match the pattern (in reality, the original already opened the tabs in a new window).
Total Sample: 55,703 • Statistical Power at 5% MDE: 99.9%

Test #215 on Vivareal.com.br by Vinicius Barros PeixotoVinicius Barros Peixoto Dec 21, 2018 Mobile Listing X.X% Leads

Vinicius Tested Pattern #92: Already Viewed Label On Vivareal.com.br

- Variant A
- Variant B

The idea of this test was to add a "Viewed" label on a listing page to indicate listings which have already been viewed by users.
Total Sample: 2,452,026 • Statistical Power at 5% MDE: 99.9%

Test #209 on Vivareal.com.br by Vinicius Barros PeixotoVinicius Barros Peixoto Nov 12, 2018 Mobile Listing X.X% Leads

Vinicius Tested Pattern #34: Open In A New Tab On Vivareal.com.br

- Variant A
- Variant B

The idea of this experiment was taking advantage of mobile browser behavior. When a link is open in a new tab on mobile browsers, and users hit the back button, the tab closes and users get back exactly where they were before without any new result page load.
Total Sample: 778,955 • Statistical Power at 5% MDE: 99.9%

Test #208 on Thomasnet.com by Julian GaviriaJulian Gaviria Nov 02, 2018 Desktop Mobile Listing X.X% Leads

Julian Tested Pattern #88: Action Button On Thomasnet.com

- Variant A
- Variant B

In this variation, the button labels were changed from "Profile" to "View Supplier".
Total Sample: 39,170 • Statistical Power at 5% MDE: 7.2%

Test #196 on Vivareal.com.br by Vinicius Barros PeixotoVinicius Barros Peixoto Aug 14, 2018 Mobile Listing X.X% Leads

Vinicius Tested Pattern #80: Persistent Filters On Vivareal.com.br

- Variant A
- Variant B

The experiment goal was automatically applying filters the users have already done in our result page, during their navigation to the site. The variation always applied the filters in the same session and asked users on new sessions.
Total Sample: 441,738 • Statistical Power at 5% MDE: 99.9%

Test #195 on Yummly.com by Kimberly CheungKimberly Cheung Aug 13, 2018 Desktop Mobile Listing X.X% Signups

Kimberly Tested Pattern #78: Tags, Badges And Structured Information On Yummly.com

- Variant A
- Variant B
Total Sample: 371,274 • Statistical Power at 5% MDE: 99.9%

Test #185 on Retrome.pl by Grzegorz JancewiczGrzegorz Jancewicz Jul 03, 2018 Mobile Listing X.X% Leads

Grzegorz Tested Pattern #49: Above The Fold Call To Action On Retrome.pl

- Variant A
- Variant B
Total Sample: 823 • Statistical Power at 5% MDE: 4%

Test #182 on Yummly.com by Marcos CiarrocchiMarcos Ciarrocchi Jun 12, 2018 Desktop Listing X.X% Engagement

Marcos Tested Pattern #6: Customer Star Ratings On Yummly.com

- Variant A
- Variant B
Total Sample: 673,296 • Statistical Power at 5% MDE: 99.9%

Test #179 on Yummly.com by Marcos CiarrocchiMarcos Ciarrocchi May 18, 2018 Desktop Listing X.X% Signups

Marcos Tested Pattern #13: Centered Forms & Buttons On Yummly.com

- Variant A
- Variant B
Total Sample: 391,537 • Statistical Power at 5% MDE: 99.5%

Test #169 on Mt.com by Vito MediavillaVito Mediavilla Apr 24, 2018 Desktop Listing X.X% Leads

Vito Tested Pattern #37: List Or Grid View On Mt.com

- Variant A
- Variant B
Total Sample: 5,934 • Statistical Power at 5% MDE: 4.5%

Test #168 on Akademiafotografii.p... by Grzegorz JancewiczGrzegorz Jancewicz Apr 23, 2018 Desktop Listing X.X% Leads

Grzegorz Tested Pattern #45: Benefit Bar On Akademiafotografii.p...

- Variant A
- Variant B
Total Sample: 4,302 • Statistical Power at 5% MDE: 5.5%

Test #156 on Mt.com by Vito MediavillaVito Mediavilla Feb 25, 2018 Desktop Listing X.X% Leads

Vito Tested Pattern #60: Repeated Bottom Call To Action On Mt.com

- Variant A
- Variant B

This test duplicated two buttons at the bottom of the page. However, the site already contained floating buttons (from the header).
Total Sample: 27,048 • Statistical Power at 5% MDE: 12.8%

Test #143 on Vivareal.com.br by Rodrigo MauésRodrigo Maués Jan 14, 2018 Desktop Listing X.X% Leads

Rodrigo Tested Pattern #56: Hover Button On Vivareal.com.br

- Variant A
- Variant B
Total Sample: 2,594,182 • Statistical Power at 5% MDE: 99.9%

Test #133 on Bing.com by Ronny KohaviRonny Kohavi Dec 13, 2017 Desktop Mobile Listing X.X% Revenue

Ronny Tested Pattern #43: Long Titles On Bing.com

- Variant A
- Variant B

In 2012 a Microsoft employee working on Bing had an idea about changing the way the search engine displayed ad headlines. Developing it wouldn’t require much effort—just a few days of an engineer’s time—but it was one of hundreds of ideas proposed, and the program managers deemed it a low priority. So it languished for more than six months, until an engineer, who saw that the cost of writing the code for it would be small, launched a simple online controlled experiment—an A/B test—to assess its impact. Within hours the new headline variation was producing abnormally high revenue, triggering a “too good to be true” alert.

HBR, September–October 2017 Issue, https://hbr.org/2017/09/the-surprising-power-of-online-experiments

Note: This experiment was a solid success and replicated multiple times over a period of months. It worked at Bing and had a profound influence. The only reason why we atributed a 0.25 point (a "Maybe") was because we don't have the exact sample size and conversion data.

 

Test #127 on Glass.net by Mark FreedleMark Freedle Nov 23, 2017 Desktop Mobile Listing X.X% Leads

Mark Tested Pattern #42: Countdown Timer On Glass.net

- Variant A
- Variant B
Total Sample: 61,034 • Statistical Power at 5% MDE: 23.7%

Test #119 on Vivareal.com.br by Rodrigo MauésRodrigo Maués Nov 01, 2017 Desktop Listing X.X% Leads

Rodrigo Tested Pattern #36: Fewer Or More Results On Vivareal.com.br

- Variant A
- Variant B
Total Sample: 2,452,622 • Statistical Power at 5% MDE: 99.9%