All Latest 608 A/B Tests

Screen

Device

Metric

Tag

Power of 80%+ for

Only Significant

p-value < 0.05

Become a member to unlock the abiltiy to see the highest impact a/b tests. Being able to see the actual test results and sort by impact allows growth and experimentation teams to take action on the biggest gains first

MOST RECENT TESTS

Test #220 on by Alex James Jan 18, 2019 Desktop Listing

Alex Tested Pattern #34: Open In A New Tab

This experiment measured the effect of opening new listing (job applications) in a new tab, against opening them in the same window. The experiment A-B was inversed to match the pattern (in reality, the original already opened the tabs in a new window).

Test #215 on Vivareal.com.br by Vinicius Barros Peixoto Dec 21, 2018 Mobile Listing

Vinicius Tested Pattern #92: Already Viewed Label On Vivareal.com.br

The idea of this test was to add a "Viewed" label on a listing page to indicate listings which have already been viewed by users.

Test #209 on Vivareal.com.br by Vinicius Barros Peixoto Nov 12, 2018 Mobile Listing

Vinicius Tested Pattern #34: Open In A New Tab On Vivareal.com.br

The idea of this experiment was taking advantage of mobile browser behavior. When a link is open in a new tab on mobile browsers, and users hit the back button, the tab closes and users get back exactly where they were before without any new result page load.

Test #208 on Thomasnet.com by Julian Gaviria Nov 02, 2018 Desktop Mobile Listing

Julian Tested Pattern #88: Action Button On Thomasnet.com

In this variation, the button labels were changed from "Profile" to "View Supplier".

Test #196 on Vivareal.com.br by Vinicius Barros Peixoto Aug 14, 2018 Mobile Listing

Vinicius Tested Pattern #80: Persistent Filters On Vivareal.com.br

The experiment goal was automatically applying filters the users have already done in our result page, during their navigation to the site. The variation always applied the filters in the same session and asked users on new sessions.

Test #195 on Yummly.com by Kimberly Cheung Aug 13, 2018 Desktop Mobile Listing

Kimberly Tested Pattern #78: Tags, Badges And Structured Information On Yummly.com

Test #185 on Retrome.pl by Grzegorz Jancewicz Jul 03, 2018 Mobile Listing

Grzegorz Tested Pattern #49: Above The Fold Call To Action On Retrome.pl

Test #182 on Yummly.com by Marcos Ciarrocchi Jun 12, 2018 Desktop Listing

Marcos Tested Pattern #6: Customer Star Ratings On Yummly.com

Test #179 on Yummly.com by Marcos Ciarrocchi May 18, 2018 Desktop Listing

Marcos Tested Pattern #13: Centered Forms & Buttons On Yummly.com

Test #169 on Mt.com by Vito Mediavilla Apr 24, 2018 Desktop Listing

Vito Tested Pattern #37: List Or Grid View On Mt.com

Test #168 on Akademiafotografii.p... by Grzegorz Jancewicz Apr 23, 2018 Desktop Listing

Grzegorz Tested Pattern #45: Benefit Bar On Akademiafotografii.p...

Test #156 on Mt.com by Vito Mediavilla Feb 25, 2018 Desktop Listing

Vito Tested Pattern #60: Repeated Bottom Call To Action On Mt.com

This test duplicated two buttons at the bottom of the page. However, the site already contained floating buttons (from the header).

Test #143 on Vivareal.com.br by Rodrigo Maués Jan 14, 2018 Desktop Listing

Rodrigo Tested Pattern #56: Hover Button On Vivareal.com.br

Test #133 on Bing.com by Ronny Kohavi Dec 13, 2017 Desktop Mobile Listing

Ronny Tested Pattern #43: Long Titles On Bing.com

In 2012 a Microsoft employee working on Bing had an idea about changing the way the search engine displayed ad headlines. Developing it wouldn’t require much effort—just a few days of an engineer’s time—but it was one of hundreds of ideas proposed, and the program managers deemed it a low priority. So it languished for more than six months, until an engineer, who saw that the cost of writing the code for it would be small, launched a simple online controlled experiment—an A/B test—to assess its impact. Within hours the new headline variation was producing abnormally high revenue, triggering a “too good to be true” alert.

HBR, September–October 2017 Issue, https://hbr.org/2017/09/the-surprising-power-of-online-experiments

Note: This experiment was a solid success and replicated multiple times over a period of months. It worked at Bing and had a profound influence. The only reason why we atributed a 0.25 point (a "Maybe") was because we don't have the exact sample size and conversion data.

Test #127 on Glass.net by Mark Freedle Nov 23, 2017 Desktop Mobile Listing