All Latest 617 A/B Tests

This experiment ran on desktop only, on all our regular product listing and search result page. In the control, clicking on products would open them up in the same page, whereas in the variation, the links opened in a new tab. Impact on adds-to-cart were measured.

This experiment ran on desktop only, on all our regular product listing and search result page. In the control, clicking on products would open them up in the same page, whereas in the variation, the links opened in a new tab. Impact on adds-to-cart were measured.

In this experiment, more popular search categories (pills) were shown on the homepage based on analyzed search volume. Impact on searches and email subscriptions were measured.

The A version showed a non-sticky checkout button at the bottom of a flyout cart. The B variation added a sticky bar with a price summary and checkout button, keeping it always visible. Impact on sales was measured.

The control had a clickable link "Use discount code" which would reveal an input field for a coupon code. The variation removed this link and input field. Impact on purchases was measured. (Desktop only)

The control had a clickable link "Use discount code" which would reveal an input field for a coupon code. The variation removed this link and input field. Impact on purchases was measured. (Mobile only)

A floating Add to Basket button was added to a product page. Impact on sales was measured.

In this experiment, the sorting link (defaulting to popular) was swapped with the filter one. In the control, the sorting appeared on the left with the filter on the right, whereas in the variation these two were flipped. Impact on adds to cart and sales were measured.

The control had a clickable link "Use discount code" which would reveal an input field for a coupon code. The variation removed this link and input field. Impact on purchases was measured. (Mobile only)

The control had a clickable link "Use discount code" which would reveal an input field for a coupon code. The variation removed this link and input field. Impact on purchases was measured. (Desktop only)

In this experiment, the shopping cart screen displayed a floating “Checkout” button with common credit card icons that directed users to the checkout page when clicked. In the control, the button was positioned inline, further down the page. Impact on progression to checkout and total sales were measured.

In this experiment, a pricing table (an upsell of product protection plan coverages) was tested against a more explicit yes/no toggle for including the plan. Impact was measured on adds to cart, orders, and upsell rate, using an 80/20 traffic split.

In this experiment (1) the zip code field position was moved up, right below the last name. And (2) entering the zip code would populate the state and city using an autofill API call. Impact on orders completed was measured.

In this experiment (1) the zip code field position was moved up, right below the last name. And (2) entering the zip code would populate the state and city using an autofill API call. Impact on orders completed was measured.

In this tightly triggered experiment, users who (1) did not complete a purchase and (2) visited a different product page saw a button in the top-right corner labeled “Don’t Forget Your Club.” Clicking this button resumed the checkout process at the point where they left off, without requiring the same information to be re-entered.

In this experiment, search results and listing pages received two additional filters to remove out-of-stock and in-store-only items. This reduced the number of results shown by default. The impact on add-to-cart actions, checkout flows, and completed sales was measured.

In this experiment, a floating "Confirm Payment" button was added on the last step of a checkout flow. The button appeared with two different states: in a disabled state, leading to the terms and conditions; and in an active state, after checking off the terms. Impact on purchases was measured. 

In this experiment, a breadcrumb was replaced with a circular progress bar (arguably more visible overall steps; but less visible/clickable accomplished steps). Impact on transactions was measured.

In this experiment, the homepage interaction, headline, and images were changed. In the control, users saw a single “Shop Now” button leading directly to products. In the variation, users first chose between “Reset” or “Lose Weight,” then selected a duration in days before being shown products. The impact on sales was measured.

A progress bar was added to the top of a 3 step signup funnel. Impact on completed orders was measured.