We achieved a 9% uplift of ARPU (CR uplift of 4% and AOV uplift of 9.5%) by introducing a cross-sell carousel in the slide-in cart.
Moonmagic.com is a fast growing online jewelry store that has 430K sessions per month (according to Similarweb). It sells rings, necklaces, earrings and other products in this category.
Our task was to increase the revenue per user (ARPU) metric by growing the conversion rate (CR) and average order value (AOV).
We looked at the revenue breakdown to identify key revenue drivers. We found that around 25% of buyers purchase more than one product per transaction. We concluded that users have a significant propensity to purchase more than one item per transaction.
So the idea at this point was to encourage purchases of multiple items in each transaction.
As a result of a heuristic UX analysis we found real estate in the cart to test a cross-sell carousel.
Next, we needed a recommendations engine to use in our cross-sell carousel. A data-driven recommendations engine would work best for this purpose, so we decided to create one. We created a regression matrix to identify various combinations of products often bought together:
We found that there were over 25 combinations of products that are often bought together.
Incorporating these combinations into our experiment would significantly complicate it. Moreover, the role of an A/B testing is to validate one hypothesis at a time. Adding the carousel to the cart was our primary hypothesis, so we decided to use the already built-in recommendation engine on the site. There were two built-in recommendation engines functioning on the site. We created a regression model to identify which one of them drives more sales. We found that users who interacted with one of them converted x3.6 better than the site average. This was our winner.
We hypothesized that by introducing a cross-sell carousel into the cart we will increase the average number of items purchased per transaction and increase the AOV.
The risk that we were running with this hypothesis is that by distracting users from conversion in the cart and leading them to other PDPs, we may negatively impact the CR. However, even if this would be the case we expected the growth in AOV to compensate for the decrease in CR.
So our solution was to introduce a cross-sell carousel in the cart.
To validate this hypothesis we conducted an experiment that measured the impact of the cross-sell carousel in the cart on the CR, AOV and ultimately ARPU.
Experiment type: A/B test
Traffic split: 50:50
Key metric: ARPU
Targeting condition: We targeted the test only toward users who added a product to the cart and viewed the cart. This was done to minimize data noise and achieve statistical significance of the result quicker.
Number of users taking part in the experiment: 2 518 users
Number of conversions recorded during the experiment: 694 conversions
The alternative variation that introduce a cross-sell carousel in the cart has shown a 9% uplift in ARPU, 4% uplift in CR and 9% uplift in AOV. The result had 95% statistical significance and the power of the experiment of 80%.
So users who experienced the alternative variation not only purchased more products per transaction elevating the AOV, but also were more likely to complete the purchase positively influencing the CR.