n11.com Basket Page: Buy Later Optimization Test
Conducting multivariate and A/B tests for the “Buy Later” function instead of “Add to Favourites” on the basket page.
The Challenge
N11.com is one of the biggest e-commerce platforms in Turkey. It sells 20 million products in different categories and has more than 23 million users.
For suggesting improvement of the basket page of n11.com; in-depth interviews, usability tests, online surveys, google analytics findings were reviewed.
​
My Role
I was the lead user experience designer in these tests and I've worked with two optimization specialists to conduct it.
The Discovery
Survey Results
According to the one survey result, there were some remarkable findings on the basket page;
-
According to a survey, nearly half of the users keep the products in their baskets for 1-2 weeks.
-
Users add products to the basket and use it as a favourite and comparison page.
-
Apart from this, they have motivations to see the coupon validity and the total price more clearly.
From different survey results;
-
80% of users wanted to reach the product they deleted from the basket page.
-
60% of the users wanted to see the products on the basket page even after they deleted them.
-
A half of these users want to see the deleted products in separate locations instead of my favourites or my lists.
Problem Definition
No matter how much the user wants to use the basket as a comparison, favourites page, the maximum number of products added to the basket is 20. It means users need to delete old products to add new products to the basket or add them to favourites not to lose them.
The Tests
The Strategy of the Test
Primary KPI’s
-
Increasing "buy later" function use and turning users into returning users
-
Increasing the motivation for the next purchase
Secondary KPI’s
-
Increasing the number of login users (If users use “buy later”, they have to log in at first.)
Goals
-
Increasing the usage of “buy later” functions by showing it more visible using texts and icons
-
In the same way, increasing the number of users who reach their deleted products from “buy later” lists.
According to the research findings, "Add to Favourites" is used for products wanted to purchase later. We also wondered how the name change in this function would affect its use, and we changed the name of the "Add to Favourites" to "Buy Later".
Multivariate Test - Web
On the original Web design, "Add to Favourites" is shown as text and a heart icon. The "Delete" was positioned to the left of this function with its icon.
For this study, we wanted to test 4 different variants.
Multivariate Test Results -Web
According to the test results, the “buy later” version, which we left in its original location (Variant 1), received 29% more engagement than the original version with “add to favourites”.
​
In addition, when we used the delete icon with blue colour to ensure consistency, we saw that its usage increased by 5% as its visibility increased.
When we examined the effect of “delete” action usage on e-commerce actions, There was no adverse effect on the transaction, revenue, average order value and checkout in the alternative version.
​
Note: The product planned to be "Buy later" is transferred to the module created below: "The products that I will buy later" on the basket page, but could not be included in this test because it needed development.
Multivariate Test - mWeb
Changed "add to favourite" with a heart icon in the basket to "buy later" with another icon.
​​
Note: Unfortunately, we could not perform a test on iOS and Android as we did not have a test tool for them at that time.
​
We tested 3 versions;
Multivariate Test Results -mWeb
According to the results, although there is no clear winner of the test, Variant 3 worked 154% better than the original version.
Since we also interfered with the "delete" action, we set it as the secondary goal. In the version with the blue that makes the most difference, the delete action has also increased by 19%.
When we look at the effect of the "delete" action on e-commerce actions, we did not observe an adverse change in the transaction, revenue, avg order and checkout in the alternative version.
​
There is a heart icon at the top right corner on the current basket page to access the "my favourites" page. This icon was replaced with the "buy later" icon.
When we examined Google Analytics, we saw that the transition from the basket to favourites was 36% more in the alternative version.
​
In addition, we saw that the action of adding to basket from favourites was 10% more in the alternative version. However, we could not see how much percentage was from the basket page.
The Validation
A/B Tests (Confidence Tests)
Based on our results, we wanted to conduct a "confidence test" on both channels. The winning versions were retested with the original.
​
The Strategy: Same as Multivariate Test
The Goal: Same as Multivariate Test
Channels: Web and mWeb
A/B Test -Web
A/B Test -mWeb
A/B Test Results
Web
Use of "save for later" on the web increased by 47%. The use of the "delete" function has not changed. E-commerce metrics were not affected negatively.
mWeb
On the mWeb, the use of "buy later" increased by 124%. Accessing "my favourites" (Actually, it also includes "save for later" products) increased by 37%.
The Take Away
-
We suggest strengthening personalisation functions by adding the “ buy later” function on the homepage personalisation dashboard and making it easy to find for users on their account pages.
-
We continue to plan tests for iOS and Android. Since the location of the gestures and functions are different, we will proceed as tests of different experiences.
-
Since we did not reduce e-commerce metrics by increasing the usage of the "buy later" function, we suggested that the winning variations should be live.
​
Thank you for reading!