Let us take a look at the variables involved in interpreting the results:
- Control and Test Variant - This is your Variant A (where you cannot make any edits) and Variant B where you can change the following: Product Name, Price, Product Description, and Product Photo/s.
- Unique user page view - Page views that are generated by the unique users during the same session. If a single user views the product page twice or more, it will only count as 1.
- Product page views - This shows how many times the product page was viewed for the product being tested. If the same user views the page twice, it will count as 2 (and so on). This is a good unit of measure when testing product name, product description, product photos, or prices because you can see if changes that were done are attracting users to click on the product more.
- Add to Cart - This shows you how many times each variant was added to a buyers' shopping cart.
- Units sold - The number of products sold
- Revenue - The total amount each variant has produced during the testing period.
- Revenue per view -This is defined as revenue divided by the number of views
Please note that the suggested minimum test duration is 2 weeks. This is designed to make sure the results are not affected by the day of the week and cycles.
Now, after the testing, you'd ask - Why did customers behave the way they did? Why did they react a certain way with one version and not with the other? Did they have a better customer experience with your control variant or your test variant? Were they driven by the price change? Were they more attracted by the new photos that were added? Are they inclined to purchase the item with a single simple photo?
This is the reason why we recommend that you change one product attribute per test, this way, it could help you set apart components that cause the results to change and help you make an informed decision in picking the winner. If you test on a number of attributes at the same time, you won't be able to identify which factor caused the change.
What are the variables I need to consider in checking the results?
How do I pick a winner?