Reduced Mobile Homepage Content

Problem

The homepage has a lot of competing information. It features a selection of heroic imagery that helps reinforce seasonal brand storytelling but, when that content is engaged with, leads to low conversion rates. How do we balance minimizing competing information on the homepage while still maximizing brand storytelling opportunities?

Hypothesis

If we reduce the content on the homepage, customers will be more focused on the content presented. If we are right, we will see increased engagement on the remaining homepage content.

Impact

This test ended with V2 as a winner. Overall, we found that the short mobile homepage with 0 content boxes presented a much better experience for users as all metrics saw positive lift. Not only did our primary metric of PDP views increase, but even order conversion saw strong positive lift at confidence.

We ended up pausing V1 after a few days due to consistent poor performance. We found that the click rate for content boxes in the control was relatively low at just under 2%. Additionally, the add to bag rate and order conversion rate coming out of content box clicks were both lower than what we were seeing in V2 — suggesting the content boxes didn't provide much incentive to order.

Estimated Revenue Unlocked:

Estimated annual revenue impact is based on general audience data from January 1, 2022 to January 1, 2023, where there were 132,948 mobile transactions from the homepage with an average order value of $213.63.

[Visits with an order last year] 132,948 × [Observed Conversion Rate Lift] 13.14% = [Additional Annual Orders] 17,469

[Additional Annual Orders] 17,469 × [Average Order Value] $213.63 = [Estimated Revenue Lift] $3,731,894