When designing a site, something as small as a font or a color could greatly affect users. To make these decisions, UX designers often turn to A/B testing, which is “a study of the impact of an independent variable on a dependent variable” (Farmen, UsabilityGeek). This is a cost-effective way to make improvements on a site that already has an existing brand. It can help users communicate needs and wants from a site by splitting site traffic evenly between two different versions of your site, the control site and the altered site. This way, you can test out new features or products before committing to them. In this test, it is important that only one feature is changed and that the rest of the site will remain the same so you can have accurate conclusions. I will be using A/B testing to try and enhance Abercrombie & Fitch’s site.
After analyzing A&F’s site, I found that they have a great existing design, but it seems a little bit flat and could use a little bit more movement, which may give users a push to explore the site a bit longer. By adding a multimedia element, they could see a higher interaction rate. I want to focus on the homepage, which is the user’s first impression.
The first step in any A/B test is the creation of a hypothesis. A hypothesis tells you what you predict how your dependent variable will change when you adjust the independent variable. This also helps set a goal for your test and create organization. For my A/B test, I want to create a more interactive homepage. When you enter A&F’s site, their landing page is filled with still photos. While they are very nice images, a user may not be inclined to click on it. This is what the homepage currently looks like:
For this A/B test my hypothesis is as follows. By adding video content to the homepage, it will generate more homepage interactions.
Now that I have my hypothesis, I will need a control group and an altered group. Half of my users will be directed to the normal A&F website with the still photos on the homepage. The other half will be directed towards the altered version of the site, which will display videos as a part of their homepage experience. Here is a clip from the A&F YouTube that gives you a good feel for the short videos that will included on the new homepage. They don’t need to be excessively long, but we want to give users a better feel for the brand itself.
The pictures on the controlled homepage are used to represent different categories, such as “New Arrivals” and “Shop Coats” as shown here:
This encourages users to interact with this by clicking “Shop Men’s” or “Shop Women’s”. In the altered version of the site’s homepage, the videos will serve the same purpose of representing a certain collection or category. Hopefully, the videos will catch the eyes of users, making them more likely to interact.
After testing, I will draw conclusions to whether or not this video element makes a positive or negative difference to users. Then, I will decide if it is a good choice to go further with this design.
There are many other companies who have used A/B testing as a way to improve their product or service. For example, Electronic Arts, the media company that distributes the Sims games, used A/B testing to see what removing a “pre-oder” option would do to sales.
This is the control sales page, which included an incentive to pre-order the game with a coupon:
The altered sales page removed this incentive:
The altered version performed 40% better than the control. Electronic Arts came to the conclusion that their target users did not care about incentives. Because this is a game with a cult-like following, fans simply wanted to purchase the game.
Another example of an A/B test is from a non-profit called Kiva. This company helps people struggling to acquire a business loan get funding through donations. They wanted to test the impact of a “Frequently Asked Questions” page would have on their conversion rate. A conversion rate is “the percentage of users that take a desired action” (Nielson, 2013). The altered site provided a “FAQ” section on the homepage. A/B testing showed that the altered page performed 11.5% better than the control page. This showed that people needed more information on their organization in order to take action.
A/B testing is an important tool that every organization should use to improve their site. It is a great way to find out what your users need from your services.
“A Comprehensive Guide to A/B Testing.” UserZoom, http://www.userzoom.com/ux-library/comprehensive-guide-to-ab-testing/.
“Conversion Rate: Definition as Used in UX and Web Analytics.” Nielsen Norman Group, http://www.nngroup.com/articles/conversion-rates/.
Farmen, Nicholas. “A/B Testing: Optimizing The UX.” Usability Geek, 3 Sept. 2019, usabilitygeek.com/a-b-testing-optimizing-the-ux/.
Patel, Neil. “3 A/B Testing Examples That You Should Steal [Case Studies].” The Daily Egg, 21 Mar. 2019, http://www.crazyegg.com/blog/ab-testing-examples/.