eCommerce A/B Testing: Ideas & Tips You Should Know

What is eCommerce A/B Testing?

eCommerce A/B testing is a scientific method of evaluating a marketing variable compared to a control. This occurs when brands show a percentage of users either a control or variable version of a marketing asset. When properly executed, this exercise helps businesses evaluate a variable’s effectiveness related to the control.

eCommerce A/B testing mirrors the scientific method and helps businesses isolate individual elements that influence results. Once brands identify a winning set, they can apply their findings for remaining users and future marketing materials.

Overall, this testing process creates an ongoing cycle of optimization and, with the use of A/B testing tools, contributes to smarter marketing decisions.

eCommerce A/B Testing Ideas

With advances in eCommerce and marketing technology, brands can A/B test almost any asset. However, we’ve seen that a few specific areas are particularly well suited for testing. When it comes to your eCommerce website and email marketing strategy, there is always something that can be A/B tested.

eCommerce Website

Calls-To-Action

Consider evaluating button color, size, font weight, capitalization, messaging and shape. These things may seem inessential, but each factor affects users differently.

Optimizing these small features can improve engagement and drive sales, so be sure to take your time and evaluate ways to constantly improve.

Page Layouts

Evaluate heading, button, form and image placement. Each of these elements contributes to the overall user experience and can either help or hurt the purchase process.

We frequently evaluate product detail and checkout pages as a part of a routine eCommerce A/B testing process. These pages offer high conversion value for brands and should be a key focus for future success.

Product Imagery

Testing the order and content of your product photography can drastically influence consumer purchasing decisions. Consider evaluating lifestyle vs isolated product images on your product detail pages.

Additionally, you can test if user-generated images perform better than branded imagery in various places throughout your store.

Pricing Strategies

Testing individual or bundling pricing can help brands better understand how these elements directly influence conversion. If possible, we encourage brands to evaluate their pricing/conversion relationships and consider how they ultimately influence profitability. You may find that you can adjust pricing to increase conversion without influencing profitability.

Promotional Offers

Consider testing free shipping, discounts and added perks throughout your site. Comparing these elements directly can offer better insight into what resonates with your customers. You can then use this information to inform future marketing promotions.

Email

Email Templates

Template design can drastically impact eCommerce email engagement. We recommend evaluating templates with identical subject lines and messaging to see what formats lend themselves best to different marketing promotions. Consider testing placement of your logo, banner imagery, CTAs and body messaging.

Subject Lines & Preview Text

This is one of the most straightforward email elements to test. Evaluating different email subject lines can help you see what version gets a larger open or click rate. Analyze your results and then use the subject line that will get you the most interaction.

Creative Imagery

Test banner imagery, product highlights and other graphic elements within your email content to evaluate what motivates your users to convert. Although content will vary from email to email, marketers can use this information to identify trends and inform future marketing correspondence.

How To Run An A/B Test

Determine A Variable

When setting up your eCommerce A/B test, there are a few settings to determine. First, just like an experiment, you will need to determine your variable. The variable should be the only thing that is different between your two alternatives.

Set Distribution Percentages

After determining what you are testing, set your distribution percentages. We recommend sending Version A to about 15% of your audience and sending Version B to about 15% of your audience. After your results come back, the remaining 70% of your audience will receive the winning version.

Determine A Winning Metric & Duration

Next, you’ll need to set the winning metric. This is the standard you will use to determine your winner. HubSpot allows you to choose from three winning metrics: opens by delivered, clicks by delivered or clicks by opens.

You then need to set your test duration. We generally recommend longer test times to ensure your results are accurate.

Set Fallbacks

Lastly, don’t forget to set your fallbacks. Just like in a regular experiment, things can go wrong. Your sample size may be too small to get accurate results or they may be inconclusive. As a result, it's best to set the version you want to send if you run into one of these problems.

Conclusion

Although we firmly believe in the benefits eCommerce A/B testing can offer, we caution you to ensure that sample testing sizes are large enough to produce statistically significant results. If your audience is not large enough, the testing will be invalid.

A/B testing is based on numbers and changes are only made if the data backs it up. When testing emails and eCommerce websites, you will typically need a large user base to create a representative sample size. 

Overall, eCommerce A/B testing is a process that takes time, dedication and attention. If you have any questions or need help developing a road map to navigate your A/B testing, contact us through the form below. Our team will be in touch!