Test #014: How Button Color Impacts 'Add-to-Cart' Rate


PARTNER: TED'S BRAIN SCIENCE PRODUCTS
COMPANY TYPE: PAIN RELIEF-HEALTHCARE, ECOMMERCE
EXPERIMENT QUESTION: IF WE CHANGE THE COLOR OF THE 'ADD-TO-CART' BUTTON, CAN WE INCREASE CONVERSION RATE?


Introduction:

Ted's Brain Science Products is a Dallas-based healthcare company that specializes in developing pain relief solutions. They found that two natural ingredients–methyl salicylate from the wintergreen plant, and resveratrol from grape leaves or Japanese knotweed–appear to work together in a brand new, previously unknown way. There flagship product, Ted's Pain Cream, is the first of it's kind.

After reviewing suggested A/B experiments in the CRO community, we found that testing button color was a common product page test suggestion. Some CRO experts imply that your button color is vital, while others say that focusing on button color is a waste of time. So we tested it and asked: Will changing the color of the 'Add-to-Cart' button to black increase conversion?

 

 
screencapture-discover-tedsbrainscience-products-pain-cream-d-html-2018-06-15-18_01_16.png

Control

This is the control of Ted's Pain Cream marketing 'Product' page. This page contains a green button that matches Ted's Brain Science brand and color scheme. Here are the observed results:

Control: 770 visitors, 36 conversions (4.7% Add-to-Cart Conversion Rate)

 

 

 

 

 

 
 
screencapture-discover-tedsbrainscience-products-pain-cream-e-html-2018-06-15-18_01_36.png

Treatment

This is the treatment version of Ted's Pain Cream 'Product' page. The only change made was to the 'Add-to-Cart' button, which is now black. Here are the results observed for the treatment page:

605 visitors, 38 conversions (6.2% 'Add-to-Cart' Conversion Rate)


RESULT: INVALID TEST RESULTS


 

Analysis: 

This experiment did not cross the 95% confidence threshold, and therefore did not produce a valid result.

It's clear that button color did not have a dramatic impact on the visitors who viewed this product page. As a consequence, the 'Add-to-Cart' conversion rate for this page remained–in terms of statistical significance–unchanged. 

Concluding thoughts: 

There are a lot of ongoing debates about what we should be testing on our webpages. I think it is helpful to read about the success and failures of other 'A/B testers.' If nothing else, they may spark a new thought in your mind about how to approach your site or advertising. However, since your site and audience is different than others, it's imperative that you test these elements/learning on your own. What works on one site may not work work with your site.

Grant Klembara