Test #005: Increasing Incentive During the Checkout Process


PARTNER: TED'S BRAIN SCIENCE PRODUCTS
COMPANY TYPE: PAIN RELIEF-HEALTHCARE, ECOMMERCE
EXPERIMENT QUESTION: IF WE STRESS 'FREE SHIPPING' IN CHECKOUT FUNNEL, CAN WE INCREASE INCENTIVE AND INCREASE ECOMMERCE CONVERSION RATE?


Introduction:

Ted's Brain Science Products is a Dallas-based healthcare company that specializes in developing pain relief solutions. They found that two natural ingredients–methyl salicylate from the wintergreen plant, and resveratrol from grape leaves or Japanese knotweed–appear to work together in a brand new, previously unknown way. There flagship product, Ted's Pain Cream, is the first of it's kind.

As we continued to review the checkout funnel for Ted's Pain Cream (single-pack), we noticed that the incentive of 'Free Shipping,' was not quickly apparent when viewing the page. We wondered: Will emphasizing 'Free Shipping'  increase incentive, and consequently, increase eCommerce conversion? 

 
screencapture-tedsbrainscience-11700108-checkouts-7914662fe6f0ed7416d2270c9e10d931-2018-05-14-18_19_36.png

Control

This was the control 'Checkout' page for Ted's Pain Cream. You can notice that 'Free Shipping' is implied on the right, but there is no other callout for it. Here are the results observed for the control page: 

Control: 1386 Sessions, 259 Conversions (16.2%)

 


 
 
screencapture-tedsbrainscience-11700108-checkouts-e91c5e26e530f318c195118a647a4120-2018-05-14-18_19_56.png

Treatment

This is the treatment version of the same 'Checkout' page. You'll notice the addition of the 'FREE SHIPPING' banner at the top of the shipping information form. Here are the results observed for the treatment page:

Treatment: 1,319 Sessions, 233 Conversions (15.4%)


RESULT: INVALID TEST RESULTS


 

Analysis: 

This experiment failed to cross the 95% confidence threshold, meaning there was no statistically significant difference in performance between these two versions.

Even though these results aren't conclusive, we can still learn a few things. First, it's clear that the 'Free Shipping' offer injected into the second treatment version was not a strong incentive for users. This may be due to the execution of the 'Free Shipping' Banner, or perhaps, the offer of free shipping itself. 

Concluding thoughts: 

We'll continue to test this approach, and we'll likely return to the 'Free Shipping' offer from a different angle. But this is a great test to illustrate the importance of testing. We assumed before the test began that emphasizing 'Free Shipping' was likely to align well with the motivation of the viewer. But as we can see, it simply did not in this experiment.

Grant Klembara