This seems simple: did my conversion rate go up or not? However, today’s objective is to explain the difference between a test’s conversion rate and the actual rate of conversion on your entire website.

There are a lot of test examples floating around online and they boast some pretty big numbers. Before you get swept up in the flurry of marketing excitement, be sure to check how they are defining “conversion rate.”

There are two definitions for “conversion rate” that I see being used. Be sure you don’t confuse the two of them:

1. The number of times a site visitor completes a desired action during a test.
2. The number of orders your website generates divided by the number of visits your website receives

When these two definitions get confused (or worse: assumed to mean the same thing), it can set false expectations. A 70% increase in ATC clicks is not the same as a 70% increase in the number of orders you receive on your website.

When the phrase “conversion rate” is used with Exclusive Concepts, we mean definition #2: the number of orders your website generates divided by the number of visits your website receives.

So, to show how these can get confused, here are some sample results from tests that I have run.

Here is two sets of data from sample test results. The percentage displayed is the percentage increase or decrease in that individual metric. Lets look at these results from the perspective of our two definitions.

If you look at these results from the first definition on the previous page and you have Add to Cart clicks set as your tests “conversion metric,” than the first set of results would show a test conversion rate increase of 20% while the second set would show a decrease in the test’s conversion rate of 6%.

However, if you look at this from the perspective of actual site-wide conversion rate and actual revenue generation, you can see that the second results example is the test that improved your site’s conversion rate.

The true story to these two tests is that the first one caused customers to click the Add to Cart button before they were ready to buy. The second result example turned more browsing customers into buying customers and removed the browsing customers from the funnel before they clicked the Add to Cart button.

The second set of test results is the test we would want to hardcode. If you don’t check what is defined by “conversion rate,” you could make a change to your website that actually hurts your business.

The only true conversion rate is how many completed orders you have divided by the number of visits. When you look at test examples online, make sure they are measuring the actual conversion rate, not an increase in ATC clicks or Click Through Rate.

Make sure the testing software you use is able to track actual conversion rate and associate that to actual revenue. Even though we want conversion rate to increase, there are instances where it could make you less money. The only metric that truly matters is how much money the improvement to your website will make you.