This was before I happened to start reading a book called Contagious: Why Things Catch On by Jonah Berger and realized that he talked about much the same thing in his book. If you really want to get a handle on creating viral content, go read this book.
Every social media marketer and content creator wants to know the secret to creating viral content. But, aside from a few qualities mentioned in my article above and in Jonah’s book, there is no “secret” to creating viral content. The clues are all there, if you know where to look.
Besides, creating content is only part of the puzzle. It’s no use creating an awesome blog post or video if you don’t promote it and give it the exposure it needs, so it gets seen by the right people (read, influencers).
As the infographic from Crazy Egg below shows, content promotion is as important (if not more) than content creation. In fact, bloggers like Derek Halpern recommend that you spend 20% of your time creating content and 80% promoting it.
Some ways of promoting your content, as laid out in the infographic, are:
Whether you use a conversion tool like Optimizely, or a simple web app like IsValid, you’ll notice that conversion rates always come with a range.
For example, check out this screen shot of a sample test:
The conversion rate is currently 4.3%, but there’s a range underneath it from 0.6 to 8.0%. This means that given a large enough sample, the conversion rate could fall anywhere in that range.
Now this doesn’t mean that the far ends (0.6 or 8.0) are likely, it just means that they are possible.
How Important is Significance?
Statistically significant — have you heard that term before?
The significance of a test tells us how confident we should be that we have the right result when we are picking from 2 or more options.
When you run a basic A/B test, you’ll have a confidence interval for each option. In many cases, these 2 confidence intervals will overlap.
See the example below, for example. The original could have a conversion rate of up to 5.6%, while the variation (the current winner) could have a conversion rate as low as 0.6%.
Does this mean the current results are useless? No, not at all.
But it means that we need to calculate the significance of the test in order to determine how confident we can be when we choose the variation as the winner.
According to the tool, the significance is currently 91.1%. This means that 91.1% of the time, the variation is the best performing option. However, that leaves 8.9% of the time where the original is actually the best.
In reality, tests are typically run until a 95% or higher significance is achieved. Even at 95%, 1 in 20 tests will end up with you picking the worst option. While it would be ideal to test everything to a 99%+ significance level, it’s not always possible due to traffic or time limitations.
A note on significance: If you can only get to 95% significance in most tests, that’s not ideal, but it’s okay. Just understand that not every lesson you learn is going to be correct, and that you should expect a conflicting result once in a while.
A Critical Variable: Sample Size
Flip a coin 10 times, and you’re fairly likely to get lopsided results, like 3 heads (30%) and 7 tails (70%), even though in theory, they should be split 50/50.
Flip that coin 100 times, and you’ll get closer to the real probability, something like 48 heads and 52 tails.
By source: Traffic comes from different places. Google, Bing, social media, email links and more. Visitors from different sources of traffic tend to behave and convert differently.
By behavior: Did they come to the test page from a certain page on your site? Do some of your visitors read 3+ pages on your site on their first visit or visit at least 5 times a month?
By outcome (conversion): Which visitors convert the best? If applicable, which of those later buy your more expensive service or product?
There are occasions where you can segment by 2 or 3 of the above types all at once. It just depends how detailed you’d like to go.
Getting back to testing validity, the point is that your results can be invalid if you do not pay attention to segments.
Your sample size calculator says you need 10,000 visitors for each variation. You do that, and see that one side is the clear winner. However, after digging a bit deeper, you see that the winner had an extra 2,000 visitors from search engines (because of variance, just like flipping a coin). You find that search-engine visitors convert really well on your site, thus skewing the results.
After running the test until the amount of traffic from each traffic source levels out, you see that the original is actually the best — mistake avoided.
You need to consider variation in the most important segments for your business and test before declaring a winner. It may be the sources of traffic, certain behaviors, country, or more. Learn how to segment your visitors with Google Analytics.
Sample Size Does Not Always Reflect a Business Cycle
Check out just about anyone’s analytics reports and what do you see? Massive variation in traffic numbers based on the day. Usually a peak during the week, followed by a massive dip on the the weekend.
However, it’s not always just numbers. You get different visitors based on the day. If you dig a bit deeper, you’ll likely notice that the traffic you get from different sources also changes a bit from day to day.
A “business cycle” typically refers to one week for most businesses, although it may be different for yours. Whatever time period encompasses most types of your typical visitors is a business cycle.
What happens when your sample size calculator says to run a test for 10,000 impressions, and you have 20,000 visitors in one day?
You finish the test in one day. But this doesn’t take your business cycle into account. You might have a valid result for visitors on a Monday, but not necessarily for visitors overall.
This particular testing problem isn’t usually a problem unless you have great traffic numbers. Nonetheless, be aware of it.
If you’re feeling a bit overwhelmed, don’t worry. We can simplify this process into 3 main steps:
1. Calculate Your Minimum Sample Size
Determine what level of confidence (significance) you’d like in your test’s results, and calculate a sample size based off of it. This will be the minimum number of impressions/visitors that your variations need.
2. Check for Discrepancies in Segments
Before the test is complete, you should already know how to segment the visitors to your website. Once the minimum sample size is complete, dig deeper to determine if there are any major discrepancies. If so, keep the test running.
3. Assess Your Business Cycle
Your tests need to run for a whole number interval of business cycles. If the minimum sample size is up after half a cycle, or 1.75 cycles, keep it running until the next whole number (1 or 2 cycles respectively in this example).
That’s all there is to it, 3 fairly easy steps. Statistics are your friends, as long as you understand them.
Do you have any questions? Leave them below and I’ll answer them the best I can.