Split Testing: How to Improve Your Site Conversions


Do you split test the opt-ins on your website?

Want to get better results?

To learn how to create effective split tests, I interview conversion expert Joanna Wiebe.

More About This Show

The Social Media Marketing podcast is an on-demand talk radio show from Social Media Examiner. It’s designed to help busy marketers and business owners discover what works with social media marketing.

In this episode I interview Joanna Wiebe, a copywriter, conversion expert and founder of CopyHackers.com–a website designed to help you improve your conversions. She’s the author of the Copy Hackers ebook series.

Today Joanna will explore how split testing can help improve your email opt-ins and much more.

You’ll discover how to alter your headlines and buttons to improve your website opt-ins, as well as what tools to use to analyze results.


Listen as Joanna Wiebe shares how to improve site conversions with split-testing.

Share your feedback, read the show notes and get the links mentioned in this episode below.

Listen Now

You can also subscribe via iTunes, RSS, Stitcher, SoundCloud or Blackberry. How to subscribe/review on iPhone.

Here are some of the things you’ll discover in this show:

Split Testing

What led Joanna to copywriting and conversions

Joanna says she fell into the field of copywriting. After leaving law school, she was looking for her next opportunity. When a friend who worked at an agency said they were looking for a writer, Joanna got the position, along with the title creative writer. (She thought copywriter sounded too boring.)

A couple years later, Joanna went over to Intuit (makers of Turbo Tax) as senior copywriter. Once there, she says she finally figured out what copywriting was … and understood that it was not boring!

Joanna explains the difference between a creative writer and a copywriter. A creative writer is a person who is more likely to come up with tag lines and concepts for ads and campaigns. It’s someone who abstracts a message from insights. On the other hand, from Joanna’s experience a copywriter is more of a scientific writer. Copywriting is not about you. It’s about listening to people who are potentially nothing like you to find the right message, she says.

copyhackers website

Joanna started Copy Hackers three years ago.

According to Joanna, split testing became more readily available eight or nine years ago, and testing tools, like Optimizely, VWO and Omniture (before it was acquired by Adobe), were starting to pop up. So the company started split testing different approaches to solving problems. They would test them using actual data: website visitors or email subscribers helped them test by voting with their clicks or their credit cards.

This led Joanna to start Copy Hackers about three years ago.

Listen to the show to discover how creativity stifled Joanna in her first position as a writer.

The ad at the bottom of Copy Hackers

Copy yHackers Popup

Copy Hacker’s ad at the bottom of the screen.

To capture email addresses Joanna uses a solution called Bounce Exchange. They have been experimenting with ways to get people’s attention.

There’s a little guy in the corner of the website and it says “Click here to get a free guide.” It appears as you’re scrolling down the page. Once you click on it, it gives you the opt-in box.

Bounce Exchange is software presented with a service, Joanna explains. For best results, you work with their creative team and they come up with variations. They split tested different content and “The Free 2015 Persuasion Guide” got the best response. Now they are testing different messaging for the guide, as well as ways to get people to opt-in.

Listen to the show to learn what other content Joanna tested against the persuasion guide.

The exit intent popup

Exit intent means when the mouse moves up into a certain range to indicate someone is leaving your website. In this case, when the Copy Hackers’ exit intent box appears, readers are given the choice. “Yes, get the free guide” or “No, I reject the persuasion guide.”

Copy Hackers Exit Intent Box

Copy Hacker’s exit intent popup gives readers a choice.

Joanna says this king of messaging is about having your audience make a decision between a choice and a consequence. Sure you can give people a choice, Joanna explains, but when you show them a consequence, things change. This is what teachers do when they are trying to get students to do things.

When Bounce Exchange first came out, Copy Hacker wrote a post about the negative messaging. In response, Bounce Exchange reached out to them about working together.

With this new messaging, Copy Hackers went from 50 signups a day to around 200 a day.

Listen to the show to discover the logic behind making your audience click before they get to the opt-in page.

The main variables to test

Joanna suggests people start by testing the offer and then get into the headlines.

When testing, you’re are not just trying to get a winner. You’re trying to learn. Even if the test loses, you still win if you learn something, because you started the test right.

For their popup, Copy Hackers started by testing three different offers against each other. The control was what was currently up there, and then they tested variations.

Joanna suggests hypothesizing why the control (the original offer) isn’t pulling in subscribers. Then, ask a series of questions to create variations.

variations shutterstock 189247589

Brainstorm variations to test. Images: Shutterstock.

For example, research questions for their original offer, The Four Part Foundational Guide to Copywriting ebook, could include:

  • Does The Foundational Guide to Copywriting sound boring? Should we write a headline that sounds more exciting?
  • Is it that website visitors are not interested in The Foundational Guide to Copywriting because they are not that interested in copywriting?
  • Are they interested in something that’s sexier? What is that sexier thing? Maybe it’s persuasion.

Once you have that initial research question sorted out, you can quickly create the right headlines and the right buttons to test.

Listen to the show to hear ideas of images to test if you have an ebook offer.

How to know when you have results

Joanna says you need to have a decent amount of traffic to test. If you’re only exposing a variation to ten people a day, you’ll have a difficult time running a test to the point where you’re confident in the results. The downside of stopping a test before you reach significance is you have a false positive. In that case you didn’t really test.

According to Joanna, some people say you should have 500 conversions per variation before you shut the test down, which is very difficult for most businesses to do. Joanna suggests aiming for 100 conversions per variation before calling a winner.

thumbs up shutterstock 150754733

Aim for 100 conversions before you declare a winner. Image: Shutterstock.

If your results look really good and there’s a significant enough difference between variations, there are tools available. For example, Evan Miller has calculators that tell you if a test is ready to complete or not.

Optimization testing never stops, Joanna believes. There are so many different things you can test. One approach is to test one element at a time. More often than not, elements (like a button and a headline) are working together. So, if you’re only optimizing the headline or you’re only optimizing the image, you’re missing out on more that could be done if you optimize two together.

One option is to do stages of tests. First, you do an A-B test of two headlines. Then the winner is the new control and you test two different buttons on that headline.

Joanna talks about how she conducted a split test in stages for Dressipi.

The first headline was tame and the second headline was a little riskier, so they expected the second one to do better. It trended better but never reached confidence.

Dressipi headlines

Joanna tested a tame versus a risky headline.

They looked at the button, which said, “Sign up now” and figured they could do better. So they took the riskier headline and tested the “Sign up now” button against a new button, “Show me outfits I’ll love.”

Effective button for Dressipi

Changing the button made all the difference for conversion.

They got incredible results: 123.9% lift in clicks with 100% confidence with the new headline and button.

Listen to the show to learn about multivariate testing.

The most important variables to test

Joanna believes the button is the most obvious element to test. You run a test with a goal and the goal is usually to take some action, which needs to be completed by clicking a button.

It’s also easier to run a button copy test, because your readers need to take an action. A headline is more difficult to measure.

There are two things Joanna suggests people can do to create button copy to test.

“Take the phrase “I want to” and whatever follows that becomes your button copy,” Joanna says.

If the phrase is, “I want to see my heat map,” then the button is, “See my heat map.”  If the phrase is, “I want to find the best house for me,” the button is, “Find the right house for me.”

Joanna’s second recommendation is to have a call to value, rather than a call to action.

People don’t want to click to, “Learn more.” It’s an action. People don’t go online to click a bunch of things, they are there to solve a problem or get value for themselves.

Listen to the show to hear more about the conversion tools mentioned.

Discovery of the Week

The Chrome extension Dropbox for Gmail links two of my favorite tools.

Install it in Chrome and, when you write a message in Gmail, you see a Dropbox icon with all the formatting tools.

dropbox for Gmail

Dropbox for Gmail is an easy way to attach and send files.

Open up the Dropbox window, and you can see all of the folders and files and you can add them instead of files from your computer. It eliminates a couple of steps of how you would ordinarily attach a Dropbox file to a Gmail message.

To find Dropbox for Gmail, go to your Chrome browser and search extensions.

Listen to the show to learn more and let us know how Dropbox for Gmail works for you.

Other Show Mentions

Today’s show is sponsored by Social Media Marketing World 2015.

It’s the world’s largest social media marketing conference.

Hear testimonials from our 2014 conference speakers.

This is just a sample of industry experts talking about their experience at Social Media Marketing World.

This podcast is being posted on the final day of Social Media Marketing World 2015. If you missed the event and still want to experience it, for a couple more weeks you will be able to get the virtual ticket. It allows you to get access to every keynote, session and panel. Access the recordings, download the audio, watch the videos, look at the slide decks and consume the content.

Click here to check out all of the speakers and the agenda, watch our video and grab your virtual pass.

Listen to the show!



Key takeaways mentioned in this episode:

Social Media Marketing Podcast w/ Michael Stelzner

Ways to subscribe to the Social Media Marketing podcast:

How to Subscribe to this Podcast on an iPhone

Watch this quick video to learn how to subscribe on your iPhone:


What do you think? What are your thoughts on split testing? Please leave your comments below.

Thumbs up photo and Assorted variations photo from Shutterstock.

Related Posts

Tags: , , , , , , , , ,

Social Media Examiner