Does AB Testing work? Yes, absolutely. It is awesome and can help you increase conversions.

BUT!!!! In order for your AB Split Tests to actually give you any decision making results you need a ton of traffic and conversions. The rule of thumb is for your AB tests to reach any sort statistical significance you need at least 100 conversions per variation on your website. (Statistical significance is a fancy way of saying “ability to make any real decisions based on a test”).

So, for a website to really learn anything from an AB test you need at least 200 conversions. Most websites get 200 conversions after a few months!

Yes it is cool to say that you are running an AB test for 2 button colors, but chances are really low that you will actually be able to determine which one works for you unless you have serious traffic or are willing to run that test for a few months. If your website has low traffic and you want to be more data driven here are some testing pointers:

Make Major Changes in Your Tests

Here are some major changes you can try:

1. Offer a Free Plan Vs. No Free Plan, Pay us $100/mo now
2. Landing Page with 1 Form Vs Regular Website (that includes a list of all your features, service offering, about us section, etc)
3. Major price differences $3/mo Vs. $30/mo

Try Sequential Testing

Sequential testing is when you test each variation in a given time period. Since you don’t have that much traffic you can try a different variation every 2 weeks. Just make sure you record all your data like total amount of traffic, paid conversions, and what you changed.

The biggest problem with sequential testing is that sometimes it can give you bad data because both variations are not running at the same time. For example towards the end of the month you may have a bunch of conversions vs. the start of the month.

Increase Conversions by Eliminating Barriers

If you need more conversions to run an AB Test, get more conversions!

Here are a few tips on increasing conversions:

1. Add a “Free Whitepaper” resource or Lightbox pop-up that requires the end user to enter in their contact information.
2. Remove any barriers like a “Credit Card Required” or a CAPTCHA field
3. Use multi-step forms. The first step should just be an email address, the second step can be everything else