Complete Tests in Half the Time WITH SMARTER TEST RESULTs

Creative testing in digital advertising is hobbled by the need for very large sample sizes, making tests expensive and lengthy.  Adacus brings the latest in A/B test statistics to digital marketers with a Bayesian-powered A/B statistics platform.  

 
Adacus Bayesian Testing Sample Size Table
 

Optimization with Offline Conversion Tracking

Wherever your orders occur - online, in bricks and mortal stores or in call centers - Adacus can still do creative testing and dynamic creative optimization.  This is a major blind spot for other DCO platforms that only Adacus addresses, through partnerships with major offline conversion tracking vendors across sectors.  

noun_488408.png

Full Factorial Test Design

Adacus leverages its dynamic creative platform to run multiple A/B tests at the same time - testing images, messages, and other creative elements in a single test.  This delivers creative insights faster for you, while also enabling you to discover multivariate creative insights (e.g. interactions between images and taglines).


REPORT WINNERS WITHIN 1000+ AUDIENCE SEGMENTs

For most creative A/B tests, there is more than one winner.  Different ads appeal to different segments of your media.  Creative A may have been the best performer overall, but Creative B performed far better with 18-24 year olds and individuals interested in travel.  Adacus leverages its built-in user segments to report A/B test results within each of 1000+ demographic, interest and other segments.  And with a click of a button, these segment-specific winners can generate a Creative Decision Tree specific for your brand.

Most digital marketers start their use of Adacus with an A/B/n test, the segment-level results of which they use to launch a Creative Decision Tree.  It's a simple and risk-free way to evaluate and on-board a Creative Side Platform and impossible to achieve with traditional statistical significance metrics based on p-values.

Flat_Laptop_AB_Stats_Demographics_Adacus.png

AUTO-Generate Creative Decision Trees

Adacus generates a suggested Creative Decision Tree that optimizes your conversion rate.  No more guessing what creatives to serve to whom, and in which order.  Our machine learning platform learns which audiences respond to which creatives in an A/B test, and automatically generates the optimal Creative Decision Tree for your brand.     

If you choose, you can even deploy the Creative Decision Tree on Adacus' platform.  Or leverage the insights on your other platforms.


Video A/B and Multivariate Testing

Test your instream, outstream and rich media video fast and for less cost than traditional panel-based video testing.  Adacus can deploy multiple variations of your video on its dynamic video platform and within days report winning variations based on video completion rates and full funnel conversion rates.

You can even automatically optimize to the winning video variation based on A/B or multivariate video testing.

noun_ab_test_video_734663.png

Additional Questions

What is Bayesian A/B testing?

Traditional A/B test statistics has held back digital advertising A/B testing because its success metric for statistical significance - p-values - require very large sample sizes when tests have low conversion rates - as is common in digital advertising banner testing.  But statistical significance based on p-values isn't even what digital marketers want to know.  Adacus cuts testing time in half by reporting the metrics you actually need to know, metrics which fall within success thresholds faster than p-values:

  • Chance to Beat Control: Unlike the misleading p-value reported in traditional A/B test platforms, Adacus reports the Chance to Beat of each creative variation.  Adacus reports the Chance to Beat a control and the Chance to Beat all other variations.
  • Potential Loss: You've selected a winning creative variation, and you know there a chance it's a false positive and the losing variation could beat it.  But do you know by how much?  Adacus reports the Expected Loss of selecting a variation as the winner. This tells you how much is at risk if the losing variation turns out to be the winner.   

How do I know when I can stop a test?

Set a threshold for the Chance to Beat and/or the Potential Loss and, when the threshold is met, a winner is declared and the test stopped. Common thresholds for the Chance to Beat in digital advertising are 75%, 80%, 90% and 95%, while common thresholds for Potential Loss are 0.5% 1% and 2%.  Adacus can provide estimated test run times for different thresholds.

I've heard Bayesian A/B tests use a prior. What does that mean? 

Bayesian statistics are distinguished by their use of prior knowledge in their calculations. Adacus uses an uninformative prior, or flat prior, in its Bayesian calculations, as most A/B tests are run one time and have no prior knowledge of the difference in outcomes available.  Want to know more about Bayesian statistics?  The International Society for Bayesian Analysis is the best place to start.

Does Adacus provide consulting support on test planning, administration, and interpretation?

Absolutely, we love supporting and empowering digital marketing teams and include unlimited consulting on test planning, administration and interpretation in our contracts.

How can I use A/B testing by audience segment to create a decision tree?

The Adacus testing platform reveals audiences with which each creative variation over-performed the control.  You can then immediately set up IF/THEN rules in the decision tree that show the best creative to each of those audiences.  



Have other questions of your own?  Let's get in touch.