Creative Rotation Attribution Testing

There are two types of tests that a digital marketer uses to optimize creative: A/B or multivariate tests and creative rotation attribution Tests.  

The second, creative rotation attribution tests, provide insights about which creatives to include in a rotation.  Digital advertisers aren't faced with the question of whether to serve one ad or the other, but what rotation of ads to serve.  Only Adacus provides creative testing that can answer that question.

So, whereas Adacus A/B tests generate insights about the elements of a creative, Adacus Creative Rotation tests generate insights about creatives as a whole, specifically the fractional attribution of each creative to conversions.

Advertisers typically produce dozens of creatives over time and end up with large rotations as a result.  In fact, with the average impression frequency of prospecting campaigns at 1-2 impressions per user per month, the number of creatives in most rotations far exceeds the number of creatives that users actually see.  That's why it's important to identify the subset of creatives to include in a rotation that will maximize engagement and conversion.  Creative Rotation Attribution tests are special multivariate tests that serve different rotation combinations to different users in order to assess the partial attribution of conversions to each creative and decide which creative to include in a rotation. 

Test Planning

Creative Rotation Attribution testing helps digital marketers identify the optimal subset of creatives to include in a rotation.  It does this by reporting the fractional attribution of each creative within a rotation to conversions.

Creative Rotation tests compare the performance of different creatives as well as all possible pairs of creatives.  This enables Adacus' attribution stats engine to report the marginal contribution of each creative to conversions.

However, this type of test also requires more traffic in order to measure the performance of all pairwise combinations of creatives being tested.  

 

So, if you have 2 creatives - say, A and B - in a rotation and want to know if serving only 1 would be better, you would run a test with three groups:

  • A
  • B
  • Rotation of A and B

These creative tests are not set up to generate creative insights about the messaging, images or other elements of creatives.  Rather, they are designed to report the marginal influence of each creative - taken as a whole - towards your conversion rate, so you can decide which creatives to include in a rotation.

Test Setup

A Creative Rotation Attribution test is set up with multiple test groups just like A/B and multivariate tests are.  And setting up creative tests in Adacus is incredibly easy.  Unlike other ad servers that require separate creation of audience segments that are split and to which separate creatives must be traficked one by one, Adacus supports testing natively in one simple interface.

Simply create a separate Rotation for each creative, and for each pairwise combination of rotations, create a new Campaign and click Assign Creative Rotation(s).

In this example, 3 creatives are being tested to identify the optimal subset of creatives to include in a rotation.  If one of the creatives doesn't contribute to conversions, this test will reveal that.  By removing the creative from the rotation, the conversion rate would be expected to go up as a result of serving only creatives that uniquely influence conversion decisions.

This test is comparing the performance of 3 creatives - Family, Millenial and No Image.  As a result, there are 3 single-creative rotations and 3 pairwise rotations, for a total of 6 rotations, that are selected for the top and only node of the Campaign's Decision Tree.

Adacus automatically splits the percentage of users assigned to each creative rotation equally.

Once the campaign is saved, simply generate the campaign-specific ad tag for the appropriate DSP (or publisher) and you are done.

Analyzing the Results

The Creative Attribution Dashboard uses algorithmic attribution statistics to report the relative influence of each creative within a creative rotation.  

This report is essential for rotation optimization, determining which creatives to include in a rotation and the weights to set for each creative.

(1) Campaign/Creative Target/Rotation Filters: A campaign in Adacus' ad server generates a single ad tag and is equivalent to a placement in Doubleclick.  Each campaign in Adacus has a single Creative Decision Tree with one or more nodes that correspond to a creative target.

A Creative Target corresponds to a node in your Campaign's Creative Decision Tree.  The Adacus Campaign/Node hierarchy corresponds to the Doubleclick Placement/Ad hierarchy.

Each Creative Target can have one or more rotations.  This is in contrast to Doubleclick where only one rotation may be assigned to an Ad.  Adacus allows multiple rotations to be assigned per Creative Target in order to facilitate easy A/B testing by automatically randomly assigning users to a rotation within a Creative Target.

(2) Creative: The legend shows the name of the creative for each section of the bar chart.

(3) Influence: The percentage influence of a creative within a creative rotation measures the percent of conversions attributed to this rotation that are attributable to this specific creative.  

Testing Overview

Pitfalls to Avoid in Creative Testing

  • Faint Signal Problem
  • Not Controlling for Media
  • Test Plan Siloed from Media Plan
  • Rotation-based Testing on DCM, Facebook

A/B and Multivariate Testing

  • Test Planning
  • Test Setup
  • Analyzing Test Results

Creative Rotation Attribution Testing

  • Test Planning
  • Test Setup
  • Analyzing Test Results