Why Can't I Split Test a Smart Campaign?

The reason you cannot split test a smart campaign is actually quite simple. In some ways, the smart triggering works like a split test in that it uses different triggers to find out what gives the best conversion rates for that specific campaign.

This means that there are actually a range of different campaigns (with different triggers) within a single SleekBox.

Apart from the technical difficulties this presents, it also means that an A/B split test of a campaign with smart triggers enabled will not yield any meaningful results.

 

A quick example:

Campaign A has smart triggers enabled.

Campaign B doesn't have smart triggers enabled.

 

Campaign A with trigger 1 has a conversion rate of 2,1%.

Campaign A with trigger 2 has a conversion rate of 3,6%.

Campaign A with trigger 3 has a conversion rate of 2,9%.

Campaign A with trigger 4 has a conversion rate of 1,8%.

 

Campaign B, has a conversion rate of 2,7%.

 

As you can see this means that Campaign B is performing better than Campaign A with triggers 2 and 3 but worse than Campaign A with triggers 1 and 4.