Test, learn, and optimize your campaigns with data-driven insights

Split testing in Spotify Ads Manager lets you compare different versions of your ads to understand which script, voiceover, or other creative element performs best. It’s a simple, versatile way to make data-backed creative decisions, improve campaign performance, and ultimately impact business results. 

What is Split Testing?

Split testing helps you measure the impact of a single variable on ad performance. When you set up a test, your audience is randomly divided between versions of an ad. Spotify then measures performance against a chosen success metric like click-through rate (CTR), completion rate, or cost per conversion.

Each ad variant runs under identical conditions — same targeting, budget, and scheduling — so you can be confident that performance differences are due to the creative changes you’re testing.

When should I run a split test?

You should run a split test when you want to validate which ad creative element is driving your campaign performance — not just observe it. Split testing helps answer why something works, not just if it does.

The following are the best times to use it:

1. When you’re launching new creative

Before scaling a new ad creative, test variations (e.g., different scripts, visuals, or calls-to-action) to see which one resonates most with listeners. This ensures you’re putting more spend behind what performs best.

2. When campaign performance is plateauing

If your metrics (like CTR or completion rate) have leveled off, use a split test to diagnose what to adjust — for instance, tone of voice, ad length, or the presence of background music.

3. When optimizing for a specific goal

If you’re focusing on a particular KPI (like conversions or engagement), use a test to pinpoint which creative elements most effectively drive that outcome.

4. Before major campaigns or seasonal pushes

Running pre-launch tests lets you validate creative or messaging ahead of large-scale or high-stakes campaigns (e.g., holiday promotions or product launches).

5. When using new formats or features

Experiment with different messages, video styles, images, etc to master each format across audio, video and display

Setting Up an Effective Test

To get tangible, statistically valid results, follow these setup best practices:

1. Isolate one variable at a time
   Only change one element — such as the main audio track, visual, or CTA — between test arms. Keeping all other campaign factors identical ensures the difference in results is truly due to that creative variation.
   
   

2. Give yourself enough sample size
   Spotify recommends a minimum budget of $500 total ($250 per variant) for reliable results. We recommend running the test for at least 2 weeks to allow enough data for informative results. Finally, remember not to narrow your audience size too much.
   
   

3. Align your KPI to your campaign goal

   • Awareness: Completion Rate or CPV

   • Traffic: Clicks or CTR

   • Engagement: Streams or CPC

   • Conversions: Cost per Acquisition or Cost per Lead or ROAS (requires Pixel or CAPI setup)
     
     

4. Align your creative to your audience 

Ensure your creative resonates with the audience and includes a relevant message - this is a good practice for any campaign, but for creative tests in particular, a mismatch will defeat the purpose of testing creative variations

5. Use GenAI creative support (for audio tests)
   If you’re short on creative resources, you can leverage Spotify’s GenAI tool in Ads Manager for creative variations. This tool is available for audio ads in English in the US, CA, UK, DK, NL, NO, SE and SG (Learn more)

Crafting Strong Creative Hypotheses

When designing your tests, start with a hypothesis — a clear idea of what creative change might impact performance and why.

Below are some examples:

Pro Tip: Use the following starting points when crafting your hypotheses:

1. Spotify’s Creative best practices (Found here)

2. Post-campaign insights to identify creative elements worth testing further: For example, if you see strong engagement among GenZ listeners, test highlighting products that resonate with this audience or a creative production style that resonates with this audience (e.g. Selfie style video)

3. Consider hypotheses that align with format-specific learning objectives. See examples below:

Step-by-Step: How to Set Up a Split Test

You can create a split test from the Experiments tab of Ads Manager.

1. Start Your Test

• Choose your campaign objective and targeting as you normally would.

• Choose your success metric (e.g., Clickthrough rate, Completion rate) that you would like to assess 

• Set your budget — the amount input in the Daily budget or Lifetime budget field is split evenly across the variants.

2. Set Up Test Variants

• Choose how many variants you want to test (up to 4).

• Add your creative details into your first variant

• Duplicate your first variant — Spotify will automatically copy settings so both versions remain identical.

• Edit the creative element you wish to test (audio file, video, CTA, Companion image, etc) for the second variant. You must make an edit, or else you will not be able to proceed.

• Review all details of the test and ensure everything looks good

3. Launch and Monitor

• Launch your test — success metric pacing and reporting across variants will appear in real time in Experiments.
  
  

• Continue to monitor results. Note the following:

  • You may pause a test, and re-enable within the timeframe of the test flight

  • You will not see the test’s outcome until the originally booked end date of the test has been reached 

  • While the test is live, you may edit the test length and budget. No other test or variant details can be changed. 

4. Analyze Results

• Once your test concludes, you will be able to see the results. If there is a winning variant, a flag will appear and you can see the confidence % that was achieved. If the test is inconclusive, a winner will not be declared.

  • Note: A variant is declared the winner if it outperforms the other(s) at least 65% of the time. 

• Duplicate your winning variant into a new campaign or continue iterating with another test.