Know Your Audience with Amazon’s Managing Your Experiments Tool

Marketing is ever fluid and changes with consumers’ desires which makes it difficult for sellers to decide what will appeal more to their shoppers. As sellers, we may think we know what is best for our brand but predicting our audience’s taste is difficult and there is a 0.5 probability that our prediction may be inaccurate.

This is where market research comes into play and a lot of the big brands spend thousands of dollars on market research alone.

Amazon’s Manage your Experiments tool assists you in enticing shoppers on Amazon to take a chance on your product. Today, we’re exploring this tool that is available to us on Amazon Seller Central.

“If you double the number of experiments you do per year you’re going to double your inventiveness.” – Jeff Bezos

Test new content and optimize your listings for higher click-through-rate (CTR) and unit session percentage (USP). Sellers with a registered brand on Amazon can experiment with changing aspects of their listing without actually committing to those changes in the long run. 

Manage Your Experiments (MYE) splits your listing content 50/50 between random Amazon shoppers, providing you with customer feedback. You will get real-time data on how your shoppers react between the original and the new content.

So, how does this tool work and what practices should you keep in mind to get the best results?

Keep reading to find out more!

Who Can Take Advantage Of This Tool?

Not all sellers are eligible to use this tool.

• First, you must have a registered brand on Amazon. 
• The ASINs need to have high traffic and sales. 
• For experimenting with A+ content using this tool, your Amazon listing should already have A+ content published. 
• MYE is not available in all regions. If you sell in other Amazon regions other than the US, you may not have access to this tool.

Here is what Amazon says makes a product eligible for running these tests: 

If you fulfill all the above requirements, you can start experimenting with this split testing tool, immediately. If you meet all of these and still don’t have access to Manage Your Experiments, you will need to contact Amazon support to gain access.

Setting Your Goal

Before you start playing around with experiments, think about what you want to accomplish. For instance, do you think that using a 3D render of your product as your main image rather than real-life photography of your product may encourage shoppers to click on your listing? 

Or maybe, you believe that having a shorter title in your niche may convert better than a longer title. Setting one clear goal before you jump into testing will help you understand the results of your experiment. If your experiment is a success, meaning there is a clear winner between the two options you uploaded as an experiment, you will want to know what made it work so you can apply your findings to other products.

Experiment on 3 Types of Content

Right now, MYE lets you test your new marketing ideas on 3 types of Amazon content.

• Product Titles
• Main Images 
• A+ Content

Once the test is over, the data will point to which Amazon listing performed best and you can select what you want to publish based on conversions and sales.

How Do I Start?

Once you’ve figured out a goal and decided what content you plan to test, you’re ready to get started. To access the MYE tool on Amazon Seller Central, look for the “Brands” tab at the top of the page and click for the dropdown menu. Click on “manage experiments” to go directly to the page you need. Another way to access it would be to click on your “Brand Dashboard” and go to “Brand Benefits”. You will find an option to access “manage your experiments” there too. 

The main landing page gives a few Amazon resources from the learning center on experiments. Underneath all that extra information, look for the blue “create a new experiment” button. 

Choose the type of experiment you want to run. As you can see, there are 3 types – A+ Content, Product Title, and Main Image. 

For this example, we picked A+ Content. On the next page, you can select a reference ASIN. Sometimes eligible ASINs will automatically show up here. 

Enter the test name and pick how long you want to run the experiment. You can write the goal you came up with previously in the hypothesis text box. So phrasing the previously used example would look like this:

“Changing my main image with real-life photography of my product to the main image with a 3D render of my product will drive more sales.”

In case you still are unsure about how to phrase your hypothesis, you can click on the text saying “View hypothesis examples” in blue to understand it better.

After you have all the information filled in correctly, make sure that the “A” content has an ASIN. “B” content does not need an ASIN as “B” content inherits the ASIN from “A” content. All you have to do now is make your changes based on your earlier hypothesis.

Interpreting Results

Amazon gives you data on units, sales, conversion rates, units sold per unique visitor, and sample size assigned to that version of the experiment. This marketing tool analyzes the data and shows objectively whether A or B performed better overall in these fields. The longer the experiment runs the more data you will see, allowing you to make an informed decision on your marketing strategies. Amazon recommends an approximately 10-week duration for getting accurate results.

We ran an experiment on our Project X egg tray’s title and the following were the results.

Pros and Cons

So, you know how to start experimenting with split testing using MYE, but what are some of the advantages and disadvantages of using this tool?

Pros:

• Manage Your Experiments works within the Amazon system. Meaning that, you can use it from the comfort of Seller Central and know that the data is all Amazon-relevant and very relevant to your listing too. 
• The data is provided in real time so you don’t have to worry about outdated information slipping into your results. 
• The changes are split evenly 50/50, between Amazon shoppers at random, during the same length of time. Conducting an experiment at the same point in time makes the data invaluable, because it eliminates the possibility of uncontrolled factors affecting your two options, A and B, differently. 

Cons:

• MYE’s split testing is limited to only 3 categories for now. Meaning that if you want to test bullet points or secondary listing images, you’re out of luck. 
• The Amazon listings that would benefit the most from A/B testing on Amazon are often ineligible. You can only test high traffic listings with MYE.

Tips for Best Practices

Now that you have figured out the basics, here are three tips for best practices on Amazon’s Manage Your Experiments tool.

1. Before you even start experimenting, figure out what you want to test or what your strategy is. Do you think that your target audience responds better to shorter product titles? Test your hypothesis to find out if they do prefer shorter titles. Choose a direction and strategy to better understand the results so you can apply the strategy to all your listings. This saves you time and effort when experimenting on your listings. Having a goal in mind also gives you clarity no matter what the results are.
2. Only ever test one idea at a time. For instance, if you are testing images on one of your listings, only change the main image and see how your target audience responds. You may feel like you want to change everything on your listing, but if it does better than the original, you won’t know what caused the positive response. Was it the change to the title or maybe your new main image?
3. Give your experiment time to run. You may see promising results within a few days, but it is always better to wait until the experiment is finished before you implement new changes. MYE gives you options for how long you want your experiment to last. The time range starts from 4 weeks. Amazon recommends a timeframe of 8 to 10 weeks. If you end your testing process too early, your results may be very limited. Short testing impacts accuracy.

Similar Split Testing Options for Amazon Marketplace

Are you a seller who is interested in split testing but not yet eligible for Manage Your Experiments? Big brands often have access to marketing tools on Amazon that beginner sellers are not eligible to use. If you are looking for a way to test your strategies with the same sort of A/B method, give Helium 10’s Tool, Audience, a shot.

The audience is open to all Amazon sellers and uses the same A/B test method as MYE. This tool for the Amazon Marketplace has more flexibility than Manage Your Experiments. You can create a poll in minutes and make informed decisions about what your target audience prefers before your listing is launched. 

You can test out any hypothesis you are unsure of. Ask about packaging, branding, images, or which benefits appeal more to consumers and get feedback from a sample size of your own choosing. You can filter out your sample by gender, age, spending habits, employment status, or any other traits descriptive of your target audience. Both MYE and Audience give useful marketing information to improve your listings with real and Amazon-relevant data.

Final Thoughts

No longer rely on the unwieldy A/B testing methods where you upload a new title or image for a month, and then do another variation the next month to see how shoppers react. With both Manage Your Experiments and Audience by Helium 10, you can test new marketing strategies by sending them directly to your target audience. 

When using Manage Your Experiments, your optimized images or text goes directly to real shoppers and gives data about how often people click on each version of your listing. This saves you time and effort when testing new content. If you and your listings are eligible for this Amazon tool, the pros outweigh the cons of taking advantage of this resource for brands.