Innovation. It’s a leading strategic priority of executives worldwide; something that companies strive for—that new product, that new way of doing something. But it’s risky. It’s expensive, timely and often results in failure because it didn’t quite hit a customer need or resonate with the public.

Why leave it to chance? Imagine what your innovation process could look like if you were able to collect all of the necessary customer feedback early and often, not only reducing the risk of failure but increasing the likelihood of success.

Enter concept testing. It’s a proven way to help you identify what works and what doesn’t, so you can be sure you’re creating the products that best connect with your audience. We use concept testing here at Method to assess, validate and prioritize client ideas and concepts. It allows them to optimize their product development processes by saving both time and resources, making it a critical part of any product strategy and innovation.

In this post you’ll learn why concept testing matters, how to develop concept tests that fit your organization’s innovation needs, and a proven step-by-step process that maximizes the value of your concept testing.

So What Is Concept Testing?

Concept testing is a step in the product development lifecycle where concepts, or glorified ideas that are typically storyboarded out, are evaluated to see whether or not there’s potential to further develop and invest. It’s a process that typically occurs after ideation and is intended to help you not only determine insights, but your most promising product concepts, as well.

Why Do Concept Testing?

Concept testing serves multiple purposes when you’re taking a new product to launch. Some of the most common include:

• Helping to prioritize your best ideas:  You can use concept testing to your advantage whether you’re testing two different concepts, or many. With customer input, you can identify which concept(s) are the strongest, and which have the potential to justify future development and investment.
• Encouraging co-creation with the customer: In working directly with your target customer, you’re able to understand what they like and what they don’t. Give customers the opportunity to work alongside you to help further develop the concept.
• Identifying your best customer segment(s): While you will be recruiting customers against your target segments, you’ll learn which segments are most apt to like the idea the most.
• Uncovering a marketing strategy: Through concept testing, you’re able to test which corresponding marketing messages resonate the most with your core audience, as well as what channels may make sense for product pilot implementation.
• Generating buy-in: Not every concept is going to be accepted by everyone in your organization. Through concept testing, you’re able to uncover customer value and sentiment, which is sometimes what you need to convince your team that this is something worth pursuing.

Smarter, More Effective Concept Testing in 6 Steps

Concept testing can be done in a variety of ways, depending on factors such as the amount of time you have to test, the audience you would like to test with, and whether or not your concept(s) must be tested verbally. This is a six-step structure you can use to get started and modify to suit your organization’s unique needs.

1. Determine your testing methodology.

After you have decided how many concepts you’re giving customers to evaluate, the next decision is whether or not you’ll be doing unmoderated or moderated testing. Factors to consider here include the amount of feedback you desire, the time you have, and of course, your budget.

• Unmoderated Testing: The customer is left to answer questions and complete tasks without a moderator present. It’s performed using an online platform such as UserZoom or MeasuringU, which can help with recruiting and analysis. This type of testing is cheaper than moderated testing and is best when you want to get a large sample quickly.
• Moderated Testing: The customer(s) is observed by the moderator, whether remotely or in-person. This type of testing can take multiple forms, from in-depth 1:1 interviews to focus group discussions.

My colleagues and I find that the most successful concept testing is often a combination of the two.

We recommend starting with unmoderated testing to not only get a large sample size but a gut check on all concepts, then refine before going into moderated focus group or in-person interviews.

2. Develop solid assumptions.

Regardless of the type of product you’re developing, you need to create a list of assumptions and hypotheses that can be tested and refined through this process. It’s important to think through every possible assumption that a concept would require you to make, as this can help minimize future risk. A good place to start is thinking of the concept through the desirability, viability and feasibility lens (DVF).

• Desirability: Does this concept solve the customer’s pain points and needs
• Viability: Does this concept achieve the right business goals?
• Feasibility: Is this concept technically and organizationally feasible?

With a list of assumptions in place, you can then prioritize and decide the order in which they should be tested. This will impact not only script development (Step 4), but the analysis plan (Step 5), as well.

3. Determine your target and begin recruiting.

Since the goal of testing is to understand what your target audience thinks of a concept, one could argue that recruiting is the most vital part of concept testing. You want to make sure that you’re getting the concept in front of the right people (or those that best fit the profile of the end user).

To find the right people, list any pertinent criteria—like demographic, psychographic, behavioral, or consumer traits that best embody the personas you hope to find to participate. As interest comes in, screen accordingly and don’t forget to have a good variance in the final selection. This will make sure you have all segments covered and can use testing as a means to uncover the intricacies between them.

At Method, we recommend always testing with an incentive (e.g., $100 Amazon gift card for a 1 hour session).

Customers are giving their time to provide valuable insights, so it’s important to reward them appropriately. And if you’re having trouble with recruiting, know that there are a variety of recruiting resources to help if budget permits.

4. Create a script.

Concept testing is about uncovering customer feedback that’s honest and unbiased.

Regardless of how well-thought out your concept and assumptions may be, your testing will be invalidated if you “lead the witness” through questioning that essentially directs the tester to an answer. Two examples of leading questions include:

• Would you rather use this website over the previous one? Here, you are implying one is better than the other.
• Was it frustrating when this link didn’t take you where you expected? In this case, you’re implying on behalf of the participant that it’s frustrating and not allowing the customer to provide their own opinion on the experience.

Also known as the ‘cardinal sin’ of concept testing, the likelihood of leading the witness can be curbed if you take the time to a script of the questions that you want answered.

Take these best practices for script development into account to ensure that you’re validating all assumptions while minimizing the risk of unintentionally forcing your participants to an answer inauthentically:

• Avoid questions that lead to one-word responses.
• Avoid questions that are written in a way where you are forcing customers to answer two questions at once. This makes it too difficult to decipher how the customer is weighing the two questions and their response.
• Allow time for introductory questions to help understand the participant’s context before going in to concept specific questions.
• Use uncomplicated, non-technical language throughout the script so that every tester can understand what you’re asking and do not get confused or hung up on a question.
• Test your script beforehand with a small audience internally to catch any leading questions that may slip through.

Keep in mind that concept testing is a chance to learn. While you will want to have questions baked to make sure you have everything covered, what you hear live may be worth exploring further—whether through further questioning or follow-up testing.

5. Determine the analysis plan.

A main driver of concept testing is to understand a concept’s strengths, weaknesses and overall effectiveness against a business objective. When you’re doing qualitative testing, you hear a lot of responses that cannot be quantitatively measured. While these responses and opinions shed light into the value of a concept, it’s important to layer in a quantitative aspect so that you can take the guesswork out of analyzing results.

Ask yourself, how will we analyze the overall acceptance of the concept?

How will we measure key indicators such as likeability, uniqueness and relevance?

We recommend that each concept test employ some layer of quantitative input, whether that means:

• Taking into account the % of task complete
• Assigning a point value to specific responses
• Asking customers to place value to the cost of something

When testing multiple concepts with one customer, you’re also able to ask them to rank concepts against each other. With this type of testing, you can minimize variables by asking the same questions for each concept, so that you can see how each drives different answers.

Setting up an analysis plan from the start will ensure all future read-outs and prioritization efforts go smoothly.

6. Test and analyze.

With your testing methods and audiences selected, it’s time to test the concepts with your audience. The concepts have hopefully taken form in one way or another during this process whether it be a clickable prototype, illustration or another type of [visual presentation], and all parties have aligned to how those will be introduced during testing based on how much time is available.

With all materials in place (and don’t forget a timer!), the moderator can facilitate the test using the script after collecting the necessary consent. And with the moderator focusing on leading the test, it’s important to have someone in the back room collecting feedback from everyone who is listening.

In our experience, the best way to collect live feedback for in-person or focus group testing is to set up and facilitate an observation room.

By doing this live, you’re able to easily group commonalities and identify trends (both good and bad) collectively. Doing so makes any later analysis quicker and more efficient.

Setting Your Concept Testing Up for Success

Regardless of the specifics of your concept testing process, apply these best practices:

• Test early & often: Don’t be afraid of testing. By doing it regularly, you are able to collect the insights needed to be successful.
• Make it what you want: Keep in mind that the beauty of concept testing is in the structure. You can make it as simple (e.g., a statement with minimal description) or as sophisticated (e.g., full, high-fidelity prototype) as you would like.
• Document and evaluate your plan: By carefully thinking through your target audience, testing methodology, analysis plan, and all potential questions, you’re reducing risk and adding unbiasedness to the study.
• Don’t dismiss the value of quantitative data: Always make sure you are, in some capacity, collecting quantitative data so to cut through any subjective opinions and/or trivial feedback.
• Listen carefully: You are not only here to test, but to learn—whether that’s to validate hypotheses, or to uncover potential new ideas that can go back to the concepting stage.

Concept testing can take many forms and serve multiple purposes. Improve your organization’s innovative processes and maximize your resources by deploying the expertise of a trusted innovation partner like Method. Want to learn more? Get in touch and see how we can help.