Researching the right thing versus researching the thing right

The design thinking world has popularized a distinction between “designing the right thing versus designing the thing right,” as visualized by the Double Diamond Design Model. When it comes to researching our customers or users, I believe we also need a similar distinction. 

A double diamond for research?

There are a plethora of resources available on the topic of “researching things right” (think of all the guides focused on methods, tooling, or approaches to data gathering). However, it’s harder to find content on ensuring we’re “researching the right thing” for our organizations. That is, how do we decide the research topics that ultimately yield the most impact for users, organizations, and the ecosystems in which we operate? 

How to synthesize data like a pro through affinity diagramming

As we continue to mature the research practice, it’s on us as researchers to ensure we spend precious research resources on answering the questions that create the most value for users and organizations alike. 

But how do we sort through all the different possible business questions we want answered to find the most impactful ones? 

One approach is this simple set of interactive exercises I facilitate at the start of every research project (both remotely and in-person) to ensure we’re always “researching the right thing.”

To the research team at Adyen, the “right thing” is usually at the nexus of value to the business and ability for the business to apply and implement the insights.

Exercise one: “Things we know”

Sow the seeds

Identify your key stakeholders—the decision-makers, designers, engineers, marketers, or anyone else who will apply and act upon your research topic. If the topic area isn’t yet defined, then the stakeholders are anyone that has a research question. Invite your stakeholders to a (physical or virtual) meeting and ask them to come prepared with:

• Questions about their target audience of interest 

• Things they believe they already know about their target audience of interest

Set the scene

Prepare a physical or virtual space where stakeholders can stick post-its throughout the workshop. The bigger, the better. For remote and virtual meetings, a Figma, Mural, or Miro virtual whiteboard works great. For in-person meetings, a big whiteboard or erasable marker surface works best.

1. The brain dump. Set a timer at the start of the meeting and ask the stakeholder attendees to spend the next 10 minutes in silence writing down all the things they believe they already know about the research target audience of interest on individual post-its. For example, when trying to understand the experiences of beauty product buyers, I asked them to write down all the things they already know about beauty buyers from their past experience with this audience. It’s fine if there isn’t much known—this is itself valuable input for research scoping. There should be one “fact” or idea per post-it

2. In the meantime, draw a vertical line titled “Confidence Scale” in the middle of the physical or virtual board. Label the top of the line “100% confident” and the bottom “0% confident.” As this exercise is intended to spark discussion more than determine scientific accuracy, confidence can be subjective to each person

   

   

   A confidence scale to place customer “facts” on, ranging from 100 percent to 0 percent confidence. We’re feeling more confident already!

3. Sharing & mapping. After the 10 minutes of silent writing are up, ask each stakeholder to come up to the board and read out each “fact” they know about the target audience to the rest of the room. After each fact, they read out, ask the stakeholder how they learned this fact. Then ask them to plot it on the vertical line you drew, based on how confident they feel about this fact. Since you stakeholders are reading their “facts” out loud, interestingly, sometimes the stakeholder participants in the room will contradict each other’s “facts,” signaling a ripe area for research insights to settle the discussion 

4. Each participant should get a chance to read out the things they “know” and plot them onto the vertical line

Now you should have an idea of what your stakeholders already believe to be true about your users. Interestingly, it often happens that information about users that stakeholders are convinced is “true” turns out to not be true (or far more complex) during the research. These delta between perception and reality are some of the most interesting insights to highlight for stakeholders during the insight sharing phase. The tension between their conviction and the actual truth is a natural attention-grabber. 

For example, I once had a group of product managers in a room convinced that their target users, restaurant owners, highly preferred to structure their reports in a specific way. Research interviews with these owners revealed a range of report preferences due to their varied organizational structures. This was an insight that I highlighted in the research insights presentation as I knew that it would be interesting. As a result, it was acted upon by these product stakeholders.

How to influence leaders and communicate the value of research

Exercise 2: “Things we need to know”

This second exercise can follow the first within the same meeting. As you’ve already asked participant stakeholders to come with their questions in mind, everyone should have what they need to map out all the questions and prioritize them as a team.

1. The brain dump. As with the previous exercise, set a timer and ask your participants to spend 10 minutes writing down all the different questions they have about the target audience. Remind everyone to limit each post-it to one question. It’s okay if the questions aren’t worded as research questions yet—you can rephrase them in your notes and share them with the group in the next step. (Sometimes, questions about technical and operational feasibility will sneak in, but beware that the primary focus should be questions about the people served by a product, service, or idea your team is working on. Reminding the stakeholders to focus on questions about experience and attitudes helps keep the session as relevant as possible)

2. Meanwhile, draw a chart with the X-axis titled “business impact” and the Y-axis titled “feasibility” in the middle of the board. It doesn’t matter where you draw it in relation to the previous exercise

   

   

   You’re already on your way to finding the ideal research questions to work on. Keep it up!

3. Sharing & mapping. Once 10 minutes have expired, as with the previous exercise, ask each participant to come up to the board and read off each question on their post-its. After each question they read out, ask the stakeholder what they would do if they had the answers to this question. Questions like, “What would the answer(s) to this question allow you to accomplish?” Based on their reply, ask them whether this accomplishment is high impact for the business (the right side of the horizontal line), low impact, or somewhere in between.

   For the same post-it question, now ask, “Is it currently feasible to implement or apply the answer to this insight to the product?” Based on their reply, ask them whether feasibility is high, low, or somewhere in the middle of the Y-axis. Ask them to plot the post-it on the appropriate point in the chart relative to the business impact and feasibility. For example, if the question on the post-it is: “Would beauty buyers use an app to purchase cosmetics?” and after you ask some straightforward questions on impact and feasibility, you might find while a new app would be beneficial to the company, there aren’t enough resources to deliver it.

   

   

   Ask your stakeholders where they would place their question relative to its feasibility and business impact.

4. Repeat sharing and mapping steps until all stakeholder participants’ have plotted their post-its. I recommend taking notes as each participant is sharing their questions to keep track of the ultimate aim of each question.

5. After each stakeholder has plotted their questions, the chart could look something like this:

   

   

   You should start to see questions emerging with both high feasibility and business impact! This means you’re going in the right direction.

   As business impact and feasibility of application are relative among the plotted questions, stakeholders might want to adjust the positioning of some of the early plotted post-its. At this point, ask everyone to look at the distribution of post-its and speak up if anything doesn’t feel right. 

6. Ideally, the research questions we prioritize are the ones that scored high on both business impact and feasibility-—the top right corner of the chart. Next, it’s time to collectively decide where the delineation for the most impactful and feasible questions are. This division will vary depending on the number of questions, the amount of time you have to answer them, your resources, etc. But the goal is to draw a dividing box where everything in the top right box is what you will scope into the first iteration of research because it’s the most impactful low-hanging fruit. Everything outside of the box can be parked and revisited later.

   

   

   Up and to the right—there’s your prioritization sweet spot 👌

There isn’t a science to determine what is impactful enough to make it into the first round of research—it should be a decision based on your unique team needs, urgency, and resources for each project. An additional or alternative axis to the question prioritization chart could be research effort needed to answer the question—though this is often something I, as the researcher, assess for each question in the top right corner after the workshop. Based on the highest mix of business impact, implementability, and effort, I report back to the team with my recommended initial research scope and follow-ups for future research iterations. 

As Clayton Christensen said, “Questions are places in your mind where answers fit. If you haven’t asked the question, the answer has nowhere to go.” In addition to prioritizing the research scope, these exercises also help create a space in stakeholders’ minds “where answers can go” by deliberately getting them to ask the research questions. This focuses precious research resources on gathering insights that have a higher chance of being acted upon and positively impacting the business. 

Have you tried other research question prioritization exercises? Or did you give these two a spin? We’d love to hear how it went! Drop us a comment on social media or join us in the Dovetail Community Slack channel to discuss all things research-related. 

Written by Katarina Bagherian, User Experience Research Team Lead, Adyen. Katarina leads the Research team at Adyen. She is a qualitatively-trained researcher obsessed with basing business decisions on reliable user needs. Her fascination for understanding human behavior at the systemic level led her to study Political Science at the University of California, Berkeley. She cut her teeth in applied user research at startups in San Francisco, where she had to prove the impact of research early and often. She developed a passion for making research efficient and accessible as a consultant for entrepreneurs and multinational corporations alike in Amsterdam. Katarina loves spreading the power of user research by guest lecturing at local business innovation university programs. She is always on the lookout for curious researchers to join the growing strategic research practice at Adyen or simply nerd out about creative research approaches.