RECAP: USER RESEARCH FOR EVERYONE

Toledo Region UX Conference

In my role as Digital Project Manager at Thread Marketing Group, I had the privilege of attending the Toledo Region UX Virtual Conference, “User Research for Everyone.” The virtual conference was hosted and broadcast by Rosenfeld Media, a well-known name in the realm of user experience expertise.

Web professionals from area businesses waited outside of the newly-built King Road Branch of the Toledo Lucas County Public Library. The clean white walls and modern feel leant the perfect atmosphere for exploring the idea of research and learning practical applications for using data to design better digital products.

Attendees claimed seats and outlets with their laptops and power cords. Some of us knew each other from previous meetings and business endeavors. After chit-chatting and exchanging news from the past couple months, we settled in for a day of listening.

User Experience Research Takeaways

The talks for the day consisted of seven 1-hour sessions, each covering a slightly different aspect of user research. While the amount of information was overwhelming and too much to absorb in one day, there were several themes and nuggets I pulled out that any company can easily learn and apply if they have a desire to do so. Here is a summary of each talk:

Just Enough Research

While all humans have a desire to want to be “right,” this desire and mindset can be a major opponent to research. For someone who races to an answer just to prove they were right, they can be perceived as annoying and a bad listener, plus a rush job increases the chance that you are wrong.

Therefore, if you ask the question and get one answer, it is a mistake to quit asking. The answer might change over time; to ask the question and quit asking is not a research mindset. Online marketers, web developers and UX professionals can use this logic to back up why periodic user research is crucial for company success.

Erika Hall stated that one of the biggest benefits of research is that it puts everyone, your internal team and the customers you are creating for, on the same page. People are challenging, so to make good design you need to work with real people in the real world. You cannot design (or write) for a world that you hope exists. Thus, is it important to get just enough research to rid your process of assumptions and give you confidence in your decisions.

However, Hall warns that just because you have the facts does not mean that leadership will automatically change their plans to agree with you. Asking questions can be scary because it means that you do not necessarily have the right answer. In some cases, the introduction of research facts changes the basis of decision-making from having the right decision internally to reflecting external reality (users’ actual needs).

How to conduct quick, easy research? Think simple.

• Form questions that are specific, actionable, and practical.
  • Bad: “How can we get millennials to like us?”
  • Good: “How do recent college grads decide what to eat for dinner?”
• Gather data (observing, interviewing, reading)
• Analyze data. Insights can feed into any process to increase your chances for success.

The Right Research Method for Any Problem and Budget

Leah Buley, the second presenter of the day, aimed to answer three of the most common questions companies have about conducting research:

1. How do I pick which research to do?
2. How can we do it faster and cheaper?
3. How do I turn the research into better products?

In most companies, research is treated as a departmental concern (“research department”) when it should be the concern of the whole company. Think of it this way instead: research is a series of questions that build on each other. All silos and roles at a company should be interested in at least one aspect of research.

• What is happening around us?
• What do people need?
• What can we make that will help?
  • Find the MVP (minimum viable product) with just the features that “have” to be there.
  • Feed the basic features to employees to get additional feedback and add more features.
• Does our solution actually work?
  • Utilize beta testing, mining analytics, dogfooding, heuristic review, RITE testing
• What is the impact? Set goals, signals and metrics for each element of the HEART tool below.
  • H - happiness
  • E - engagement
  • A - adoption
  • R - retention
  • T – task success

How to Find and Recruit Amazing Participants for User Research

One of the trickiest elements of collecting user research is finding people to participate. When building a new website or digital experience, the amount of data that goes to the design team is likely 99% data and 1% human interaction. In an ideal world, those numbers should be radically different… probably closer to 75% human interaction and 25% data.

Nate Bolt suggested that not all user research has to be long and arduous. Some human input, even if it is “man on the street” research or literally the first person you see, is better than nothing. Of course, the method and type of tester will vary greatly depending on whether or not you are getting input on something that is already built or something that is in the mid-design cycle.

Here is a summary of Bolt’s suggested eight methods of finding recruits:

1. Literally the first person you see. Use when there is little to no budget, time pressure, and there are no skeptical stakeholders.
   1. Pros: Little to no budget, broad audience, no skeptical stakeholders
   2. Cons: Dynamic screening -  the questions you ask the first person could be different than the questions you ask the second question.
2. Someone in office/friends/family. Use when the project you are working on is top secret and the world cannot see, you are early in the design cycle, and there is no budget or time.
   1. Pros: They have an attachment to you and therefor are more likely to participate easy to find, fast and free
   2. Cons: Bias, can be hard to schedule, no incentive.
3. Friends/Family. Same as someone in the office.
   1. Pros: Same as #2
   2. Cons: Same as #2
4. Use when participants need to be minimally screened.
   1. Pros: All walks of life, inexpensive incentives
   2. Cons: No attachment, sometimes slow, embarrassing to admit.
5. Recruiting agency. Use when you have more budget than time.
   1. Pro: Least amount of work, fairly fast
   2. Cons: Expensive, users have no attachment to success of end product.
6. Panel. Use when you have an online research tool that provides users, and your organization does not allow other methods.
   1. Pros: Vetted, known technology has been tested.
   2. Cons: Limited targeting, professional tester alert
7. Facebook Ads/Twitter. Use when you need to reach an audience that you have zero contact with. Example: triathlete mechanics
   1. Pros: Can be incredibly targeted; access to populations even agencies might not have access to.
   2. Cons: Attachment (potentially); gets expensive if you run targeted ads.
8. Live User Intercepts. Use when your website or app has decent traffic (>50 pageviews a month).
   1. Pros: In the moment, targeted
   2. Cons: Scary, requires some engineering resources or high traffic.

View the full list of pros and cons for each method on Nate’s slides on SpeakerDeck.

Do-It-Yourself Usability Testing Discussion, Q&A

The lunch hour was occupied by a Q and A session with Steve Krug and Laura Klein. All conference attendees were able to submit questions virtually as Steve and Laura discussed them on-screen. They deepened an earlier discussion on what companies and agencies can do to simplify user research. Some tips:

• Choose research activities and methods that nearly everyone can do, then just do them without wasting time in a long approval process.
• Perform regular, quick tests. Perhaps one morning a month, with three or four people. Participants can picture fitting one morning a month into their schedules. Just be careful that the goal of what the testing is intended to teach/what the company wants to learn remains the focus, and the recruiting does not take front and center.
• Do usability testing of competitors to see where they are lacking and where your product could exceed.
• Beware that when there are few (1-4) test subjects some could be outliers. However, major problems should be so obvious that there is no need for “proof.” Example: a person walks by and trips on the carpet. A second person walks by and trips on the carpet. How many people need to trip before you identify it as a serious problem?

Creating a Virtuous Cycle

Even when regular or continuous user testing is a part of your regular process, it is easy to get sucked into a scary cycle. When the focus of the team is so narrowly zoomed-in on fixing the issues that were discovered during research, the fixes can be quite less than elegant.

Take the example of this stylecode buying experience on Amazon. When I try to add the item to my cart and hover my cursor over the button, I am greeted with an ugly, alienating “Do not pass” symbol and a message prompting me to select a size.

The experience is jarring. It is likely that this issue was discovered from usability testing, and instead of altering the design in a more elegant fashion to place the size selection near the “Add to Cart” button they ended up with this.

If multiple issues are discovered over time, the reactive, fast solutions that are common such as interstitials and pop-up can get messy and cluttered.

Julie Stanford’s suggested solution? Make sure the researcher and the design team are on the same page. Actually, test the specific attributes that need to be tested. This can be handled in two ways:

1. Research and design are the same person.
2. Follow a process

Regarding #2, Julie suggests the “Virtuous Loop” formula:

1. Question
   1. List all the questions you have about the design.
   2. Prioritize the questions based on the 20 cases/tasks.
   3. Answer most burning 1-2 questions.
2. Mind meld
   1. Researcher and designer go over each section or page and agree on what will be covered together.
3. Attend and participate
   1. People responsible for product decisions should attend research sessions. (“Too busy” is a bad excuse.)
4. Evaluate findings
   1. Give each issue a severity rating.
   2. Remember to do the “forest through the trees” check.
5. Brainstorm
   1. Don’t just go with the first solution that comes to mind, even if it seems obvious. Think of multiple solutions.
6. Evaluate solutions
   1. Ask questions such as, “What impact does this solution have on the overall page?” and “Will this work on subsequent use or is it only useful once?”
   2. Sketch break. Stop talking, sketch quickly, discuss.

Some readers might react to the above by saying, “I don’t have time to be this thoughtful.” Think again; you might be moving fast in the wrong direction.

View the rest of Julie’s slides.

Making Sense of Research Findings

A common thought related to research is that the output of findings is just another mess to make sense of. While this situation is frequently true, the mess can be prevented by planning a few elements of the research process more carefully.

First off, consider the sources. Effective research does combine various sources, but confusing the sources can lead to more time in laborious interpretation after the fact. Abby Covert places her “sources” into three categories:

• Observations, which the test facilitator sees the test taker do.
• Verbatims, which the facilitator hears the test taker say.
• Interpretations, meaning what the facilitator or researcher believes that the test taker means.

Regarding note-taking, be sure to choose a format that is best for you, regardless of whether you are taking notes. Well-organized note-taking can save hours of time in analysis later. Abby showed a response matrix that was color-coded for personas to quickly find trends amongst personas.

Protip: do not rush the “conducting” part of the user research. Schedule in time to vamp up notes immediately after the session you just completed, when the spirit of that participant is still there. Get observations and verbatim onto paper as quickly as possible. Interpretations can be constructed later.

When it comes to organizing the results later, group the results to easily see trends.

• Equivalence
• Association
• Hierarchy
• Taxonomy creation

For easy and engaging presentation of your findings to executive leadership or whatever other passive audience watches you present, present each finding with the following elements:

• Headline
• Supporting points
• Methodology
• Impact of finding
• Recommended action
• A/V support (video clip of user using something, screenshot with arrow, chart of data)

Infectious Research

To wrap up the day, Cindy Alvarez expanded on the theme of how to present research findings in an engaging, easily-digestible way. If the research team can present findings in a “contagious” way, it can make research a higher priority in your company or agency and ultimately lead to better products.

First and foremost, keep results short! It is common to write a multi-page report that is text-heavy and a burden to read. While this type of documentation is useful as evidence for change, it is not conducive to an exciting presentation. Make a tl;dr (too long; didn’t read) version that is 5 bullet points. That’s it!

When explaining research findings, start with a hook! The hook can be anything compelling. Good examples are an easy-to-understand statistic, or maybe a photo of the regular environment where users interact with your product (think beige cubicles).

When trying to make your research infectious in other departments and level in your company, talk about it and share it frequently. Because of the mere exposure effect, people start liking things merely after hearing about them or seeing them repeatedly.

Furthermore, your colleagues and upper management want to sound smart.  Equip delegates/allies with your research so they can remind other team members of it.

And finally, be infectious.

• Focus on the mantra of “Better results,” not “better research”
• Don’t worry about who gets the credit.
• ABS: Always Be Storytelling.
• Give everyone a way to help, whether that is by notetaking or by becoming an ally in sharing the findings.
• Repeat, repeat, repeat!

Do you have any user experience pro tips to share? We’re listening at Thread. Please share your idea as a comment on our Facebook page.