Many companies now realize that creating user-centric products is the only way to succeed. They’ve seen the benefits of prioritizing user research as a way to identify user needs, inform and validate design decisions.

While there are many techniques to conduct user research, they mostly fall into qualitative or quantitative user research categories. Qualitative studies provide subjective information, while quantitative studies provide objective information. Both research techniques help designers evaluate a product and decide whether a full or partial redesign is required.

In this article, we’ll discuss:

• What is quantitative UX research
• Why do quantitative research?
• When to conduct quantitative user research
• Examples of quantitative methods
• What is qualitative UX research?
• Why is qualitative UX research important?
• When to conduct qualitative UX research
• Examples of qualitative methods
• Differences between quantitative and qualitative UX research
• Combining quantitative and qualitative user research to achieve your research goals

What Is Quantitative User Research?

Quantitative UX research collects and analyzes numerical data. It also predicts and generalizes findings about a topic or target audience. Quantitative research leverages a large sample size to produce objective, measurable data about a user population. The goal of using a quantitative research method is to measure the usability of a product. You can also use quantitative research to analyze product performance against competitors.

Quantitative research methods aim to answer questions such as: 

What

Where

When

Why Do Quantitative User Research?

• Collect information quickly
• More trust in quantitative data because the result is numerical and can be measured
• Data can be easily presented as charts, graphs, and other graphics
• Easier to avoid human bias as it is difficult to sway participant outcomes in a highly controlled testing environment
• Assign numbers to the outcome, which gives your research more validity and makes it easier to get buy-in from stakeholders
• Understand if specific changes lead to different outcomes
• Less contact with research participants since it can be conducted remotely

When to Do Quantitative User Research

Conduct quantitative user research when you want to:

• Measure usability and satisfaction with standard measures like the Net Promoter Score and System Usability Scale
• Compare your product to competitors
• Evaluate key performance indicators, company goals or justify investment in UX design
• Validate or explore designs and concepts with a large sample size of at least 50 participants
• Capture user behavioral data for task-based research, heatmaps, and click-path 
• Determine if a product redesign is worth doing

Examples of Quantitative Research Methods

Analytics

Analytics shows what users are doing on your live website or app. It’s one of the most valuable sources of quantitative data. Data to focus on include:

• Click-through
• Entry pages
• Exit pages
• Conversion rate
• Pageviews
• Bounce rate

The information helps you track content page performance, product features, and UI. You can also use the data to determine if you need a partial or full redesign.

Tree Testing

During tree testing, you remove the user interface. Then, participants navigate the site and complete tasks using the category structure of your website. It’s a great way to evaluate your website’s information architecture by isolating it from other parts of your UI.

For example, imagine an online clothing store that sells both Female and male clothing. The hierarchy would look like this:

You could ask participants to find six-inch heels. However, they’d only see the top-level categories of men and women. It’s only after clicking an option that they see the correct categories.

Use the tree testing result to determine if your information hierarchy and placement align with users’ expectations.

Card Sorting

During card sorting, you give research participants a list of items. Next, you’ll ask them to group and label the items in a way that makes logical sense to them. The test can be conducted remotely through a card sorting platform or in-person with physical cards.

It’s a great way to get inside the user’s mental model to understand the terminologies they use and how they logically group concepts.

Options for card sorting include:

• Open card sort where you ask participants to group and label cards into categories that best describe the content
• Closed card sort where participants sort cards into predefined and already labeled groups
• Hybrid card sort where participants sort cards into already defined groups but also have the option to create their categories if they think the architecture is inaccurate

A/B Testing

During A/B testing, you create experiments to determine which UI design performs better. Experiments can be conducted through third-party software that helps you set up two web pages with different elements you want to test.

You could decide to test:

• Call to action text
• Headline text
• CTA colors
• Product placement

It’s important to note that you’re testing one element at a time during A/B testing. If you want to try multiple design elements at once, then you’d run multivariate testing. 

Desirability Studies

Desirability studies attempt to measure and quantify some qualities of your brand or its products. Qualities could be brand strength, aesthetic appeal, or tone of voice. While you can customize quantitative desirability studies around research questions, exposing participants to your product is the first step.

The product could be your live website, a product prototype, or even a marketing copy with images. Next, you’ll ask research participants to describe what they see from a pre-selected list of descriptive words.

Eye Tracking Testing

During eye-tracking, you can see where participants are looking on a screen as they perform tasks. Eye tracking requires special equipment. 

Source: Usability.gov

As the user looks around the screen, the equipment generates a heat map of where they concentrate the most. The results help UX researchers to determine which content elements and interface need to be emphasized or deemphasized. It ensures users reach their goals when using an app or website.

Surveys and Questionnaires

An online survey allows you to collect feedback from users. You can administer the test as an intercept on your live website, via a usability test, or email. Surveys are quantitative because you’re asking rating-type questions to get a large enough sample size that backs your decisions. 

You can create a custom survey or use established questionnaires like the Net Promoter Score or the System Usability Scale.

What Is Qualitative User Research?

Qualitative user research is any UX research technique where results are comments, observations, feelings, or narrative descriptions. Qualitative methods assess the usability and user experience of a product. 

The goal of qualitative research is to collect and analyze non-numerical information. Qualitative research helps you understand the why behind quantitative research. For example, if you notice that a particular product page on your website has a high bounce rate and poor conversion, qualitative data clarifies the bounce rate.

Why Do Qualitative User Research? 

The benefits of qualitative UX research include:

• Allows creativity to be the driving force
• Easy to organize since you only need a few participants 
• An open-ended research process that enables researchers to unearth emotional data that influences the design process
• Reveals information you won’t get from quantitative research and explains why statistical trends occur
• It’s easier for users to express themselves in their own words instead of predetermined values or numbers

When to do qualitative user research

Scenarios for qualitative research include:

• Make formative decisions early in the design process
• To identify usability issues when the product is in development or completed
• Find solutions to usability issues
• During product redesign, when you have the resources to conduct more in-depth research and consider a broader range of possibilities

Examples of Qualitative Research Methods

Ethnographic Field Studies

During ethnographic field study, you observe research participants as they perform tasks in their natural habitat. Unlike other field study methods, ethnography takes a holistic view to learn about the needs and goals of your target audience. 

By leaving the comfort of your home to visit participants in their environment, you can find issues that a lab-based result won’t show. This is because you observe how users interact with your product in their environment, not a controlled environment that may influence user behavior.

Focus Group

A focus group is a research technique used to identify and explore how people behave and think. You collect data through group interactions of three to 10 participants who gather in a room to discuss a topic. You moderate the focus group and ask open-ended questions during the session.

Use Aurelius to analyze results and present research findings to stakeholders

You can use focus groups to:

• Explore public perception of a product or brand
• Identify how consumers use a product
• Learn more about customer needs
• Understand whether customers are satisfied or dissatisfied with a product, service, or brand
• Uncover usability flaws with your product

User Interviews

User interview is a popular technique that helps researchers understand a user’s feelings, needs, and motivation towards a product. During user interviews, you’ll meet one-on-one with a research participant to ask questions about using a system, behavior, and habits.

You can conduct user interviews during exploration when you have an early model or after launch to observe user interaction with the product.

User interviews are a great way to plug knowledge gaps in your product and discover your best features. You’ll also find out if your personas are an accurate representation of your target audience.

Diary Studies

A diary study is a UX research method that captures user experiences, behaviors, and attitudes. It’s not a one-day activity but takes place over an extended period to properly integrate the product into the participants’ daily lives.

During diary studies, the user logs daily activities as they happen. The goal is to provide contextual insight about real-time user needs and behavior to help researchers define UX feature requirements.

Conduct diary studies to understand long-term behavior such as:

• Motivators for users to perform a task
• What users are thinking and how they feel after completing the task
• The time of the day when users engage with your product
• The problems they solve with your product or app as well as other products in their workflow
• Changes in brand perception over time as they continuously use the product

Moderated Usability Testing

A trained moderator facilitates moderated usability testing. The moderator works with the test participant to guide them through the study and answer questions when they encounter challenges during the task.

Moderated testing is conducted either remotely or in person. For example, the venue could be a UX lab or remotely via a usability testing app. When performed remotely, participants share their screens with the moderator as they complete the test in real-time. In addition, they think aloud to help the moderator understand their thought process as they go from touchpoints A to B.

What Are the Key Differences Between Quantitative vs Qualitative UX Research?

Combining Quantitative and Qualitative User Research to Achieve your Research Goals

One is not better than the other. You can combine both qualitative and quantitative methods to get the best results.

For example, while Google Analytics tells you the number of visitors that visit your site, it doesn’t tell you why they visited your website. Combining that data with a live intercept tool like Ethnio will help you understand user motivations for clicking on your link.

Without quantitative research, all you have is data and a false narrative. False narrative undermines your hard work because you’ll be optimizing based on intuition, and that never bodes well for user-focused research.

Qualitative vs Quantitative UX Research Methods: Which Is Better?

Both quantitative and qualitative research provides different datasets. However, they are equally essential for conducting holistic research that yields actionable insight. Alone, neither can give you the complete picture. Together, you’ll make informed product decisions that align with your business goals. 

They reinforce each other to produce unbiased research results. There’s more trust in the findings when quantitative and qualitative techniques yield concurrent results.

FAQ about Quantitative vs Qualitative User Research

When should you do qualitative versus quantitative user research?

Use qualitative research at the beginning of the design process when you’re still exploring or trying to uncover thoughts and opinions. Use quantitative analysis when you have a prototype or working product, and you need to define a problem or fix usability issues with your product. You can also use qualitative research to dig deeper into findings from quantitative research.

Is UX research qualitative or quantitative?

UX research is both qualitative and quantitative. They complement each other to collect data and understand the why behind human behavior and emotions.

What are the similarities between qualitative and quantitative research?

They both rely on a theoretical framework. You can use quantitative research to test a theory and to generate hypotheses. In the same vein, you can use qualitative research to test theories even though it is used to generate theories. Also, they discover behavioral patterns to provide insight into how consumers use a product or service.

Find out how Aurelius can help you organize and analyze both qualitative and quantitative research data 

(no credit card required)