A complete user research approach involves both the quantitative and qualitative research process.
Seasoned experimenters know that quantitative research without qualitative research leads to an incomplete conversion user research that leaves many questions than answers.
The questions you’re seeking to answer are not how my web visitors interact on my website/app (quantitative research already covers that) but why are they acting the way they do? What are their intentions and motives?
This is where the qualitative research approach comes into play.
Except you’re a mind reader, qualitative research as a conversion rate optimization process is how you get to know the why of your customers and web visitors.
You get to mine unfiltered customer insights by asking them open-ended and closed-ended questions.
As the name suggests, open ended questions are free-form questions that allow customers to air their opinions without restrictions. In contrast, close ended questions have a YES/NO reply format or minimal options.
The mistake I see many marketers and CRO specialists make is the assumption that it is just questions, so they’ll be easy to formulate and ask.
No! You couldn’t be more wrong.
Here’s a scenario that drives home my point.
If you ask the wrong questions, you’ll get the wrong answers, leading you to wrong conversion blockers that you’ll then prioritize, run A/B tests, and get skewed results.
Now, we don’t want this type of monumental error, which is why this article.
We’ll be looking at what’re open ended questions, and close ended questions, what user research is, examples of open ended and close ended questions, different types of qualitative research, how to apply open ended questions and close ended questions, and the impact of open ended questions and close ended questions on user research.
Ready to learn more? Strap in, and let’s get started.
What Are Open-Ended Questions?
Open-ended questions are questions that don’t restrict the participants in the answers they provide.
What this means is when you ask a participant a question, you expect them to elaborate on their points.
Open-ended questions offer a deep pool of insight into your customer’s mind because the interview process is free-form, and the participant is expected to give you feedback instead of stock answers from a limited answer pool.
What Are Close Ended Questions?
On the other hand, closed-ended questions require a yes/no reply or have limited responses that the participants can give.
Close-ended questions give limited insight to the research’s conversion rate optimization specialist because the participants are expected to respond from a limited pool of answers.
A significant advantage of close-ended questions is that they can be asked to many participants, and you can analyze them for quantitative data (more on this later).
Types of Survey Questions
The beautiful thing about survey questions is that they can be used in open-ended questions and close-ended questions.
An important point to note is this; before writing your survey questions, ask yourself this question; how do you intend to use the answer from this survey? Do you want quantitative data or insights into the mind of your customer?
This will help you in the framing of your questions.
Below are some popular types of survey questions you can ask for quantitative data and qualitative insights.
-
Dropdown questions:
It’s an easy way to display a long list of answers to your respondents only when they click on it. This question type makes your survey look clean and not busy.
-
Multiple choice questions:
This question type offers multiple answers and allows your respondents to select one or more solutions. They’re intuitive and easy to use.
-
Open-ended questions:
This question type does not present any predefined answer; it allows the respondents to answer however they like without restrictions.
-
Slider Questions:
The question type is represented by a slider with a figure starting from 0 or 1, and it only moves higher gauging customer satisfaction.
-
Demographic questions:
This question type is perfect for when you want to gather information on your audience’s background, income level, what interests them, etc.
Open Ended Question and Close Ended question; What’s The Difference?
The significant difference between open and close-ended questions is that close-ended questions provide the respondent with limited answers they must choose from. You can’t air your opinion the way you want, and your response is minimal.
On the other hand, open-ended questions allow the participants to respond as they wish, no holds barred. You’re given the opportunity to share your opinion and your thoughts entirely.
Examples Of Open-Ended and Close-Ended Questions.
Improving the customer experience on your website does not start with an A/B test experiment; it begins with asking the right questions.
Only a mind reader does not need to ask people questions to know how they feel and how they’ll respond.
As a marketer, if you want to know more about your web visitors’ experience on your website and uncover conversion blockers, you need to hear from them by asking the right questions.
Here are some examples of open-ended questions and close-ended questions we’ve asked our client’s customers that have helped our conversion strategy and informed our hypothesis and A/B tests.
The Importance Of Asking Open-Ended Questions.
The Importance Of Asking Open-Ended Questions.Open-ended questions are the way to go if you want to mine a lot of information on your web visitors and hear about their experience and what you can do better.
Open-ended questions don’t put your respondents in a box. You’ll be surprised to hear answers and strategies you didn’t know about and won’t usually factor in.
At Invesp, we use open-ended questions in our Jobs To Be Done (JTBD) process when interviewing customers of clients, and we have uncovered mental models and conversion blockers that otherwise told us we won’t have known.
When To Ask Open-Ended Questions.
1. When you’re conducting testimonial interviews.
2. Discovery calls with web visitors who are yet to purchase from you.
3. Informational interviews with customers who have purchased at least once.
4. User persona research to know which set of web visitors to build out your processes for.
5. For design research to know which problems to solve and what solutions are most important to your ideal persona.
6. Use case research.
Advantages of open-ended questions:
1. Encourages deeper thinking:
Open-ended questions require more thought and reflection, leading to deeper understanding of a topic.
2. Allows for creativity:
Respondents have the freedom to express their thoughts and ideas in their own words, leading to more creative and diverse responses.
3. Provides richer data:
Open-ended questions often yield richer and more nuanced data than closed-ended questions, as they allow respondents to share detailed opinions and experiences.
4. Unearths unexpected insights:
Open-ended questions can reveal new information, ideas, or perspectives that the researcher may not have considered.
5. Enhances communication:
These questions promote meaningful discussions and can help build rapport between the participants and the researcher or interviewer.
Disadvantages of open-ended questions:
1. Time-consuming:
Analyzing and interpreting open-ended responses can be time-consuming and labor-intensive, as they require thorough examination and coding.
2. Requires more effort from respondents:
Open-ended questions demand more cognitive effort from respondents, which can lead to response fatigue or reduced response rates.
3. Potential for bias:
The way an open-ended question is asked or framed can influence the responses, potentially leading to biased or skewed data.
4. Difficulty in quantifying data:
The qualitative nature of open-ended responses can make it challenging to quantify and analyze the data using statistical methods.
5. Inconsistency in responses:
Open-ended questions may result in varied or inconsistent responses, making comparing and contrasting the data difficult.
Tips On Asking Open-Ended Questions
If you aim to ask open-ended questions, your questions should follow this format;
1. Why, how, describe, tell me, can you explain further, etc. These question types let the respondents know they should air their minds, opinions, and thoughts.
2. When interviewing a participant, aim to collect more information and not one-worded responses because that defeats the purpose of open-ended questions.
3. Don’t make your questions so broad that people get confused.
4. Don’t get carried away with asking open-ended questions that you forget to obtain quantitative data in your customer research stage.
5. Don’t guide your respondents into giving you specific answers; that leads to bias which distorts the results.
The Importance Of Asking Close-Ended Questions.
It’s true that close-ended questions limit respondents’ answers, but that’s where its power lies.
You can limit unnecessary answers that often happen in open-ended questions with close-ended questions.
Unlike open-ended questions, where you have to book several meetings many times with respondents, with close-ended questions, you get a higher response rate because you don’t require much from the participants.
When To Ask Close-ended Questions.
1. In surveys and polls where you’re expecting a large number of responses.
2. When the answers to your questions are limited for specific reasons.
3. When you need to be able to measure user responses quantitatively.
4. You want to convert participants’ opinions into reliable data.
5. You want to collect consistent and reliable data.
Advantages of close-ended questions:
1. Easy to analyze:
Close-ended questions produce quantifiable data, making it easier to analyze, compare, and draw conclusions from the responses.
2. Time-efficient:
Respondents can answer these questions quickly, resulting in reduced survey or interview time and higher response rates.
3. Standardized responses:
Close-ended questions ensure uniformity in the answers, which allows for accurate comparison across respondents.
4. Simplifies decision-making:
These questions provide clear options for respondents, making it easier for them to choose and express their opinions.
5. Facilitates statistical analysis:
The quantifiable data collected from close-ended questions can easily be subjected to statistical analysis to identify trends and patterns.
Disadvantages of close-ended questions
1. Limited depth of information:
Close-ended questions restrict the range of possible responses, leading to less detailed and nuanced information.
2. Predefined options:
The answer choices provided may not accurately represent the respondent’s views or experiences, potentially leading to inaccurate or incomplete data.
3. Reduced creativity:
These questions discourage creative or unique responses, as respondents must choose from a limited set of options.
4. Potential for bias:
The way answer choices are presented can influence respondents’ decisions, introducing bias or skewing the results.
5. Lack of context or explanation:
Close-ended questions don’t allow respondents to explain their choices or provide additional context, which can limit the depth of understanding.
Tips On Asking Close-Ended Questions
This section is not meant to be exhaustive but to serve as a guide to form your questions.
1. Begin your questions with have, do, will, or did.
2. Write your questions in an easy-to-understand format. Understand that it’s a mixed multitude that’s likely to see your questions, and they’re not the expert; you’re. Write the question in a way that they will understand it.
3. Provide relevant answers to the questions you’re asking. This is a significant advantage of close-ended questions because you can filter the noise.
4. Don’t encourage lengthy answers to your questions. You can follow up with an open-ended question to get a more detailed response.
5. To avoid boring your respondents, switch up your response structure from yes/no to multiple answers, checklists, etc.
6. Don’t use the format of a close-ended question to explore the emotions/feelings of respondents; it’s not meant for that.
7. When crafting the answers to your questions, ensure they cover the range of possibilities.
How Do Open-Ended Questions and Close-Ended Questions Affect Sales?
Open and close-ended questions are critical in the sales process. Understanding when to use each question type can have a massive effect on any sales interaction.
Open-ended questions:
1. Build rapport:
Open-ended questions encourage the prospect to share their thoughts, opinions, and experiences, helping to establish trust and rapport between the salesperson and the prospect.
2. Gather information:
These questions allow the salesperson to gather valuable information about the prospect’s needs, pain points, preferences, and decision-making process.
3. Enhance engagement:
Open-ended questions promote engagement and active participation in the conversation, helping the prospect feel more involved and invested in the discussion.
4. Identify opportunities:
By exploring the prospect’s responses, salespeople can identify opportunities for upselling, cross-selling, and tailoring their offerings to meet the prospect’s needs better.
Closed-ended questions:
1. Confirm understanding:
Closed-ended questions can be used to confirm understanding, verify facts, or quickly obtain specific information from the prospect.
2. Narrow focus:
These questions can help direct the conversation toward a specific topic, allowing the salesperson to address a particular concern or issue.
3. Establish control:
Closed-ended questions can be used to guide the conversation or regain control of the discussion if it veers off track.
4. Facilitate decision-making:
By presenting the prospect with clear choices or options, closed-ended questions can help move the sales process forward and encourage the prospect to make a decision.
The Impact Of Open Ended Questions and Close-Ended Questions On User Research
The mistake of assumption is why many marketers make decisions not backed by user research and wonder why there’s no improvement in conversions.
Asking your web visitors the relevant questions to get the correct feedback in their voice is critical in understanding your quantitative results (the data you get from Google analytics and other web analytics platforms).
At the end of the day, if you decide to go with open ended questions or close ended questions, make sure your customers have sufficient knowledge of your website/product and the questions you will ask them fit their phase in your funnel so they can give you the correct answers.
You don’t want to be caught asking a first-time visitor about his experience purchasing your product when he’s just at the homepage. This question is meant to be seen by people who just placed their order.
Pro tip: Both open-ended and close-ended questions have their place in the customer research process. With a proper balance of both types of questions asked smartly, one after the other, the goals of JTBD, online surveys, and other qualitative research processes will be met.
