Tickbox

All articles

The difference between qualitative and quantitative survey questions

The difference between qualitative and quantitative survey questions is the type of answer that you get. A qualitative question lets people answer in their own words, so it shows why they think or act as they do. A quantitative question collects answers that you can count, such as a choice from a list or a rating, so it shows how many people think or act in a certain way. This guide gives 10 paired examples for marketing work and explains when to use each type. It also gives a step-by-step method to code open-text answers into themes and count them.

The difference at a glance

The same difference applies to research in general. SurveyMonkey describes qualitative research as a way to explore people’s experiences, opinions and motivations through information that is not numbers. It describes quantitative research as the collection of numerical data that can be measured and analyzed statistically. Source: SurveyMonkey: Qualitative vs. quantitative research.

AspectQualitative questionQuantitative question
It usually asksWhy? What? How?How many? How much? Which one?
Usual formatA text boxSingle choice, multi-select, rating scale, Likert scale or a number
The answerThe words of the respondentA number, or an option from a list that you wrote
Best forFinding reasons, and answers that you did not expectMeasuring size, comparing groups and tracking changes
How you analyze itRead the answers, code them into themes and count the themesCount the answers, then calculate shares and averages
Main limitIt takes more time to analyze, and some people do not answerIt can show only the options that you listed, and it does not show why

SurveyMonkey lists these limits, among others: text-based data can be difficult to analyze, and quantitative data does not capture the ‘why’. Source: SurveyMonkey: Qualitative vs. quantitative research.

Is qualitative the same as open-ended?

Not exactly. ‘Open-ended’ describes the format of the question, and ‘qualitative’ describes the data that you get. The two usually go together, but not always. A text box that asks ‘How many people work at your company?’ is open-ended, but the answers are numbers, so the data is quantitative. For the format decision and the other question types, read 10 types of survey questions, with examples.

10 paired examples for marketing work

Each pair asks about the same marketing topic in two ways. These are example questions that we wrote. They are not results from a survey. Change the words to match your product and your audience.

1. Brand awareness

Qualitative: When you think of email marketing tools, which brands come to mind?

Quantitative: Which of these email marketing tools have you heard of? Select all that apply.

Why use the pair: Ask the open-ended question first, because a list can remind people of brands. The difference between the two results shows which brands people remember without help.

2. Message testing

Qualitative: In your own words, what does this headline promise?

Quantitative: How clear is this headline? Not at all clear / Slightly clear / Moderately clear / Very clear / Extremely clear

Why use the pair: The rating shows which headline people think is clear. The words show if they understood what you intended.

3. Pricing page

Qualitative: What, if anything, is confusing about our pricing page?

Quantitative: How easy was it to find the price for your team? 1 (Very difficult) to 5 (Very easy)

Why use the pair: Track the rating when you change the page. Use the words to decide what to change.

4. Lead source

Qualitative: How did you first hear about us?

Quantitative: Where did you first hear about us? Search engine / Social media / A colleague or friend / A podcast / Other

Why use the pair: Use the open-ended answers to write the list. Then use the list, because answers from a list are faster to count each month.

5. Reason to buy

Qualitative: Why did you choose us and not another option?

Quantitative: Which one of these mattered most in your decision? Price / Functions / Ease of use / A recommendation / Other

Why use the pair: The count shows which reason is the most common. The words give you language for your landing pages.

6. Cancellation

Qualitative: What is the main reason that you canceled?

Quantitative: Which one of these is the main reason that you canceled? Too expensive / Did not use it enough / Missing a function / Moved to another tool / Other

Why use the pair: The list gives you a number to track each month. The open-ended answers show what ‘too expensive’ means, for example the price itself or too little use for that price.

7. Landing page

Qualitative: What information did you look for on this page and not find?

Quantitative: Did you find the information that you came for? Yes / Partly / No

Why use the pair: The count shows the size of the problem. The words show which content is missing.

8. Content plan

Qualitative: What topic do you want us to write about next?

Quantitative: Which two of these topics do you want to read about next? Pricing / Case studies / How-to guides / Industry news / Product updates

Why use the pair: The open-ended question can show new ideas. The list makes people choose between the ideas that you already have.

9. Webinar feedback

Qualitative: What was the most useful part of this webinar?

Quantitative: How useful was this webinar? 1 (Not at all useful) to 5 (Extremely useful)

Why use the pair: Use the rating to compare webinars. Use the words to plan the next webinar and to write its description.

10. Recommendation

Qualitative: What is the main reason for your score?

Quantitative: How likely are you to recommend us to a friend or colleague? 0 (Not at all likely) to 10 (Extremely likely)

Why use the pair: This is the Net Promoter Score question with a follow-up. Get the score with our NPS calculator, and then code the reasons separately for each score group.

When to use each type

Use a qualitative question when

Use a quantitative question when

Three ways to use both

Two facts to consider before you choose

An answer list changes the results. In a Pew Research Center poll after the 2008 US presidential election, respondents were asked which one issue mattered most to their vote. When the closed-ended version listed the economy, 58% chose it. In the open-ended version, 35% named the economy, and 43% gave an answer that the closed-ended list did not include. Source: Pew Research Center: Writing Survey Questions. If your list is not correct, your count can look exact and still be misleading, so test a new list with an open-ended question first.

Open-ended questions can have high rates of nonresponse. A Pew Research Center study examined 30 open-ended questions on a panel of US adults. The share of respondents who gave no answer ranged from 4% to 25%, with a median of 13%. The median was 12% for questions that asked for one word or a short sentence, and 17% for questions that asked for multiple sentences. It was 15% on a tablet or cellphone, and 11% on a desktop or laptop computer. Women, younger adults, Hispanic and Black adults, and people with less formal education were less likely to answer. Source: Pew Research Center: Nonresponse rates on open-ended survey questions vary by demographic group, other factors (2023). So ask for a short answer, and do not assume that the people who answered represent all respondents.

How to code open-text answers into themes and count them

Coding means that you read each answer, tag it with one or more themes from a fixed list, and then count the tags. A word cloud can show which words appear often. Source: SurveyMonkey: Qualitative vs. quantitative research. But it counts words and not meanings: ‘price’, ‘cost’ and ‘expensive’ are three separate words for one theme.

You need a spreadsheet and, if possible, a second person. The steps below are our own short version for marketing teams. They use practices from a coding study by Pew Research Center and from the guidelines by O’Connor and Joffe. The advice that has no source link is our own.

  1. Set the decision and the unit. Write the decision that the answers must support. Then decide what one coded item is: the full answer, or each separate idea in the answer. O’Connor and Joffe say to make this decision before coding starts. Source: O’Connor and Joffe: Intercoder Reliability in Qualitative Research: Debates and Practical Guidelines, International Journal of Qualitative Methods (2020). For short survey answers, we suggest that you code the full answer and let it have more than one code.
  2. Prepare the data. Export the answers to a spreadsheet, with one answer on each row and an ID for each row. Keep the original text. Remove names and other personal data that you do not need. Count blank answers and answers with no content, such as ‘N/A’, separately, and do not include them in your base.
  3. Read a random sample and draft the codes. Read a random sample of the answers and write down the themes that you see. Pew Research Center researchers also started their codebook with random samples of responses. Source: Pew Research Center: Appendix B: Coding methodology (2026). If you have fewer than 100 answers, read all of them. We suggest that you start with 5 to 10 codes.
  4. Write the codebook. For each code, write a name, a definition and one example answer. O’Connor and Joffe describe a coding frame as a list of codes with definitions and example data segments. Source: O’Connor and Joffe: Intercoder Reliability in Qualitative Research: Debates and Practical Guidelines, International Journal of Qualitative Methods (2020). Add a code named ‘Other’ for clear answers that do not fit.
  5. Test the codebook with a second person. Give the codebook to a second person. Each of you codes the same random sample alone. O’Connor and Joffe say that 10–25% of the data is typical, depending on the size of the data set. Source: O’Connor and Joffe: Intercoder Reliability in Qualitative Research: Debates and Practical Guidelines, International Journal of Qualitative Methods (2020). Then compare the results, discuss each disagreement and make the definitions clearer. In one 2026 study, Pew Research Center researchers reviewed disagreements and refined the codebook after each round of testing. They report a Krippendorff’s alpha, which is a statistic for agreement, of at least 0.70 for each of the 30 codes. Source: Pew Research Center: Appendix B: Coding methodology (2026). There is no universally accepted level of agreement. O’Connor and Joffe report one suggested standard of 80% agreement on 95% of codes. Source: O’Connor and Joffe: Intercoder Reliability in Qualitative Research: Debates and Practical Guidelines, International Journal of Qualitative Methods (2020). For a low-risk marketing decision, we think that it is enough to discuss each disagreement and record your agreement. If you work alone, code the sample again a few days later and compare. This is not as strong a test as a second person.
  6. Code all the answers. Add one column for each code. Put 1 in a column if the code applies to the answer. Keep the codebook open while you work. If more than about 10% of the answers go to ‘Other’, read them. Add a code if you see a new theme, and then check the rows that you already coded.
  7. Count and report. Add up each code column. Divide each total by the number of answers in your base, and show that number. One answer can have more than one code, so the shares can total more than 100%. Show the count, the share and one short quote for each theme. If you have few answers, show counts and say that the shares are not estimates for all customers.

Worked example, with invented answers

Everything in this example is invented: the company, the answers and the counts. It shows the method, and it is not survey data.

An invented software company asks new customers this question: ‘What almost stopped you from signing up?’ The question goes to 15 people. 2 people leave it blank and 1 person writes ‘N/A’, so the base is 12 answers.

The codebook (steps 3 and 4)

The team reads the answers and writes 6 codes.

CodeDefinitionExample answer
PriceThe amount that the product costs, or how easy it is to find that amount.A1
Setup effortThe work that the person expects, in order to set up the product or move data to it.A3
TrustDoubts about the company or its proof, including questions about data security.A4
Missing functionA function or a connection that the person needs and thinks that the product does not have.A5
No barrierThe person says that nothing almost stopped them.A9
OtherA clear answer that does not fit another code.A12

The test (step 5)

Two people code all 12 answers alone, because 12 answers is too small for a sample. They agree on 11 of the 12 answers (92%). They disagree on A7: one person chooses Other and the other chooses Trust. They decide that A7 is Trust, and they add ‘including questions about data security’ to that definition. The codebook above shows the final definition.

The coded answers (step 6)

IDInvented answerCodes
A1The yearly price is hard to justify for a team of three.Price
A2I could not find the price without booking a demo.Price
A3I was worried that moving our contact list would take weeks.Setup effort
A4I had never heard of your company, and I found few reviews.Trust
A5It has no connection to our CRM.Missing function
A6It was too expensive, and the setup looked long.Price / Setup effort
A7My manager asked where our customer data is stored.Trust
A8Moving from our old tool looked like a lot of work.Setup effort
A9Nothing. It was an easy decision.No barrier
A10The monthly cost goes up a lot after the trial.Price
A11I needed a connection to our CRM, and I wanted more customer stories.Missing function / Trust
A12The sign-up email went to my spam folder.Other

The count (step 7)

CodeCountShare of 12 answers
Price433%
Setup effort325%
Trust325%
Missing function217%
No barrier18%
Other18%

The shares total more than 100%, because A6 and A11 each have two codes.

What the team can say

Price is the most common theme, with 4 of 12 answers. But read the answers in a theme before you act: A2 is about finding the price, and A1, A6 and A10 are about the amount. So there are two possible actions: show the price more clearly, or change it. With only 12 answers, the shares do not estimate the view of all customers.

The next step is to make the themes into a closed-ended question, and then ask more people. This is the same pilot process that Pew Research Center describes: ‘What almost stopped you from signing up? Select all that apply. Price / Setup effort / Trust / A missing function / Nothing / Other’. Source for the pilot process: Pew Research Center: Writing Survey Questions. To plan the number of responses, use our sample size calculator.

Open-text answers in several languages

If your form runs in several languages, decide how to code the answers before you translate them. For a survey with open-ended responses in 12 languages, Pew Research Center compared two methods. Professional translators coded answers in the original language, and in-house coders coded answers translated by Google Translate. Across all the non-English answers, the agreement scores were similar: a Krippendorff’s alpha of 0.70 for the translators and 0.67 for Google Translate. But for Greek and Korean, the Google Translate coding was markedly worse. Source: Pew Research Center: The pros and cons of using professional translators vs. Google Translate to analyze open-ended survey responses (2021).

  1. Use one codebook in all languages, so that you can add the counts together.
  2. Keep the original text next to each translation.
  3. Check each language separately. When translators coded the answers, Pew Research Center got full translations and codes for a random 10% of the non-English answers from each pair of country and language, and it used them to check the work. Source: Pew Research Center: The pros and cons of using professional translators vs. Google Translate to analyze open-ended survey responses (2021). A smaller team can ask a speaker of each language to code a random sample.

Common questions

Is a Likert scale qualitative or quantitative?

It is quantitative, because you count closed-ended answers. For scale labels and how to read the results, read Likert scale: definition, examples and how to read the results.

Can one survey use both types?

Yes. Research that combines qualitative and quantitative questions is called mixed methods research. Source: SurveyMonkey: Qualitative vs. quantitative research.

How many open-ended questions should I ask?

Our sources give no fixed number. We suggest one or two, and each one must support a decision that you have to make. Longer open-ended prompts have higher nonresponse. Source: Pew Research Center: Nonresponse rates on open-ended survey questions vary by demographic group, other factors (2023).

How many answers do I need to code?

To find themes, our advice is to code until new answers stop adding new codes. To estimate a share of a larger group, you need a representative sample, so read the method in our sample size calculator.

Can AI code the answers?

Software can suggest codes, but test the result in the same way as you test a second person. Have a person code a random sample, and compare the two sets of codes. Do this for each language. To make a first draft of the themes, you can use our free open-ended survey response analyzer. It groups similar answers in your browser, and a person must then read and correct the themes.

Where Tickbox fits

The method above is manual, and the work grows with each answer. Tickbox is a form builder with voice and video. Forms receive text, audio and video responses, and AI analyzes the responses so they are easy to read. AI analysis is on all plans, including Free. Forms support multiple languages. You can use this guide with any form tool, or start from a free form template.

Read what Tickbox is

Sources for the facts in this article

Each external fact above links to its source, and the list below shows what each source supports. We checked these sources on 4 November 2026. For O’Connor and Joffe, we used the copy at University College London. The definitions in the comparison table, the example questions and the advice with no source link are our own. The answers and the counts in the worked example are invented, and they are not survey data.