Write better questions to get better quality data

We often think that a survey is going to bring us a solution to all of our problems. That our customers opinion is going to increase our revenue like a magic stick. The truth is, out of many surveys, very few give us relevant information improving our business. We are often lost with the responses, not knowing what is the next step. In order to collect relevant answers, you need to ask relevant questions. Some questions can be very interesting, but not relevant for you. Here are 3 simple steps to create perfect surveys.

Define your goals and expectations

To write relevant questions you need to define your goals for each question. Why are you asking that question? What is this question going to highlight? You also need to establish your expectations.

For example:

Goal: You want to determine on what social media should you focus your marketing strategy.
Expectations: You expect that 60% of the responses will say Facebook, 20% Instagram and 20% Twitter.
Your question will be : What platform you are most reachable on? Facebook, Instagram, Twitter.

Writing down your expectations will help you make sure the answers you will collect are the ones that are the most relevant to you. Comparing your expectations and the actual responses will make your analysis easier.

You could have asked:

“Indicate the platform you go on: Facebook, Instagram, Twitter”
Respondent might have checked all options and it won’t have helped you decide on which social media should you focus your efforts.

Improve your questions

Watch the length!

A question should not be more than 20 words. If your question is too long, your chances of collecting complete answers are low. For example with multiple answers, the longer the list of propositions the more likely the answer is to be incomplete. Respondents get lazy or bored, to read until the end, above all if they already checked some proposition of the beginning of the list.

Prefer dichotomous questions.

Dichotomous questions are questions having two possible answers. For example, yes or no. These questions are easier and quicker to answer. They will drastically reduce the time needed for your survey. However they don’t let any other possible answer and your respondent might feel frustrated that you don’t ask him for his own opinion.

Avoid double barrelled questions.

“How would you describe the service and the food tonight?” Poor - Good - Very good
This question implies that you will rate the same way both the food and the service. You should write two different questions:
—> How would you describe the service?
—> How would you describe the food?

Avoid overlapping answers.

The choice of words is very important in a survey. Adapt your language to the way your audiences talks. Use your audience’s jargon, not yours. Avoid any technical concepts.
Do not hesitate to define terms of the question.

Choose your words

The choice of words is very important in a survey. Adapt your language to the way your audiences talks. Use your audience’s jargon, not yours. Avoid any technical concepts.
Do not hesitate to define terms of the question.

For example: “Do you prefer answering to dichotomous questions or multiple choice questions? Dichotomous question / MCQ”
If the respondent doesn’t know what is a dichotomous question he will be stuck and will have to either search in a dictionary, answer MCQ by default, or skip the question. This behaviour would result in poorer data quality.
You could easily transform the question in:
“Dichotomous questions is a type of questions offering only two possible answers.
Do you prefer answering to dichotomous questions (two possible answers only) or multiple choice questions? Dichotomous question / MCQ”

Some words have a powerful impact on the respondent's decision process. Try not to use them for your questions to be as neutral as possible.
For example, the word “prohibit” is very powerful.
“Should we prohibit dogs in parks? Yes/No” —> this question implies that dogs in parks are negative. Replace it with:
“Are you for or against dogs in parks? Yes/No”

Define everything that is subject to interpretation. Give intervals if needed. Try to insert equal sized intervals only. It will be easier for you to analyse.
“Do you wash your car regularly? Yes/No”
This question doesn’t explain what does regularly means. Is it once a week, once a month? You can make this question a MCQ with intervals:
“Do you wash your car regularly? Yes, at least once a week / Yes, at least once a month / Not regularly / I never wash my car.”

Avoid biased and leading questions

Avoiding biases in a survey is a difficult task. In general we want to show something with the survey so we tend to write questions the way we interpret them. It is important to be as objective and neutral as possible when writing the survey.
“How much did you like the play?” implies that you liked the play.
Even if the proposition “I did not like it” is available, the respondent will be more likely to answer that he did like it. Transform this question into “Did you like the play?” and give several ordinal options, from the best critique to “I did not like it at all”.

Be careful of social desirability.

Voting Obama or Trump has a completely different effect socially speaking. Some respondent may answer according to what is expected, rather what they actually think and do. This is called the Bradley effect.

Gather privacy details

Gathering private information is always tricky. Respondents don’t like giving their email address, their level of income, or even their full name through a survey or a feedback form because it is an impersonal support and they are afraid of the confidentiality level.
Some adjustments in your survey can help you collect more data.
1. Write a incentive and assurances of confidentiality. Refer to it in your introduction. Your respondent will feel more secure about sharing private data.

2. Order your questions so that your respondent will never be bewildered. Place the privacy related questions at the end. If everything has been going smoothly during the survey, your respondent will be more likely to share his details. If the respondent had some difficulties with the survey, the chances that he gives his details are very less.

3. Insert a “I prefer not to answer” option. It might sound curious, because you want your respondents to answer, but giving them the choice not to answer will better your data quality and will put the respondent into a comfort zone. He will be more likely to make his own choice and to answer the question. Also you will avoid the risk of respondent answering randomly, hence impacting your data quality.

Watch your order

The order in which you organise your questions is very important. Your survey will be easier if you never bewilder your respondent. As a consequence try to respect his expectations. It is commonly known that easier questions are coming first and more difficult questions then. If you start with difficult questions, the respondent will think that the level of difficulty of the question is going crescendo. The survey will appear to be difficult and the respondent will be more likely to give up and quit the survey.

It is advised to put the easier questions at the beginning, and then continue with more difficult questions, or questions people are less likely to want to answer to such as confidential questions: income level, phone number, etc.

If your survey is rather long, you structure it with the “funnel” order. It consists on putting the broad and general questions first, then the specific questions, and ending with the easy to answer questions such as demographic data related questions.

The respondent won’t be bewildered by the order, he will be happy to see that the difficult questions are coming rather quickly, and won’t have any problem answering few easy demographic questions at the end.

Also be careful about the impact of your order. For example if you ask a question about Hiroshima, and then you ask a question on the necessity to develop the nuclear industry - the answers will be biased by the impact of the first question. Try to always be as neutral as possible.

To conclude, 3 simple steps to create the perfect survey: defining your goals and expectations, improving your questions, and focusing on the order. You are now ready to create beautiful surveys and to collect many useful and relevant responses.