The Art of Questionnaire Design

Easy access to online survey platforms and a readily available sample - via customer databases or online panels - provide businesses with unparalleled opportunities to gather information about their consumers, without engaging market research professionals.  

But there are dangers in surveying without expertise.  Unless questionnaires are thoughtfully constructed, the right data collected and objectively assessed, it is quite easy to to get meaningless data… or, even worse, data which doesn’t necessarily reflect reality.   And this can severely impact on the decisions you take as a business.  

So, if you are eager to mine your user-base for useful insights and learnings without the help of an expert, here are six important principles to adhere to in creating your survey questionnaire.

Flow & Structure 
This may seem like a no brainer, but you would be surprised at some of the things we’ve seen.  The order matters, and it needs to make logical sense to the person who is filling it in.  Also, responses can be influenced by previous questions, so you need to keep that in mind when you are structuring the questionnaire.

• Start with a short but clear explanation about what the survey is about and why you want their help filling it in.
• Think of the structure as a funnel, starting broad and narrowing down as necessary.
• Keep questions grouped by sensible areas of interest or topics. Cue new subject areas, with written phrases to guide the respondents through the questionnaire. For example, "Now we would like to talk about your media usage…”  
• Be logical. Ensure the questions flow naturally from each other. 
• Keep sensitive questions – like demographics – for last.

Clarity
For results to be even vaguely useful, the questions have to be interpreted and answered as you intended, and in the same way by all the respondents.  Questions therefore need to be simple and clear, with no room for confusion about what could be meant.  This is harder to achieve than you might expect.    

• Avoid complex sentence structures – make it as simple and direct as possible.
• Define things clearly – eg: If you are asking about income, clarify whether this is household income or personal income. Before or after tax?
• Avoid vague terms that could be interpreted differently by respondents.
• Avoid using technical terms and jargon.
• Avoid asking about two things at once – no double barrelled questions.
• Be as objective and unbiased as possible!

Closed Questions
Closed questions - that is, providing a set range of options from which people make choices - is the most efficient and useful way to survey responses.  Open-ended questions have value, but all responses have to be sifted through and coded, which is time consuming and adds to the expense and length of the questionnaire.   However, when providing a range of options, you must make sure all the appropriate responses are accounted for, and that they are unique from each other.  Also,

• Wherever you have a range of choices that are not necessarily definitive, it is worthwhile giving an “other” option.        
• When asking “how often/how much/when” type questions, ensure options make logical sense, can be interpreted in the same way by respondents, and are consistent in the type of measurement used. Eg: don’t use “sometimes” and “monthly” in the same scale set.          

Randomisation
Randomisation is particularly important if you are showing respondents more than one concept or idea.  To avoid bias, ensure concepts are randomised or rotated, so that across the sample each concept has an equal chance of being seen first, and the overall order in which they are seen changes. 
You can and should also randomise the order in which respondents see attributes within a question, to minimise order effect.

Timing 
Now, there is no hard and fast rule for the timing of a questionnaire, but think about it from the perspective of the respondent (who, I promise you, is not as interested in sitting down and doing this questionnaire as you might imagine).  As short as possible is key.  The shorter the survey the more likely you are to get it completed, and the more likely it is to be thoughtfully answered. 

Pre-test!
Of course you need to keep testing it yourself, ironing out the kinks.  Even the professionals do this, so don’t skimp this stage.  But more importantly, you need some willing outsiders to test it to.  By this, I mean people who are similar to your target sample.  People who have the same level of understanding on the subject that your consumers do.  The questionnaire is the collector of the data – if it is not right, then your data is not going to be trustworthy.  So, test and test and test again. 

 Of course, this is just the tip of the iceberg.  We haven’t even mentioned the types of scales you can use, how to read the data or how to identify meaningful insights from it.  But it is a start.  If you are looking to make important decisions on the data you collect (and why are you doing the survey if you aren’t?), it really is worth engaging an expert to help you along the way.   

About us 
Bedrock Insight facilitates and nurtures consumer understanding.  We help clients combine consumer research with internal expertise to deliver rich, market-relevant insights. 

Want to know more? Here are some other resources:

• http://psr.iq.harvard.edu/files/psr/files/PSRQuestionnaireTipSheet_0.pdf?m=1357530492 
• http://www.dpcdsb.org/NR/rdonlyres/138975AC-B110-4D1E-902F-81C8E69BF9A0/107060/How_to_design_a_questionnaire.pdf
• http://www.mymarketresearchmethods.com/types-of-data-nominal-ordinal-interval-ratio/

Sharlene Zeederberg
www.bedrockinsight.com.au