Making a Good Survey

Just by avoiding some common mistakes you can increase the effectiveness of an online survey.

Use the Right Answer Format

So often the difference between an effective an ineffective online survey will be down to using the correct answer format.

Making a Question Open Instead of Closed

A quick way of losing all the major benefits of online surveys is by making a question that could be closed open. (A closed question is question where the respondent must choose from a selection of answers, an open question is where the respondent is able to enter any response they like).

These can be questions like 'How old are you?' or 'How much would you pay for the described service?'.

Let's not beat around the bush and be polite here, making either of the two example questions open is just plain wrong and will lead to disaster. Given a free range it is very likely that respondents will respond to the first question by entering a response along the lines of:

• 21
• Twenty-three
• Thirty
• 30.5
• Nearly 40
• 28 in April
• 23
• Too old
• Not telling
• Twenty three

Collecting such data is pointless, a mixture of text and numbers, with a sprinkling of non-answers would, if this were a large survey, make an analysis using the respondent's age impossible.

Even the three twenty-threes in the above example would be treated differently and the collated information could not be represented as a meaningful chart.

Limit 'open' responses to places where they are necessary, such as for when asking for a respondent's comments.

Two Questions in One

A question like 'Did you find the service both efficient and friendly?', does not appreciate that a respondent would be correct to answer 'No' if they found the service was 'friendly' but not 'efficient' or 'efficient' but not 'friendly'.

Typically respondents who may have found the service 'friendly' but not 'efficient' may be tempted to answer 'Yes', masking a potential problem with 'efficiency' and vice versa.

Ask only one question at a time, even if you think two things are synonymous with each other.

Allowing Multiple Selections when Only One Answer is Applicable

If a respondent is asked their gender they can only give once answer, even in instances where 'transgender' was going to be allowed as an answer option it should be added as an answer option as it would still be fundamentally wrong to allow the respondent to pick both 'male' and 'female'.

If you allow multiple selections to be made, you can guarantee that by either mistake, or deliberate intent, some respondents will select more than one answer option. A radio formatted response forces the respondent to only make one selection.

Allowing Only a Single Selection when Multiple Answers may be Applicable

A question such as 'How did you travel', where the answer options are limited to only one selection such as 'Air', 'Sea', 'Road' or 'Rail' does not cater for those people who travelled more than once using a different form of transport. Using a single selection radio button will give inaccurate results.

Not Allowing for the Exception

Always anticipate that in a large survey sample there is likely to be the exception. A question such as 'Select the country where you currently reside' may include a comprehensive list, but there is always the possibility that if a respondent's country is minor or newly formed, it may not be listed. Instead of making the question open, which would compromise the majority of the collated data, include the 'Other' option so that exceptions can be handled correctly.

If the 'Other' responses indicate a major omission from the list of countries remember to include the country in future surveys.

Understanding the Question

Questions should be phrased in a way that everyone will have the same understanding and it is wrong to assume that everyone has the same understanding and that they have the same knowledge. Consider the following instances where misunderstanding the question is likely to be rife.

Avoid the Double and Triple Negative Question

A question such as, 'Didn't you not like the lesson?' will produce unreliable results. The person asking the question is very likely to know what they mean, but it is pot luck whether those answering the question will interpret the question in the same way.

Use Abbreviations with Care

In some cases, abbreviations can be used without the fear that they could be misunderstood by the target audience. For example, in a survey on a medical topic aimed at medical personnel the question 'Would you consider administering any AED to a person under 15 who was also diagnosed with COPD?', may be perfectly acceptable.

However, if a survey was aimed at the general public, where each respondent may be technical or non-technical, asking a question like 'Who is your ISP?' may completely flummox the non-technical respondent. In such cases it is better to explain the abbreviation either by rewriting the question to 'Who is your Internet Service Provider (ISP)?' or by making additional information available to those who may not know what the abbreviation, ISP, stands for.

Avoid Subjective Responses

The question 'Do you regularly read a book?' seems at first to be fairly straight forward, but 'regular' is subjective. Some respondents may read a book may read a book every day, others twice a week, others a couple of times a month. The survey's publisher needs to define clearly to the respondent what they consider to be 'regular'.

Consider Timescales

A question like 'Do you go to the movies?' does not indicate as to the timescale that the survey publisher wants the respondent to relate to. Do they want to know if the respondent has been to the movie ever, in the past five years, the past year or the past month? Do they want to know if the respondents go regularly, or have they been once?

It is up to the publisher of the survey to make clear how they want the respondent to answer; if the timescale is ambiguous then each respondent will apply their own interpretation resulting in unreliable information being collated.

For more information or to discuss how online surveys can help you please contact surveys@surveygalaxy.com or visit www.surveygalaxy.com the quick, easy and cost effective way to do online surveys.