New Rules for Online Surveys and Questionnaires

Online survey websites bring a new dimension to designing and deploying surveys when compared to the more traditional printed survey.

When trying an online survey service for the first time it is common to want to design surveys in the same way as you would for a printed survey. Although this is not a problem there are times where the unique features and benefits of the online survey often means that the traditional methods are not always best practice.

With printed questionnaires you are very much in the lap of the God's as to whether those that participate in the survey complete the survey correctly. Questions that ask 'please tick only one box' will often results in multiple ticks and it is therefore safe to reason that a 'tick all that apply' type question is likely to suffer the same logic in reverse.

How do you ensure questions are not missed by participant and how do you differentiate between those that were missed and those that were deliberately not answered? With online surveys you have the ability to address these questions.

This article looks at the 'online' advantage over the printed questionnaire and will also discuss a few general guidelines that will help you write effective surveys and questionnaires regardless of how they are to be deployed.

Branching

In a questionnaire 'branching' is the term used when the respondent is asked a question and based on their answer they are then asked a follow up question or asked to skip to later question.

Although a requirement in some situations branching in many cases can be eliminated altogether and the questionnaire streamlined and made more 'respondent friendly'.

Take the following example:-

The above is an extreme example of branching. The respondents are having to think, and even where the survey software supports an automatic branching facility it introduces a needless level of complexity.

The following demonstrates how this question could be just as easily re-phrased.

This second question/answer format covers all possible permutations, simplifies the survey and ensures that all respondents follow the same clear path.

Scale

Possibly to help simplify the design of printed survey a scale was often used as a way to rate a particular response.

For example:-

With online surveys in particular you can eliminate the need to have a scale as the composing tool will automatically format the headings for you.

This second approach is more intuitive for the respondents as they don't have to mentally translate their opinion into a scale and it also allows for the introduction of an important additional category 'No comment/Didn't attend'.

Subjective Responses

Care should be used when using units of measure that are subjective. Take the following example.

Although all respondents should have no problem answering the above question it will be unclear when analysing the results as to what were the individual definitions of 'Not very often', 'Sometimes' and 'Often'. Some respondents may have interpreted 'Sometimes' as meaning once a year, others once a month.

When using units of measurement that can be subjective it is better to qualify your own interpretation to ensure that all participants respond using the same scale.

Making Questions Mandatory or Voluntary

Unlike a printed questionnaire with an online survey you have the option to make questions mandatory, i.e. where the respondent must respond to a particular question before moving on to the next question. When analysing the results it is always better to have had a positive response than to have no response.

It is often a common assumption that to make a questionnaire more 'respondent friendly' then it is good practice to make all questions voluntary. However, providing the respondent with a positive 'No comment' option will show the participant that you appreciate that they may not wish to answer a particular question but will also ensure that the survey results can be properly analysed.

When a response to a question is made voluntary an unanswered question is more difficult to analyse.

The following is an example that might be taken from a delegate feedback questionnaire with the question being made voluntary.

With the above format there is no way of knowing from unanswered questions if the respondent didn't answer intentionally, if they accidentally missed the question or if they dropped out of the survey completely.

A better method would be to make the question mandatory and to add additional options:-

This will ensure that all respondents are canvassed and that incomplete surveys can be identified as being incomplete and not confused with those respondents who did not wish to comment or who did not attend the film.

When analysing the success of a survey it is useful to be able to see if respondents failed to complete a survey and where they dropped out; was it prompted by a particular question or the length of the survey? Such information can be used to improve the design of future surveys.

There are some question/response formats that should not be made mandatory. For example:-

As a rule the above type of question should not be mandatory as a 'no response' would in this context be a valid answer.

The following example of a check box should not be mandatory as a 'no response' would again be a valid response:-

However, there are many advantages in modifying it slightly so that it can be made mandatory.

Free Text

Questions that require a free text response, that is where the respondent is not asked to choose from a list of possible replies but able to write any response they like, should be used sparingly.

There are some questions where free text is essential for example requesting contact numbers and email address.

However, take the following example:-

This type of question is likely to get responses that are in units of years, months, days and weeks, as well as miscellaneous comments such as 'A few years', 'Not long'.

If this information is important for analysis it would make that analysis very difficult, the larger the volume of respondents the more difficult the analysis.

The above question would be better as radio buttons:-

Here in designing the questionnaire a decision has been made as to what criteria is to be used when analysing the data and the format has now made this analysis automatic by grouping the respondents into those chosen categories.

Designing the Questionnaire

When designing an online questionnaire you do not need to start at the beginning. For example if your survey contains standard demographic questions at the start of the survey such as name, age and place of residence you can leave these until you have finalised the body of the survey and then add them at the last minute.

This allows you to focus on the main purpose of the survey and keeps the survey lean so that manipulating the survey, changing the order of the questions, modifying the text is both easier and quicker.

Using Survey Galaxy's template library you can store common series of questions and available responses and add them in the final stages of designing your survey.

Keep in mind that many of the restraints imposed when designing printed questionnaires such as page length, page width and numbering are not present, or are fully automated, with online surveys.

For more information or to discuss how online surveys can help you please contact surveys@surveygalaxy.com or visit www.surveygalaxy.com the quick, easy and cost effective way to do online surveys.