BUZZ CHANNEL LIMITED
Buzz Channel have been involved in online research for a number of years. As well as designing our own surveys, we set up and run surveys on behalf of many other professional researchers.
This has placed us in an invaluable position to observe the different styles and approaches across literally thousands of studies and we have paid particular attention to the feedback we receive from respondents taking part in surveys.
This paper does not attempt to guide people on research practice. It is concerned with designing online surveys that respondents will find intuitive and enjoyable, with the aim that they are engaged and focused from beginning to end.
To those people who feel a positive respondent experience will not impact on the robustness of results, this paper is not for you.
Having said that we are not guiding you on research practice, we will start with one small suggestion:
Before you give thought to anything else, ask yourself…
"What decisions will we make as a result of this research? Precisely what questions do I need answered by this survey, to be confident making these decisions?"
Your answer to these two questions should drive your survey design, dictating what you really need to ask and keeping the survey as short and succinct as possible.
1. Adopt a conversational one-to-one style of communication.
This is perhaps one of the biggest differences between the online mode, and phone surveys run through call centers. In the latter case, surveys are very tightly scripted to ensure low operator error. However, online surveys are much more in the style of a conversation between two people.
The respondent has picked a time to take part that suits them (compared to being rung at home at 6pm as they just sit down). If you can engage them with your opening email and survey introduction, making them feel that you, as the survey designer, are personally interested in that individual respondent's view, you have made a great start.
You may even like to consider a short video instead of an introduction, explaining why the study is being run and why YOUR VIEW as the respondent is so important to ME, the survey sponsor.
2. Keep the survey short; Over 10 minutes may be too long.
We simply do not believe that people will be concentrating on your questions after ten minutes, UNLESS the survey is about something the respondent feels passionate about.
If you are surveying your customers, your staff, or other stakeholders, and you anticipate surveying them again in future, we really recommend that you work to this ten minute rule. This means next time you invite them to give feedback, they will trust that you'll not take too much of their precious time.
3. Respect the respondent, and explain what they are about to do, breaking the survey into sections.
Would you ask your friends to respond to your questions? If so, how would you preface your questions to prepare them? As you introduce your sets of questions to respondents, try to set an appropriate framework for them so that they are carried smoothly through your survey.
4. Ensure respondents receive questions that are relevant to them, or in consultations, that are of interest to them.
In terms of relevance, there is often no point asking people about products, services, or ideas that they are not aware of or would be unable to talk about. By using logic skips, you can ensure that the time respondents are in the survey (that you'll often be paying for) is used to your best advantage if they only get questions and answer options that they have said they are familiar with.
Similarly, in public consultations, Buzz Channel recommend showing respondents at the beginning the areas that you wish to cover (your whole survey may be well over ten minutes), but allow them to pick the areas they wish to read more about or to respond to. When you do have extensive information to share, make this optional so that they can select whether to read more before answering. If you have well-informed people taking part, they may drop out of your survey if confronted with copious information that they already know.
5. Do not make questions mandatory.
If you have followed our guidelines for a one-to-one, friendly style of writing and respect for respondents, people will answer your questions if they can. If they have nothing to say, and you make your questions mandatory, you'll get rubbish or you'll risk them dropping out altogether.
The only exception is where a question has logic applied to it to route them through various survey questions. In this instance, the respondent needs to give an answer so we know where to send them next. Our system will automatically validate (make mandatory) these questions.
6. Make full use of open-ended questions, (but don't make them mandatory).
Many researchers who have worked extensively in traditional modes of research will avoid open ended questions due to the historical difficulties with their recording and analysis. However, with online research, if you follow our other guidelines, you'll find respondents are very willing to contribute real gems when given the chance, especially if it is on a topic of interest to them.
If they don't have anything they want to say, they'll just skip ahead.
Thanks to the real-time reporting, and ability to use cross tabulations or filters to group your open comments, you often don't even need to have these responses coded. In fact, many of our end clients get enormous value from reading the verbatim comments.
6. Include one open ended question at the end so respondents can add additional comments on the topic or the survey itself.
You'll be amazed at what you learn!
And a couple of technical guidelines….
7. Keep batteries of rating questions short enough to show on one page without the respondent needing to scroll down the page.
We know from respondent feedback that large attribute rating questions are very off-putting. If you need to use these, try to keep them short. A good rule of thumb is that more than 7 questions on a rating table starts too look long and onerous to answer.
If they need to be long, break them onto several pages (and even apologize as they hit the second and subsequent pages).
Alternatively consider using visual question types such as our Drag n Drop as this has the same reporting structure as a Radio Button Matrix, but is much more engaging for respondents.
8. Be consistent with scales throughout the survey.
Your low numbers to the left of the scale should mean poor, bad, negative or unimpressed. Your high numbers to the right of the scale should mean high, good, positive, or impressed.
For a practical demonstration of these points, click here to view a mock up Buzz Channel online survey.
You may also be interested to read "Project Management Guidelines" or "A (non-technical) Guide to Sample Planning".