How to create a survey

If you want to gather new information, a wide variety of views or to learn about something deep or complex you would be better off using qualitative research methods, like a focus group or an interview.

To create an effective survey, it should be easy to complete and allow respondents to provide thoughtful answers. There are two choices about what kind of survey you could use to gather the information you need – online surveys and paper-based surveys.

Online Survey or Paper-Based?

Traditionally surveys paper-based and it’s interesting to note that research suggests that paper surveys get more responses than online surveys – an in-person request from the researcher can influence people to complete survey which is worth remembering if you are surveying clients. Distributing the survey through your staff and ensuring confidential locations where the surveys can be left, might be the best option for some organisations, particularly where internet access may be limited, or digital literacy skills aren’t strong.

However, there is no doubting the efficiency and usefulness of online survey software. It not only helps you construct your survey but will also collate the responses and help you analyse the results.

There are several ways people can respond to an online survey. One is to embed the survey in your website for completing, another is to send an invitation by email that includes a link to the survey, the third is to send them an online survey directly.

Purpose of survey

No matter which kind of survey you chose to do, the reason why you’re surveying is important to clarify. Write down what you want to know (eg, ‘is our service meeting people’s needs?’). Next write down what you need to find out in order to answer that first, most important question. These topics should break the main question down into parts like this:

What we want to know: is our service meeting people’s needs?

Breaking it down:

• Are clients satisfied with the way we conduct assessments?
• Does the way we do case coordination, support and referral work for them?
• Do they think we are helping them achieve the outcomes they want?
• Are there services clients want that we aren’t providing?

Make sure the subsequent decisions about who you survey and how your survey is structured relate closely to these basic ideas.

Choosing who to survey

Once you know your survey purpose, then the questions you need to ask are your survey objectives.

Survey purpose: to learn whether our service is meeting clients’ needs.

Survey objectives: to learn about client satisfaction and collect feedback on the following:

1. Assessment processes
2. Case coordination
3. Client outcomes
4. Gaps in service provision

Make a list of the questions you need to ask to find out what you want to know. Use your objectives as categories to organise these questions. Your example might now look something like this:

An example

Survey purpose: to learn whether our service is meeting clients’ needs.

Survey objectives: to learn about client satisfaction and collect feedback on the following:

1. Assessment processes
   1. Do clients understand why they’re being asked the assessment questions?
   2. How did the clients feel about the assessment process?
2. Case coordination
   1. Are clients satisfied with the quality and responsiveness of the case coordination model?
   2. Do clients have any feedback for service improvement?
3. Client outcomes
   1. Are clients making progress towards, or achieving their goals?
4. Gaps in service provision
5. What needs do clients report are not being addressed?

We hope this example is close to something you know well – that its close to how many community service organisations work. But what if you want to run a survey to find out new information about a topic that you don’t know much about?

In that situation you need to do some preliminary research so that you can design your survey. The best way to do that is to do a short survey of a group of relevant people using some open-ended questions (for more on these see section 4 below). The answers you get to this preliminary survey will tell you what you should explore in your survey proper.

There are a couple of things you need to avoid when you are thinking up your questions.

Don’t ask questions that people can’t answer accurately

To avoid this mistake do the following:

• Ask screening questions to make sure your survey is being done by the right people
• Use words people understand
• Keep your survey short
• Give people the chance to choose ‘don’t know’ or ‘neutral’ or ‘not applicable’.
• Don’t ask questions people don’t want to answer because they are embarrassing

Optimise your survey for the devices it is going to be viewed on

Surveys that look good on computers can cause all kinds of problems on mobile devices. Even people with desk top computers might open your email on their phone and look at your survey there − and possibly delete it if it isn’t easy to read.

Some online survey providers (eg SurveyMonkey) automatically adjust the format of the survey for the size of the screen of whatever device they are viewed on but it’s still a good idea to test them and see.

Here are a few things you can do to make sure your survey works on a mobile device:

• Don’t ask many open-ended questions. Normally a mix of closed and open-ended questions is recommended in a survey, but not if the survey is being viewed on a mobile device; it’s too hard to type the answer, plus, it’s likely a person using a mobile device is on the move somewhere.
• Don’t put links to images in your survey. It slows the download.
• Pretest your survey on a mobile device. There are lots of design differences, like different light settings on screens that make colours look different. You might need to modify your design so that it can be seen well on a mobile device.

Some online survey providers (eg SurveyMonkey) automatically adjust the format of the survey for the size of the screen of whatever device they are viewed on but it’s still a good idea to test them and see.

Keep the subject line in your email short! You want to inform readers and attract participation. 

You might decide that you don’t want to alter your survey in these ways too much. For example, you might think that it is very important to include the open-ended questions. The trade-off of that decision is that you might get fewer responses from people who use mobile devices heavily. That decision might be fine if it is staff in other community organisations that you want to survey. But if, for example, you want to hear from young people it’s worth adjusting your survey for mobiles.

Check and double check

Get a variety of people to test your survey to make sure it makes sense and that it works.

Your paper survey may say ‘if yes, skip to question 12’ so make sure they are correct.

Online surveys have features which help people skip from question to question, missing the questions that aren’t relevant to them. Make sure you double check that any commands like these that you have placed in your survey are working.

When you place the link to the online survey in your email invitation check and double check that it works.

Get feedback on every aspect of your survey, from the message in the email subject line to the messages on the opening and closing pages.

Get as many responses as possible to your survey

In the social sciences getting 50% of your surveys filled in and returned is the general goal; more than that and you are a survey maestro. However, you will be very lucky to get that many surveys back. Online surveys have lower response rates than paper surveys, and telephone surveys have the lowest response rates of all.

This means you have to do your best to get the most responses that you can. Here are some ways to do that.

If you are posting your paper-based survey in the mail

Make your letter personal by using people’s name and ensure it is well-written. Keep it short but make sure it has this information:

• Who you are and the purpose for the survey
• The survey’s benefit to the person you are asking to complete it
• The length of the survey
• What confidentiality you are ensuring

If the survey is short emphasise that but be honest about how much time it will take.

Research has also found that small things, like using a pretty stamp and putting the letter into the envelope with the recipients’ name and address facing the back flap (so that people see their names as they pull it out) can make a difference to their willingness to fill a survey in.

Four or five days after you have sent your invitation letter send the survey with a second invitation to people to complete i. 

After a week send a reminder message with another copy of the survey.

After a further week send a final reminder that looks quite different – a postcard or a card instead of a letter perhaps.

Make the effort to remove the names of anyone you know has completed the survey from your mailing list for reminders.

If you are doing an online survey

Make your email invitation personal – use people’s names. There is software that helps you to personalise correspondence (eg MailChimp).

Make the most of your email subject line. Research has found that the following words increase the chance that people will open your email: words that imply time sensitivity (urgent, breaking, important, alert), words that sound intriguing (announcement, invitation) and words that imply gratitude (thank you). Also See Mailchimp – How to write catch Email Subject Lines

If your subject line has intrigued people enough to open the email, you should have a well-written invitation. Again, keep it short but make sure it has the following:

• Who you are and the purpose for the survey
• The survey’s benefit to the person you are asking to complete it
• The length of the survey
• What confidentiality you are ensuring

If the survey is short emphasise that but be honest about how much time it will take.

Then, send reminder emails at intervals. Don’t send more than two. If you can, filter out the people who have already completed the survey.

Overall – give people enough time

It’s recommended that you allow at least two weeks for people to fill in a survey, but be flexible with your timeframes.

You can consider offering incentives – gifts or prizes. This needs to be thought through. See Information on Prizes, Reimbursements and Incentives. Incentives don’t have to be large – you can even make them so that people choose whether they want them or not and they will still increase response rates.

And finally, don’t put people off! Poor design can make people unwilling to respond to your survey; this is called non-response bias. This could happen if you ask about information that is very sensitive, or because your email link didn’t work. If your survey doesn’t work well on smartphones you might not get much of a response from young people who you might need to hear from.

Make sure the people who respond actually finish your survey

The percentage of survey starters who make it to the finish is called the survey completion rate. Getting people to complete the survey is very important because some of your most important questions will be towards the end. But it’s quite hard to achieve; a lot of people start surveys and give up. Here are some tips to help your completion rate be as high as possible:

• Keep your survey as short as possible
• Keep the purpose of the survey in your mind while you are writing your questions
• Make your questions flow logically so people are clear about what you’re asking them to do and don’t get frustrated.
• Show a ‘progress bar’. This is a feature on automated online surveys which visualises how much of the survey a person has completed, and how far they have to go.

How many people should we send the survey to?

Obviously the more people you can survey the more accurate your information will be. If you think that the people you are surveying are going to have very different views then you’ll need to survey more to capture that variety.

If you are doing a survey of a broader group of the community and you want to be able to work out whether what you were concluding is true of all Tasmanians you have to consider your ‘sample size’ but unless your organisation has a funded research team it is unlikely you will do research on this scale. Just try to give everyone who falls into the category of people that you want to survey an equal chance to complete the survey and be clear about the size of the group you are talking about when you report the results.

Be accountable to those who completed your survey 

Thank those who took the time to complete your survey and ensure any incentives are delivered as promised.

Share the result

Be sure to share the results, particularly with the people who participated, so they know their efforts to participate were worthwhile.

You can also publish your results them online and let people know through your newsletter or email distribution list.

Evaluate your survey

Did your survey provide the information you were seeking? What worked well? What feedback did you receive? Any learnings for next time? Be sure to take the time to consider and note down feedback from those involved in the survey process, so your next survey will be even better.

More Resources

Microsoft Forms – How to create a form

Microsoft Forms – Survey templates

Survey Monkey