Skip to main content
It looks like you're using Internet Explorer 11 or older. This website works best with modern browsers such as the latest versions of Chrome, Firefox, Safari, and Edge. If you continue with this browser, you may see unexpected results.

Survey Design

This guide provides information to assist in creating effective surveys.

Tips for Creating a Better Survey

The Basics of Survey Creation

  • Voluntary Participation is a requirement for all surveys.
  • State exactly how long the survey is but...
  • Keep it short, otherwise you risk *satisficing behaviours skewing your results.
  • It should look official or important so that respondents take it seriously.
  • Be sure it is user friendly. This will improve data quality.
  • Number the questions so respondents can gage their progress.
  • DO NOT overuse text! This increases the cognitive load and respondents will lose motivation, corrupting your data.
*Satisficing - giving an answer that satisfies without necessarily being honest or accurate. 

Question Types

Question Types Example
Simple Questions Yes/No
Running Prompt "How many times have you visited the library? 1 = rarely, 5 = every day
Closed question Choosing an answer along a scale: Mark all = multiple response closed question
Filter question Receiving a different next question depending on your answer
Open ended question "Please tell us how we can improve your experience"

 

The Agree/Disagree Scale

Pros Cons
Easy to administer Respondents may adopt *acquiescing strategies
Fewer "don't know" answers Respondents may adopt the satisficing strategy
Respondents prefer these In some cases the answer is too ambiguous

 

*Acquiescing - giving an answer the respondent believes to appropriate rather than factual

 

Avoiding Bias

It is particularly easy to accidently introduce bias into these types of questions, for this reason, some social scientists recommend NEVER using agree/disagree scales. See Krosnick, JA. Survey Research. Annual Review of Psychology Vol. 50:537-567 (Volume publication date February 1999)  https://doi.org/10.1146/annurev.psych.50.1.537

Example
Avoid Use
Computer hubs are an essential element of an academic library: strongly disagree - - - strongly agree Some people believe computer hubs are an essential part of the library experience but others do not believe they are necessary. Which is closest to your opinion? strongly disagree - - -strongly agree

 

 

The above graphic represents the process survey respondents go through when answering each individual question on a survey. They often adopt satisficing or aquiescing behaviours when any one of the above steps presents a cognitive challenge. Below are some helpful tips to prevent these types of behaviours.

Satisificing

Satisficing = giving an answer that "satisfies" without regard to accuracy
Problem: One of the most common reasons for this behaviour occurs in step 1 of the process when a respondent encounters comprehension problems.

Lexical Solution: Use vocabulary choices that are accessible and have a commonly understood definition.

Semantic Solution: Provide definitions or offer them verbally

Pragmatic Solution: try to block respondents' unintended inferences

Example: The word "counselor" without a qualifier, may immediately bring to mind a psychologist for one person and an attorney for another. Be sure to qualify this type of word.

Acquiescing & Sensitive Questions

Acquiescing = giving an answer the respondent believes to be appropriate rather than factual

Sensitive questions = those topics with a social stigma, illegal behaviour, and private information

Problem: This behaviour can occur with comprehension problems but is especially common with sensitive-type questions.

Solutions: For sensitive questions, do the following:

  • do not be present during the answering of sensitive questions
  • make the question easy (use familiar words)
  • embed the question among non-sensitive one
  • use ONLY ONE sensitive question per survey
  • consider using an open-ended format
  • assure respondent of confidentiality

Things to consider: 

  • Sensitive questions have a 25% higher non-response rate
  • they yield less accurate data
  • misreporting (acquiescing)