This is “Online Surveys: Gathering Data”, section 18.4 from the book Online Marketing Essentials (v. 1.0). For details on it (including licensing), click here.

For more information on the source of this book, or why it is available for free, please see the project's home page. You can browse or download additional books there. To download a .zip file containing this book to use offline, simply click here.

Has this book helped you? Consider passing it on:
Help Creative Commons
Creative Commons supports free culture from music to education. Their licenses helped make this book available to you.
Help a Public School
DonorsChoose.org helps people like you help teachers fund their classroom projects, from art supplies to books to calculators.

18.4 Online Surveys: Gathering Data

Learning Objectives

  1. Learn how to develop surveys online.
  2. Learn how to get responses.

When developing surveys you can combine qualitative data with quantitative data—it just depends on how the questions are asked. Conducting online surveysA quantitative research method conducted via the Internet that aims to collect information about a population. allows for data to be captured immediately, and data analysis can be performed easily and quickly. By using e-mail or the Web for conducting surveys, geographical limitations for collecting data can be overcome cost effectively.

Developing technology also allows for sophisticated and user-friendly surveys to be compiled. For example, as opposed to indicating impressions on a sliding scale, respondents can indicate emotional response.

Figure 18.4

Metaphorix uses animations when creating surveys. Visit http://www.metaphorixuk.com.

Compare the images above to the following survey question:

Rate how you feel about a brand:
negative neither positive nor negative positive

Developing Surveys: Asking Questions

The success of a survey in gathering useful data is largely determined by the design of the survey, and particularly by the questions that are asked. A survey can comprise any number and types of questions, and these should be structured in such a way that more complicated questions only appear once users are comfortable with the survey.

Be careful when creating questions that you do not introduce biasAn influence on the research process that affects one’s ability to analyze the research results objectively. by asking leading questions.

Example of Leading Question Bias

An example question might be the following:

We have recently introduced new features on the Web site to become a first-class Web destination. What are your thoughts on the new site?

Replace this with the following to get rid of bias:

What are your thoughts on the changes to the Web site?

Questions in the survey should be brief, easy to understand, and most of all, easy to answer.

Types of Survey Questions

Open-Ended Types

Open-ended questions allow respondents to answer in their own words. This usually results in qualitative data. Take the following example:

What features would you like to see on the Web site for the eMarketing textbook?

If there are enough respondents to an open-ended question, the responses can be used quantitatively. For example, you can say with some certainty, “37 percent of people thought that case studies were an important feature.”

Closed-Ended Types

Closed-ended questions give respondents specific responses to choose from (i.e., they are multiple choice, with one answer or multiple answers). This results in quantitative data. Take the following examples:

Do you use the eMarketing textbook Web site? Choose one that applies.

—Yes

—No

What features of the eMarketing textbook Web site do you use? Check all that apply.

—Blog

—Case studies

—Free downloads

—Additional resources

Ranked or Ordinal Questions

These questions ask respondents to rank items in order of preference or relevance. Respondents are given a numeric scale to indicate order. This results in quantitative data. Take the following example:

Rate the features of the eMarketing textbook Web site, where 1 is the most useful and 4 is the least useful.

—Blog

—Case studies

—Free downloads

—Additional resources

Matrix and Rating Types

These types of questions can be used to quantify qualitative data. Respondents are asked to rank behavior or attitude. Take the following example:

The eMarketing textbook Web site is a useful tool for further studies.

Figure 18.5 Different Types of Rating Scales

Or

The eMarketing textbook Web site is a useful tool for further studies.

Figure 18.6

Rating scales can be balanced or unbalanced. When creating the questions and answers, choosing balanced or unbalanced scales will affect whether you are collecting data where someone can express a neutral opinion or not.

Figure 18.7

Figure 18.8

How to Get Responses: Incentives and Assurances

As the researcher, you know what’s in it for you in sending out a survey: you will receive valuable data that will aid in making business decisions. But what is in it for the respondents?

According to SurveyMonkey, the ways in which the surveys are administered play a role in response rates for surveys, and these can be relative:

  • Mail. 50 percent adequate, 60 to 70 percent good to very good.
  • Phone. 80 percent good.
  • E-mail. 40 percent average, 50 to 60 percent good to very good.
  • Online. 30 percent average.
  • Classroom pager. More than 50 percent good.
  • Face to face. 80 to 85 percent good.SurveyMonkey, “Smart Survey Design,” June 12, 2008, http://s3.amazonaws.com/SurveyMonkeyFiles/SmartSurvey.pdf (accessed September 29, 2008).

Response rates can be improved by offering respondents an incentive for completing the survey, such as a chance at winning a grand prize, a lower priced incentive for every respondent, or even the knowledge that they are improving a product or service that they care about.

There is a train of thought that paying incentives is not always a good thing. Among less affluent or educated respondents it may predispose them to feel that they need to give so-called good or correct answers that may bias your results. Alternatively you may attract respondents who are in it just for the reward. One approach could be to run the survey with no incentive with the option to offer one if responses are limited.

Figure 18.9

An incentive is offered for completing a survey.

Designing the survey so as to assure respondents of the time commitment, and privacy implications, of completing the survey can also help increase responses.

Conducting Research Surveys: A Step-by-Step Guide

As with all things eMarketing, careful planning goes a long way to determining success. As market research can be an expensive project, it is important that planning helps to determine the cost versus the benefit of the research. Qualitative research and secondary research are critical steps in determining whether a larger-scale research project is called for.

Bear in mind that many tasks that fall under the umbrella of research should be ongoing requirements of eMarketing activities, such as conversion testing and optimizing and online reputation management. Polls and small surveys can also be conducted regularly, and nonintrusively, among visitors to your Web site.

Step 1: Establish the Goals of the Project—What You Want to Learn

Secondary research can be used to give background and context to the business problem and the context in which the problem can be solved. It should also be used to determine alternative strategies for solving the problem, which can be evaluated through research. Qualitative research, particularly using established online research communities, can also help in determining what the business problems are that need to be solved. Ultimately, determine what are the actions you will be considering after the research is completed and what insights are required to make a decision on those actions.

Step 2: Determine Your Sample—Whom You Will Interview

You do not need to survey the entire population of your target market. Instead, a representative sampleA group of people who are representative of the population referred to in the research. can be used to determine statistically relevant results. In selecting a sample, be careful to try to eliminate bias from the sample. Highly satisfied customers, for example, could give very different results compared to highly dissatisfied consumers.

Step 3: Choose Research Methodology—How You Will Gather Data

The Internet provides a multitude of channels for gathering data. Surveys can be conducted online or via e-mail. Online research panels and online research communities can all be used for gathering data. Web analytics can also be used to collect data, but this is a passive form of data collection. Determine what will provide you with the information you need to make decisions. Be sure to think about whether your research calls for qualitative or quantitative data as this determines the methodology as well.

Step 4: Create Your Questionnaire—What You Will Ask

Keep the survey and questions simple, and ensure that the length of the survey does not overwhelm respondents. A variety of questions can be used to make sure that the survey is not repetitive.

Be sure when creating the questions that you keep your goals in mind: don’t be tempted to try to collect too much data, or you will likely overwhelm respondents.

Step 5: Pretest the Questionnaire, If Practical—Whether You Are Asking the Right Questions

Test a questionnaire to determine if questions are clear and that it renders correctly in various browsers or e-mail clients. Ensure that test respondents understand the questions and that they are able to answer them satisfactorily.

Step 6: Conduct Interviews and Enter Data—How You Will Find Out Information

Run the survey! Online surveys can be completed by respondents without your presence; you just need to make sure that you get them in front of the right people. A survey can be sent to an e-mail database or can be advertised online.

Step 7: Analyze the Data—What You Find Out

Remember that quantitative data must be analyzed for statistical significance. The reports should aid in the decision-making process and produce actionable insights.

Room for Error

With all research, there is a given amount of error that needs to be dealt with. Errors may result from the interviewers administering a questionnaire (and possibly leading the respondents) to the design and wording of the questionnaire itself, sample errors, and respondent errors. Using the Internet to administer surveys and questionnaires removes the bias that may arise from an interviewer. However, with no interviewer to explain questions, there is potential for greater respondent error. This is why survey design is so important and why it is crucial to test and run pilots of the survey before going live.

Respondent errorAn error that occurs when respondents become desensitized to the research process. also arises when respondents become too used to the survey process. There is the possibility of respondents becoming desensitized. There is even a growing trend of professional survey takers, especially where there is an incentive involved. The general industry standard is to limit respondents to being interviewed once every six months.

Sample errorAn inaccuracy in the results of research that occurs when a population sample is used to explain the behavior of the total population. is a fact of market research. Some people are just not interested in, nor will ever be interested in, taking part in surveys. Are these people fundamentally different, with different purchasing behavior, from those who do? Is there a way of finding out? To some extent, Web analytics, which tracks the behavior of all visitors to your Web site, can be useful in determining the answer to this question.

When conducting any survey, it is crucial to understand who is in the target universe and what the best way to reach that target universe is. Web surveys exclude elements of the population, due to access or ability. It is vital to determine if this is acceptable to the survey and to use other means of capturing data if it is not.

Conducting Research: Who’s Going to Pay?

Regular research is an important aspect of the growth strategy of any business, but it can be tough to justify the budget necessary for research without knowing the benefit to the business. Conducting research can cost little more than the time of someone who works for a company, depending on the skills base of employees, or it can be an expensive exercise involving external experts. Deciding where your business needs are on the investment scale depends on the depth of the research required and what the expected growth will be for the business. When embarking on a research initiative, the cost to benefit ratio should be determined.

Testing should be an ongoing feature of any eMarketing activity. Tracking is a characteristic of most eMarketing that allows for constant testing of the most basic hypothesis: is this campaign successful in reaching the goals of the business?

Key Takeaways

  • Conducting surveys online allows for data to be captured immediately.
  • Developing technology allows for sophisticated and user-friendly surveys to be compiled.
  • The success of a survey can be determined by how the survey is designed.
  • Questions should be worded in a way that allows for an honest answer from the user. All questions should be easy to understand and answer.

  • There are four main types of survey questions:

    1. Open-ended
    2. Closed-ended
    3. Ranked or ordinal
    4. Matrix and rating
  • Survey respondents should get something in return for the data they provide you.
  • Response rates improve greatly when there is an incentive for the respondents. However, some people do not believe this is good to do, as responses may be biased. Survey takers may be in it just for the incentive.
  • Careful planning is essential to any market research initiative.

  • The steps to executing market research properly are as follows:

    1. Establish the goals of the project.
    2. Determine your sample.
    3. Choose research methodology.
    4. Create your questionnaire.
    5. Pretest the questionnaire, if practical.
    6. Conduct interviews and enter data.
    7. Analyze the data.
  • Be aware that there is room for error.
  • Costs for a research initiative can vary, depending on the scope of the project. It may be appropriate to do the study from within the company, or it may be necessary to hire external experts.

Exercises

  1. Develop survey question examples for each of the types of survey questions outlined in this section.
  2. Evaluate the seven steps to conducting research. What do you think might happen if one of these steps is missed or poorly handled? Give an example.