Paid Surveys

Marc Tillman asked:


This Article discusses best practices to consider in the administration of mail surveys.  There is no single idea that alone will dramatically improve response rates.  Rather, it is a combination of properly planned steps that can result in overall enhancement.  By keeping these practices in mind, it will help insure a quality experience for both the survey participants and the company on whose behalf the survey is conducted, as well as potentially improve response rates. 

At the outset, surveying companies should accept that most properly administered mail survey projects are fairly complicated.  If done correctly, they are far more involved than simple print, mail, tabulate and report.  Companies might want to consider the factors mentioned below when planning a mail survey: 

1.  Is the length of the survey instrument appropriate for the audience?

2.  Is there a well-written cover letter and instructions?

3.  Is there a method of reaching someone in the event of support issues?

4.  Is the mailing envelope properly identified?

5.  Is the mailing being sent out by 1st class USPS?

6.  Is a business reply envelope being used?

7.  Are awards or other incentives needed?

8.  Will online survey hosting increase the response rate?

Design of the Survey Instrument:

The length of the survey instrument must be appropriate for the audience.  A survey participant who pays $50.00 for a product that is readily available from multiple vendors and that is not a priority item to the participant will be less likely to complete a longer questionnaire than a participant who purchased an expensive car or an enterprise product.  This is a matter of common sense.  Consider how much time you would be willing to spend responding to a questionnaire - given the subject and offered incentives, if any.  In many cases, a one-sheet questionnaire printed on both sides (called “duplex” printing) is the appropriate length.  This still might be too long depending on the subject of the survey.

To some extent, consideration of length is accommodated by the topic.  For example, there are only so many questions that can be asked about the taste of toothpaste.  Yet some companies might include six or seven basic questions, 10 demographic questions, and another bank of questions on buying habits.  Before you know it, the questionnaire is close to four pages long.  If this happens, the surveying company might want to consider using incentives to actually keep the costs down (more about this later).

Packaging the Survey Instrument:

There are certain best practices a surveying company should consider relating to the packaging of the survey instrument.  This starts with what the participant first sees when he/she receives the mailing.  The mailing envelope should include the logo and colors of the surveying company so that the participant views this as an “official” mailing.  Printing of color logo will increase the costs of the project, but is something that should be strongly considered unless color logo is an expensive add-on given its design. 

Use of windows envelopes are preferred; although labels can be used if need be.  The advantage of a window envelope is the personalization of the cover letter without the need for and cost of a match with the mailing envelope.  We recommend the use of a clear and concise, personalized cover letter signed by a person of high authority at the surveying company.  The letter should set forth the purpose of the survey, confidentially requirements, support information, and the end date of the survey. Instructions on completing the survey instrument should be included on the cover letter - - or even better at the beginning of the instrument.

Except for rare instances, the use of 1st class USPS and business reply postage are important practices.  Each will add to the cost of the project - and can result in significant cost increases - but they each can have a significant impact on response rates.  Use of bulk rate postage can also result in delays in receipt of the mailings for varying percentages of participants. 

Use of Incentives:

Deciding whether or not to use incentives is for the most part a budgetary consideration; although in certain instances, it can also have a customer-relations element attached to it.  Assuming large quantities of available sample, one can always send out more mailings instead of offering incentives.  However, the key consideration is whether the use (and costs of administration) of an incentive saves money by producing more completes at a lower overall cost than sending out additional mailings.  If the survey population is limited in size, then awards might be needed to produce an adequate number of completes for actionable data.  Keep in mind that there might be quality of data issues relating to the use of incentives.  A surveying company should consult with an experienced survey company regarding the use of incentives and what the options might be.

Coordination of Mail Surveys with Online Survey Hosting:

It is not uncommon for surveying companies to consider the use of online survey hosting in conjunction with a mail survey.  If this is done, best practices suggest a variable insert of a unique login id into each mailing along with survey login instructions.  It would also be correct to variable insert a tracking id onto the questionnaire to de-dupe responders who complete both the mail survey and online survey. Technical support should also be available for online responders.

As is the case with incentives, the use of an online survey is primarily a budgetary consideration.  That is, will the use of an online survey enhance overall response rates to offset the added costs of offering this option? If not, then the use of an online survey might not be justified unless it is desirable from a client-relations standpoint.  Also, keep in mind when calculating the comparative costs that mail surveys have data entry fees while online surveys do not.  

In summary, when conducting mail surveys, a surveying company should carefully consider each and every step of the survey administration process.  That includes length of the survey instrument; packaging of the survey materials including mailing envelope and type of postage; formatting and personalization of the cover letter; survey support options; use of incentives; and viability of coordinated online survey hosting.



CHASE

Marc Tillman asked:


Survey hosting can mean different things to different people. At one end of the spectrum it can refer to an ASP offering market research software where users can program and administer their own surveys. At the other end of the spectrum it can refer to a full-service survey solution where the survey company’s professional staff programs and administers the survey on its own platform. This article describes differences in each approach and discusses some of the key considerations in the selection of a survey hosting vendor.

Survey Hosting and Software:

There is a limited number of survey and market research companies using their own research software to conduct surveys. End user clients of hosted survey solutions should note that the survey vendor might not be the actual developer of the software and/or could be hosting its surveys on another vendor’s platform. Many survey companies license their survey software from a 3rd party vendor specializing in survey engine development and install the software on their own platform. Other companies license the use of hosted survey solutions on another vendor’s platform to circumvent the need for software installation and ongoing hosting. In the case of licensed software, if the software developer does not provide a complete roadmap of the installation requirements, the survey software alone might not provide an adequate solution. Relevant factors can include: processing power of the hardware; proper selection and configuration of application servers (if needed); database selection and configuration; and the amount of bandwidth.

General Considerations:

In evaluating the best choice of vendor in the survey hosting space, here are five considerations — some of which are more important than others depending on your specific requirements:

1.  Do you prefer to program and administer the survey on your own (ASP model) or would you rather contract these functions with a full-service hosted survey provider?

2.  Does your survey instrument require special features or custom programming? 

3.  If you are engaging in market research surveys, how easily does the software coordinate with sample providers i.e., does the software offer redirects and/or recording of PINs?  Does the software offer quota stops? Does the software offer respondent filtering? 

4.  Does the software and its environment offer fast page views and an error-free survey experience for the volume of participants you are expecting? 

5.  Will you be able to provide technical support for survey participants? 

Pick the software that matches your needs: 

Regardless of whether you select an ASP or full-service solution, you need to identify the requirements needed for your survey projects and whether the software can meet those needs.  For example, if you expect to use such features as skip logic programming, branching, piping, percent tables, etc., then you want to select a vendor that offers these features. 

Similarly, if you expect to have a large volume of survey takers and/or use survey instruments which are long (such as 15 minutes or longer), make sure that the software has the capacity to handle this with fast page views and error-free data collection. 

If you require custom programming of question types or programming logic to meet your survey needs, make sure that the vendor can provide a solution.  For example, if the survey company is not the developer of the software and does not have a IT development staff, there might be limitations on the custom features it can offer.  Before selecting a vendor, ask for a general cost estimate on the development and deployment of your custom survey solutions.  

What about technical support? 

Obviously, you will want the vendor to provide you with adequate technical support.  If you select an ASP model, you may have questions on different stages of the user process.  A full-service approach passes the responsibility of deployment and administration onto the vendor; although there is a cost for this service.   

Just as important as your own technical support is the technical support provided to survey participants.  There needs to be a process in place to promptly respond to their questions, collect and remove incorrect email addresses from the system, and manage unsubscribes.  If you adopt an ASP approach that does not provide tech support for survey participants, be sure to designate several persons at your organization to handle support and make sure they are appropriately trained to answer technical questions.  

Full-service versus ASP — do you need professional services? 

The field of survey data collection generally requires as close to perfection as possible in every step of the process to produce a reliable data set.  One of the most important considerations in deciding whether an ASP approach to survey hosting will best meet your needs is your ability (and availability) to program, administer, and monitor the survey.  This can depend on such factors as the complexity of the survey instrument; length of the survey instrument; number of survey participants; and time in the field.  Your own (or staff member’s) survey expertise is also important, as is the ease of use of the ASP interface.  It is important to carefully examine the processes of each ASP vendor to make certain that you are comfortable choosing such a solution over full-service. 

Special considerations in the case of market research surveys: 

Programming and administration of market research surveys can require special expertise.  Screening questions need to be properly programmed to terminate non-qualifying participants and quota stops may need to be used to prevent excess completes in particular cells (manual monitoring is possible but it can require constant attention).  In addition, survey panel vendors providing survey sample may require interaction with their own platform.  Some will accept recording of PINs for each respondent; while others will require the insertion of re-direction links.  It is important to be certain that the hosted software correctly interfaces with the sample provider you will use.  

Another relevant consideration for market research surveys is the ability of the survey hosting platform to provide respondent filtering.  This can take many forms at different vendors.  At a minimum, you will want to be able to remove survey data of respondents who provide incomplete, contradictory or straight-lined responses.  The availability of respondent timers is also a nice feature because it allows you to identify and remove respondents who rushed through the questionnaire.  It is also helpful if the platform allows you to record each respondent’s PIN against his/her responses and use this information to re-sample those respondents, if needed. 

In summary, when evaluating the concept of survey hosting, it is important to determine which model - ASP or full-service - will best meet your needs and fully price out each vendor based on your specific requirements.  Start with your underlying software needs to determine which solutions meet all or most of your requirements.   Consider your programming and administration requirements and decide if you or a member of your staff has the desire, time, and ability to self-administer the survey.  A properly administered survey can be a time-consuming process.



ABDUL