Cint provides buyers with the option to exclude respondents who have provided unusable responses. Although this is an available option, it is a better practice to prevent these respondents from completing the survey. Eliminating poor or unqualified respondents before they complete the survey can save researchers time reviewing responses manually and prevent the need to go back into the field to replace poor-quality respondents.
Additionally, it's important to be aware that post-completion reconciliation can be harmful to the Cint Exchange when used excessively. It is a poor experience for respondents to complete a survey and not get paid, and it reduces trust for suppliers that their respondents will have a good experience in your survey. Both sample providers and researchers can reduce the number of poor-quality respondents. Below are some strategies researchers can use to increase quality and reduce negative reconciliations.
Cint Exchange pre-screener setup
Logical targeting order
It is best practice to think of Cint Exchange targeting like a funnel. The broadest qualifications should be asked first, narrowing the qualifications in a logical order. This will reduce the likelihood of asking leading questions of respondents and increase the likelihood that qualifying respondents are sent into your survey. Example: When looking for respondents who work for a company with $1 Million+ in annual revenue, you should ask if a respondent is employed before asking them how much revenue their company delivers.
Standard qualifications
Cint offers a robust library of standard qualifications. Using these wherever possible helps ensure Cint Exchange suppliers know exactly which respondents to send to your surveys. When Custom Qualifications are used instead of Standard Qualifications, you may see a higher rate of unqualified respondents attempting to access your survey. Custom qualifications are not always avoidable or bad, but should be used with purpose. If you have any questions about their use, please contact your Cint Representative.
Red herring or attention-check questions
Attention checks and red herring questions added to the Cint Exchange prescreener can help terminate inattentive or unqualified respondents very early in the respondent experience, reducing the need to clean data and reconcile. Cint has included several of these questions in the custom qualification library to make them easy for any user to add.
Red herring questions can be used alongside surveys that require specific knowledge of the respondent. Typically, the question is knowledge-based, and at least one answer is the “red herring”. Example: Which of the following financial software programs have you personally used in performing your job?
Attention-check questions generally do not require specific knowledge and are used simply to ensure the respondent is engaged. Example: Which of the following objects are NOT blue?
Experience disclaimer
Giving respondents an honest look at what will be asked of them during a survey can help them prepare for the specific requirements of your survey. This can also give the respondent an opportunity to opt out before the survey, should the described experience not be one they are willing to engage in. By giving the respondent clear expectations, they are more likely to answer each question truthfully and thoughtfully. These types of disclaimers can be built as a custom qualification in the Cint prescreener or placed at the beginning of your hosted survey.
Note: These disclaimers should not reveal survey subject or other details that may lead to biased data. Example: The following survey experience will require you to watch a 5-minute video and then answer 10 questions about it. You will need to have your volume turned up to answer the questions. Are you willing and able to participate in this exercise?
Survey experience and setup
Length of interview
Respondents experience survey fatigue in long surveys. This leads to a higher frequency of inattentiveness, poor-quality responses, and, ultimately, higher Negative Reconciliation Rates. As a baseline, the Cint Exchange average for length of interview is 12 minutes. Cint's suppliers and individual respondents have provided feedback indicating that they consistently prefer shorter surveys, even at lower incentives. LOIs shorter than 15 (10 or fewer preferred) minutes will typically have lower negative reconciliation rates, the highest traffic, and better overall quality.
Recommendations: Remove any questions that will not be needed for analysis, and prioritize questions that you ask (sometimes less is more). This is, of course, easiest to accomplish in the survey design phase, so having conversations with researchers early in the process is key. As a general rule, for every 3 questions you ask, add 1 minute to your LOI.
Additional reading: Patrick Comer published a white paper on the topic of LOI and how it relates to data quality and trends in market research called Comer’s Law.
Open-end frequency
Online survey respondents are typically exposed to 3 or fewer open-ended questions. We recommend thoughtful consideration when deciding when/where to ask respondents to provide detailed answers.
An overuse of open-ended questions can lead to:
Diluted feedback
Increased the length of the interview
Increased drop rate
The survey experience is not friendly for mobile devices
Increased negative reconciliation rates
Recommendations: Limit the number of open ends to 3 when possible, and avoid asking a respondent “Why?” after every question—this can become tedious and, in some cases, annoying. Too many or repetitive OE questions can lead to increased negative reconciliations. It should also be noted that open-ended questions can prove to be more difficult for respondents using mobile devices to access your survey. Since over half of Cint Exchange respondents use mobile devices to access surveys, careful consideration of the mobile respondent experience is recommended.
Excessive/lengthy grid questions
Excessive, repetitive and/or lengthy grid questions can cause respondent fatigue and boredom. This often leads respondents to straight line, speed, or otherwise become inattentive. These behaviors are correlated with an overall increase in negative reconciliation rates. This is especially true for respondents using mobile devices.
Recommendations:
Repeat your column headers every 5 lines—this helps break the grid into more manageable sections and also reminds respondents of the selections they are making.
Don’t ask multiple grid questions in a row. Instead, change how you graphically display grids by using “card sorts” or other exercises that gather the same information, but are more engaging to respondents.
Consult with your survey programming team for additional advice! They may have other ideas for gathering useful insights without resorting to repetitive grid questions.
Leading questions and logical progression
The goal of most research is to gather unbiased responses from qualified individuals. Much like the recommended logical progression of questions in the Cint Exchange prescreener, your survey screener should be intelligently structured. Giving away the survey's subject or leading respondents to qualify will result in poor-quality data (or data from respondents who shouldn’t truly qualify). All questions should be asked in a non-leading way to ensure high quality and fidelity of the survey data.
Recommendations: Ask questions in a logical progression and avoid those that lead respondents to make a specific selection. This includes not asking yes/no questions, in most cases. Instead of asking: “Do you work in the IT industry?”, ask: “In which industry do you work?” Asking questions in a neutral way will encourage more qualified respondents to answer in the body of your survey, leading to fewer quality issues and fewer negative reconciliations.
Using a survey platform to auto-terminate
In many cases, survey hosting platforms can add automatic quality checks that terminate respondents if they fail one or more of them. Some of these features, which are included in popular survey platforms like Decipher, Qualtrics, Alchemer, etc., are:
Automatic speeder termination
Automatic straightliner flags
Automatic answer logic checks
Note: When using these automatic termination features, you should redirect respondents back to Cint Exchange using the security/duplication redirect. If you need help implementing, reach out to your Cint representative or survey programming team.
Mobile friendliness
As of Q1 2021, over 60% of Cint Exchange traffic worldwide is accessed via mobile devices. The best practice is to ensure your survey is not only mobile-compatible but also mobile-friendly. Mobile-compatible surveys allow mobile traffic to enter, while a mobile-friendly survey makes the survey experience easy on mobile devices. If a survey is not optimized for mobile devices, the resulting frustration could leave a survey respondent unable or unwilling to provide high-quality feedback.
Recommendations: A mobile-friendly survey should minimize scrolling, adhere to best practices for long-form open-ended questions, keep the LOI to a reasonable time, and ensure all images and videos scale correctly. Many of these items are the same for non-mobile devices as well! Your survey programmer may have additional advice on optimizing your survey for mobile traffic. If your survey is not mobile-friendly, you should consider using the MS_is_mobile qualification to restrict mobile traffic. This will impact the amount of traffic available, so making a survey optimized to mobile traffic is preferred.
Personally identifiable information (PII) collection
Collecting PII can sometimes be necessary. Collecting PII, though, can increase the possibility of receiving false information, as people are generally protective of their personal information.
If you do need to collect PII, follow these recommendations:
Make PII Collection an optional step, if possible
If PII Collection is not optional, collect the PII as early as possible in the survey, and give the respondent the opportunity to refuse to participate, and redirect the respondent back to Cint as a terminate
Provide an explanation for why you need the respondent’s PII, and how it will be used
Other possible improvements
Redirect errors
Without properly placed, secure redirects, your survey may not perform as expected or may be subject to fraudulent activity. It is also important to ensure your redirects are correctly placed in your survey. If a redirect is placed incorrectly within your survey platform, this can cause Cint Exchange to recognize respondents as a completed survey, when they are, in fact, not. Recommendations: Be sure to test your survey before launching to catch any setup errors before sending it to live respondents. Also, always use the most recent version of the redirects provided by your Cint Representative. If you have any questions about redirects, please contact your Cint Rep.
Survey load times
Surveys with long load times due to downloading large video or graphic files can cause impatience among survey takers. Consider reducing the number of images and videos, or their quality, to ensure that mobile and desktop users can access the survey without long wait times.
Survey translation
When translations are poor, it can increase the risk of bad responses due to misinterpretation. Avoid Google Translate! If you need a recommendation for a survey translator, please contact your Cint Rep.
This is not an exhaustive list. If these solutions do not reduce your negative reconciliations, or if you are having difficulty with specific types of surveys, please reach out to your Cint Representative.