Yield management best practices guide

Yield management is the strategic process of carefully selecting and prioritizing viable survey opportunities to ensure that the highest-performing studies receive the most traffic, while surveys falling below desired metric thresholds are either deprioritized or not considered at all.

To help our supply partners build out an effective yield management system we recommend applying the following approaches, amongst a few other considerations. This content is not relevant for our Match Integrated partners.

• Creating suitable survey filter rules

• Identifying viable surveys

• Ranking eligible surveys

• Pacing survey entrants

• Tracking survey performance

• Shutting down poorly performing surveys

• Building a testing framework

Creating suitable survey filter rules

When starting out your integration it is important you monitor if any initial survey filtering rules applied to your Feed Opportunities subscription or within your own system are heavily restricting the survey inventory available. Having a significantly reduced pool of surveys to match users with can impact potential earnings.

A recommended best practice is to keep your survey filtering criteria broader during the soft launch phase. As you gain a better understanding of how your respondents perform against different survey types, length of interview (LOI), buyer preferences etc, you can gradually refine and test your survey filtering rules to optimize performance. See here for more information on creating a testing framework.

Feed opportunities allows you to easily adjust the criteria for surveys you receive notifications about, simplifying the process of gradually filtering out surveys that are not relevant to your specific business model. By adopting this approach, you can adapt and tailor your survey selection process to align with your users' preferences and maximize your survey-based revenue.

Example Survey Filtering

Example Opportunities Subscription Request

{
    "callback" : {{callback}},
       "opportunities": [{
            "country_language" : {"in": ["eng_gb", "eng_us"]},
            "length_of_interview": { "lte" : 20 },
            "conversion": { "gte" : 0.05 },
            "revenue_per_interview": { "gte" : 0.50 },
            "study_type": {"in": ["adhoc", "recontact"]}
     }]
}

Identifying viable surveys

Before presenting the best survey opportunities to your respondents, it's crucial to ensure that the following criteria is met:

• The survey is still live

• The total quota is available

• Any subquota(s) matching respondent profiles is available

• Data profiled on a respondent meets the qualification criteria

• Respondent has not attempted this survey before

• Respondent has not attempted a survey in this survey group before (if applicable)

Ranking eligible surveys

Cint provides a comprehensive set of data through our APIs and redirect information, empowering suppliers to target the most suitable and high-performing surveys. To assist you in effectively utilizing this data, we've compiled a list of what we consider to be crucial data points when creating a survey ranking algorithm.

Please note some of these metrics should be used as a starting point for assessing the overall health of surveys from a global perspective, it's essential to track your internal performance as you direct respondents to surveys.

Pacing Survey Entrants

When sending respondents to a survey, we recommend to initially pace the volume of users sent in, testing with a small group of 20-30 respondents to understand the performance.  This practice is particularly important for those opportunities that are brand new to Cint and have no global data.  If your system's internal metrics surpass the desired performance benchmarks, start to open the survey to a larger audience, continuing to monitor the performance on a rolling survey entrant basis, altering survey rankings based on the latest data.  

For surveys that fail to achieve completion or fall short of performance thresholds, refer to the guidelines provided in the Shutting down underperforming surveys section.

You should also ensure that you are effectively pacing respondents into surveys based upon the total number of completes still available.  An elevated overquota rate may occur if a survey is being offered to an excessive number of respondents.  A good rule of thumb for how many respondents should enter a survey at any given moment is to not exceed the number arrived at using this formula:

Quota remaining / System conversion rate

Tracking Survey Performance

Before sending respondents to surveys, you should utilize the data retrieved through our APIs to understand the global performance for each survey. As the performance of surveys evolves in-field, it's crucial to continuously collect and update the performance information in your database.  

Once you start sending users to a survey, it becomes important to track, record and use your own internal performance metrics to determine whether to continue sending respondents to that survey or not.  While global data offers a valuable indication of survey performance, it's important to note that the performance may vary significantly when it comes to your own users.    

To make informed survey ranking decisions, we recommend employing the following logic for storing and utilising performance data:

• Global Performance: Use the global performance metrics (e.g. conversion and RPC) returned from the API’s prior to respondents attempting a survey

  • In cases where the performance metrics are zero, consider testing and storing an average value based on prior performance data. Ideally, adjust this value based on historic buyer and country-specific performance.

• Internal Performance: Once respondents begin their participation in a survey, start calculating internal performance metrics, using this information as the primary metric for ranking decisions.  We recommend using the internal performance data after 20 sessions have finished and the session outcomes have been collected.  

  • When calculating internal performance metrics, we recommend excluding Cint Survey Protection Services session outcomes. These terminations are based on user quality and are not an indication of the performance of a survey.  Additionally, if a respondent hasn't returned to your platform after 2x the survey's median LOI (or if LOI isn't available, use BidLOI), we advise classifying them as a "drop". Update your database with the returned session outcome data if they later return.

To aid with calculating your internal performance, there are two methods for collecting session outcomes.

1. Redirects: To collect the Cint Exchange (Marketplace) prescreener and Client response codes for each respondent sessions ensure the below parameters are appended to all of your redirects.  

   1. InitialStatus=[%InitialStatus%]

   2. ClientStatus=[%ClientStatus%]

      1. If these are missing, speak to your Account Team to have them included. Our collecting data from redirects resource provides more information on what parameters can be appended and the formatting options available.

2. Feed Respondent Outcomes: Create a outcomes subscription to receive notifications to your webhook on respondent session outcomes.  Our recommendation is to receive events for all available response codes, but if you want to limit these to specific response codes, you can achieve this through the filtering options available. See our Feed Opportunities and Respondent Outcomes FAQs for some example subscriptions.

Shutting Down Poorly Performant Surveys

An essential component of effective yield management involves establishing minimum survey performance thresholds within your system and deactivating surveys that fall below the desired criteria. Continuously directing a large volume of respondents to surveys that perform poorly not only adversely affects metrics such as conversion rates and RPC but also impacts respondent satisfaction.  Ideally, these performance thresholds should be configured at the country level, as survey performance can vary across different countries.

As mentioned in the previous section Tracking Survey Performance, it is recommended to initially utilize the global data provided through our APIs to determine the performance of surveys. This will help identify surveys that fall below any performance thresholds and should consequently be marked as 'inactive' in your database and not considered in your survey ranking decisions. If the global survey performance improves over time, you should update your system with the latest information and set the survey status as 'active'.  

As you begin directing users to survey opportunities, make use of your internal performance metrics to determine whether to increase or halt traffic to specific surveys.  Surveys that do not result in a complete or fall below your performance threshold should be marked as inactive.  Where you increase send, continue to monitor the performance on a rolling survey entrant basis, altering survey status/rankings based on the latest data.

Building a Testing Framework

Establishing the right yield management algorithm typically involves an iterative process, where different strategies are tested to understand the best approach. Therefore, we recommend developing a system framework that facilitates split testing with different survey ranking variations and enables the seamless implementation of successful strategies. Testing incremental changes in survey targeting and prioritization is a low-risk approach for acquiring valuable insights into effective methods without majorly affecting overall performance. This approach allows you to adapt to evolving trends and remain competitive in the Cint Exchange.

After Soft Launch is often a great place to start testing, you will have accumulated an initial dataset related to survey integration performance. Utilize this data to plan and implement futures changes to be tested. Based on your data-driven hypotheses, introduce these modifications in the form of test variations and closely monitor the performance.

For those surveys you have tested and made inactive due to poor performance, we suggest keeping an eye on the global data to observe any improvements made by buyers in-field that has enhanced the survey's performance. It can be beneficial to re-test a survey to assess whether any changes made by the buyer result in better outcomes for your respondents.

Additional Considerations

Respondent Segmentation

Having a diverse pool of respondents means that applying a single, catch-all, survey ranking algorithm can restrict the success of your integration.  Segmenting your panel allows you to target these respondents with opportunities that will improve performance, customer satisfaction and reduce your risk to reconciliations.  We have outlined below a few options to consider including in your yield management algorithm:  

• Respondent Age: For new users prioritizing surveys with a shorter LOI and/or higher conversion rate is a good way to ensure they have a favorable experience.  The more opportunities they are able to complete and earn rewards from, the less likely they are to churn.

• Respondent Quality: If a respondent has a low-quality score, either a function of failing quality checks or they are brand new users, regulate the types of inventory presented.  For example, excluding them from higher revenue per interview surveys.

Predicting Survey Performance

In the Pacing Survey Entrants section, we spoke about the importance of testing new opportunities with a small pool of respondents to understand performance and adjust sample send based on your survey metrics.  We are going to take this concept one step further and focus on how you can optimize your send by prioritizing the best new survey opportunities.  Particularly in our high demand markets, where there can be a large volume of new surveys launching at any given time.

Typically survey rankings are formulated based on in-field metrics such as conversion, RPC and RPCM.  So how can you rank new opportunities when this data is not available?  What we recommend is to apply a machine learning approach and train your system on what are the main characteristics and data patterns (outside of field data) behind good converting surveys, by each buyer.

Once your system has identified the different data patterns attributed to survey performance, assign a prediction score against the criteria based on the confidence levels of achieving the desired outcome.  

This scoring system should then be applied to your overall survey ranking algorithm.  In larger demand markets you may want to apply a conservative approach and only present the best-predicted surveys to users once you have exhausted all surveys with in-field metrics above your desired thresholds.  Or, you may want to take the approach to weigh new opportunities, where you are confident in the performance outcome, on a similar scale to those surveys where you have in-field data.

Irrespective of the option you apply we recommend when sending to new surveys you continue to track the first 20 entrants and adjust rankings based upon your own internal data.

Reporting

Utilize the Supplier Portal reports below to gain insights into the performance of your API integration. After logging into the Portal, click on 'My Reports' to access this valuable information.

Supplier Cint Exchange Feed Dashboard:

This report offers a comprehensive overview of your integration, encompassing key metrics such as conversion and GMV. Additionally, it provides detailed insights into why survey respondents are terminating, whether it's in the prescreener or client survey. You can toggle the data between the past 6 months and the past 14 days.

Supplier Yield Management Dashboard:

This report examines the performance of your integration at the survey level over the past 3 days. It serves as a valuable resource for identifying instances where you might be directing excessive traffic to underperforming surveys and not terminating poorly performing surveys promptly

Survey ranking example

Having shared our recommendations for building an effective Yield Management system, we have concluded this article with a simple example of how you can rank surveys within your system.

Objective: Achieve a high user retention rate and maximize earning potential by filtering out longer LOI surveys and prioritizing the higher-converting studies.

Steps

1. Create entry links for eligible surveys based on set filtering rules.  In this example we have used the following rules:

Filter Rules

2. Create a table that stores data collected from the Feed Opportunities data for each eligible survey.  These data points will be used as inputs for the columns added in the next step.   Survey ID, Global Conversion %, and LOI will all be stored in this example.

Example Table

3. Add calculated columns to the table.  These data points will be used as inputs for the columns added in the next step.  In this step, the columns added will be Internal Conversion and Functional Conversion %.

Example Table:

4. Add survey rank and status columns.

Example Table