Interlocked quotas vs control quotas vs balanced fill

Introduction to quotas and qualifications

Qualifications are the specific criteria a respondent must meet to be eligible to participate in a market research study. These are the characteristics, traits and experiences that make a person suitable for the research topic. From a respondent's perspective:

• Qualifications are typically assessed through screening questions at the beginning of a target group or during the respondent's requirements.

• They may include demographic factors (age, gender, location), behaviors (product usage, purchase frequency), or specific experiences (job role, industry).

• Respondents who do not meet the qualifications are usually "screened out" and will not be allowed to complete the full target group.

Quotas are predetermined limits on the number of respondents with specific characteristics that can participate in a study. From a respondent's perspective:

• Even if a respondent qualifies for a study, they may be turned away if the quota for their particular group has been filled.

• Quotas ensure a balanced representation of different subgroups within the sample.

• Common quota categories include age, gender, and region, but may also include more specific criteria such as industry or job title.

Outcome scenarios (non-interlocked quota)

Based on the qualification and quota checks, several outcomes are possible:

See the below target group example with 18 to 24-year-olds (age) and males (gender) qualifications.

1. Full qualification: If the respondent meets all qualifications and falls into open quota categories, they may continue with the full target group.
   Example: If Jack is a 23-year-old male, he may continue with the full target group.

2. Disqualification: If the respondent doesn't meet at least one qualification or if all relevant quota cells are full, they are typically screened out of the target group.
   Example: If Jack is 23 years old and not identified as male or if the quotas are already filled, he will be screened out of the target group.
   Example: If Jack is a 30-year-old male and the quotas have not yet been filled, he will be screened out.

3. Quota full termination: Even if a respondent qualifies, they may be terminated if the quota they fit into is already full
   Example: If Jack is a 23-year-old male but the quotas are already filled, he gets terminated from the study.

Introduction to interlocked quota qualifications

Interlocked quotas involve multiple demographic or characteristic variables that are considered simultaneously, rather than independently. This means that a respondent must fit into a specific combination of quota categories to qualify for the survey.

Outcome scenarios (interlocked quota)

Based on the qualification and quota checks, several outcomes are possible:

See the target group example below with one age-and-gender-interlocked quota for 18- to 24-year-old males, as well as one Japanese (ethnicity) qualification.

1. Full qualification: If the respondent meets all exact combinations of characteristics defined in the interlocked quota quals and any other non-interlocked quals available, and fits into open quota cells, they are allowed to continue with the full target group. The main purpose of the interlocked quota is to honor the number of assigned quotas per condition combination.
   Example: If Jack is a 23-year-old Japanese male, he may continue with the full target group.

2. Disqualification: If the respondent does not meet the qualifications (either interlocked or non-interlocked) or if all relevant quota cells are full, they are typically screened out of the target group.
   Example: If Jack is a 30-year-old Filipino male, he gets screened out.
   Example: If Jack is a 23-year-old Filipino female, she gets screened out.
   Example: If Jack is a 30-year-old Filipino female, she gets screened out.

3. Quota full termination: Even if a respondent qualifies, they may be terminated if the quota they fit into is already full.
   Example: If Jack is a 23-year-old Japanese male, but the quotas are already filled, he will be terminated from the study.

Interlocked quotas vs control quotas vs balanced fill

Interlocked quotas

Interlocked quotas involve multiple demographic or characteristic variables that are considered simultaneously by allowing users to assign respondent counts to multiple combinations of conditions across profiles.

In the example below, if you have 3 separate profiles with multiple conditions, like

• AGE: 18-24 & 25-35

• GENDER: male & female

• ETHNICITY: White & Black or African American
  After interlocking, you will see that this allows the users to assign respondent counts to all the possible condition combinations within those selected profiles. This resulted in 8 quotas to be set by the users.

Advantages

1. Ensures assigning respondent counts to multiple condition combinations

2. Allows for more precise sample composition

3. Better reflects complex population structures

4. Facilitates analysis of interactions between variables

Limitations

1. Can significantly reduce the eligible sample pool

2. May lead to increased fieldwork time and cost

3. Risk of creating unrealistic or hard-to-fill quota cells

Control quota

Control quotas are a subset of Interlocked Quotas. They are used to ensure that certain key characteristics of the sample population are represented in predetermined proportions by allowing users to assign a respondent count to a specific condition combination across profiles.

In the example below, if you have 3 separate profiles with multiple conditions, like

• Age: 18-24 and 25-35

• Gender: Male and female

• Ethnicity: White and black or African American
  By creating a control quota, you can specify specific condition combinations across 3 selected profiles to assign the number of respondents to each. In this case, 18-24-year-old white females are assigned to 25% of the total quotas. This resulted in 1 quota being set by the users.

Advantages

1. Ensures assigning a respondent count to a specific condition combination across profiles.

2. Less likely to result in hard-to-fill cells

3. More efficient in terms of fieldwork time and cost

Limitations

1. May not capture complex interactions between variables

2. Can lead to the over-simplification of sample representation

3. May experience a lack of balance as a result of over-popular qualifications.

The outcome scenarios on interlocked vs control quotas follow the same rules per quota and qualification.

Balanced fill

This is a feature as part of our Automate Fielding tool, that automates the workflows of the survey when the survey goes live, considering the customer’s goals, such as controlling the budget, cost, pace and filling portions.

This feature ensures the survey gets filled proportionally by pacing the survey at the same speed as the most difficult quota to fill. The imbalance causes some surveys to be infeasible in the later stages, as the inferred combination of quotas that need to be satisfied targets a very difficult audience in the end.

For example, if your survey collects more cat owners than dog owners within the same duration, by selecting the ‘Balanced Fill’ option, you make sure your survey paces at the same speed as the dog owners responding to your survey. Therefore, you make sure you get the proportional amount of dog and cat owners within your survey, although the dog owner respondents were more difficult to get.

Advantages

1. Ensures the survey gets filled proportionally

2. Paces the survey with the same speed as the most difficult quota.

3. Increase the feasibility of the survey

4. It improves the filling rate and the accuracy of surveys when small quotas are used with interlocked quota quals.

Limitations

1. It could make the survey slower when it involves an attractive qualification, deliberately slowing respondent recruitment until the unpopular qualifications are filled proportionally.

2. It could make the survey more expensive as a result of deliberately slower respondent recruitment while making sure the survey doesn’t get overquota.

It is recommended to use balanced fill in automate fielding in order to improve the survey