How To Use Premise for Surveys

App-based opt-in panel surveys are the most common type of survey at Premise. They are used and trusted by commercial customers like Serta Simmons; international development partners like the World Health Organization; research institutions, like the Vaccine Confidence Project at the London School of Hygiene and Tropical Medicine; and media organizations, like The Economist. 

To better understand app-based opt-in panel surveys, let’s start by defining two key terms:

1. 1. App-based surveys – A type of online survey that uses an app instead of a website. This is the most popular type of survey in the world because the process of collecting survey data is fully digital and easy to manage. 
   2. Opt-in panels – Panels where respondents have signed up or were recruited to take surveys in exchange for money or other rewards. Opt-in panels have become extremely common in polling and market research because they make data collection fast, panelists provide their demographic data in advance, and research has shown that they are as reliable as telephone surveys.

App-based opt-in panel surveys have become very popular with pollsters in recent years. Pew Research has shown that this methodology provides results consistent with probability-based surveys over time, and YouGov uses this approach as its primary methodology. 

Bias

All survey methodologies have biases. The bias inherent in our app-based opt-in panels is that our panelists must own their own smartphone, have at least intermittent internet access, and download the Premise app. Fortunately, smartphone ownership and internet availability is growing rapidly in low and middle income countries. In 2019 the GSMA estimated that 65% of the world owned a smartphone and forecasts an increase to 80% by 2025. These numbers hold up well across low and middle income countries, and internet availability usually exceeds smartphone ownership.If smartphone ownership is particularly low in the country you want to survey, we suggest you consider mixed methodology or face-to-face surveys. 

Sampling

Premise offers three main sampling options for app-based opt-in panel surveys. In countries where our network is sufficiently large (e.g. Ukraine, Colombia, or the Philippines) stratified random samples are available and recommended when doing national surveys. In countries with smaller networks, quota samples with quotas for age, gender, and geography are most commonly used for national surveys. Lastly, convenience samples can be used when targeting niche demographic or geographic populations (e.g. mothers of children under five living in the Nairobi metro area.) Premise has an automated process for creating sample frames based on Worldpop modeled census data, but custom sample frames based on other census data are available as well. 

When To Use App-Based Opt-in Panel Surveys

App-based opt-in panel surveys in low and middle income countries are useful in several scenarios:

1. 1. When time or budget is an issue. There is no faster or cost-effective survey methodology than app-based panel surveys. 
   2. Countries with relatively high smartphone ownership. Nationally representative surveys can be conducted using our app-based opt-in panels in countries with relatively high smartphone ownership with the application of post-stratification weights. If the target population is narrower and smartphone ownership is lower, then one of the other methodologies will be more appropriate (see below).
   3. Recurring surveys. When making inferences from change over time, even a non-representative, homogenous sample is statistically valid. Premise’s panel is homogenous in two ways: all respondents opted in, and they’re all interested in generating income by offering their opinions and collecting data in their community.

For more information, check out our full PDF guide, How to Use Premise for App-Based Opt-In Panel Surveys. 

At Premise, we believe anyone should be able to provide insights and information about the communities in which they live. To make that a reality, we’ve established a method for Contributors to survey other community members. Our platform and policies enable us to replicate the best practices in training, sampling, and ethics from traditional data collection but without the significant overhead requirements encountered when using professional enumerators to conduct face-to-face surveys.

Qualification and training

Premise’s approach to qualifying and training Contributors leverages a unique advantage of our platform: each of our contributors has a history of data collection on the platform already. As such, Premise’s approach to training begins by offering our most prolific and trusted Contributors—those who have greater than 50 lifetime completed tasks and have a 70%+ trust score, derived from the percentage of tasks they complete that pass our data quality control modules—the opportunity to access interview tasks. Push notifications are sent to Contributors who pass this threshold informing them that they’re eligible to complete higher paying tasks by participating in a training series.

Premise staff then screen interested Contributors before enrolling them in a two-part training series. The first part of the training is standardized and applies to every face-to-face interview with Premise. The second is customized to our partner’s needs and usually involves topics such as how to perform a random walk within the context of the country in question, training on how to appropriately use local languages, and donor-specific compliance topics such as prevention of security exploitation and abuse. Contributors who pass the training then gain access to the interview tasks.

Sampling

Premise offers standardized sampling for nationally representative surveys and a variety of custom sampling options appropriate for both household and site surveys. To create standard national samples, we follow three steps:

1. 1. Generation of sampling frames for each of the administrative level one units of a country (the largest subnational administrative unit of a country, e.g. state or department) using their latest census data or modeled census data from WorldPop 
   2. Random selection of administrative level two units which are overlayed with population density maps on a grid using an internally-generated process
   3. Random selection of hexagonal polygons that are only from populated areas 

Our platform then automatically generates face-to-face survey tasks for each of the selected polygons and makes those tasks available to Premise Contributors living in or near those areas. Sampling for site surveys or surveys targeting specific towns and villages can be developed in partnership with Premise in-house statisticians. 

When To Use Face-to-Face Surveys

Face-to-face surveys in low- and middle-income countries are useful in several scenarios that are inadequately covered by opt-in panel surveys:

• • National baseline surveys. Most baseline or endline national surveys seek to make statistical inferences between highly granular geographic units. This includes activities like immunization coverage surveys, humanitarian multi-sectoral needs assessments, and baseline surveys for USAID Mission Performance Management Plans. Premise’s opt-in panel alone is usually not large enough to fulfill requirements for these, so face-to-face interview surveys can be used to bridge the gap. 
  • When you need to target specific geographies. If your program has a very narrow geographic focus—e.g. only has interventions in 15 villages—then face-to-face surveys are the best option because our panel won’t be large enough to adequately represent your target population. 
  • Site surveys. Surveys that must take place at a specific site or time— e.g. patient, caregiver, project beneficiary, or teacher interviews—can only be conducted using the face-to-face method. 

For more information, check out our full PDF guide, How to Use Premise for Enumerator Interviews.

Smartphone ownership and internet access in low- and middle-income countries is growing at an incredible rate. Yet ownership and access is not equal across all demographic and geographic groups. Mixed methodology surveys—in this case a mix of app-based opt-in panel surveys and face-to-face surveys—offer Premise partners a way to increase representation in the sample without the full time and cost requirements of face-to-face interviews. 

Sampling 

Mixed methodology surveys involve creating two samples, one for opt-in panel surveys and one for face-to-face interviews. The opt-in panel samples typically use quota sampling for age, gender, and geography. Premise then compares those quotas to the number of respondents in our panel and builds the face-to-face survey quotas to cover the projected gap. Sometimes the opt-in panel and face-to-face surveys are launched simultaneously. Otherwise, the face-to-face survey is launched after the panel surveys are completed so the actual delta is covered. Customers with time constraints typically launch their surveys concurrently while customers with more time to spare launch them sequentially. 

Enumerator selection

The face-to-face survey qualification and training features described above aren’t always necessary for mixed methodology surveys. This is because the face-to-face survey sample is typically used to cover age groups or genders that are underrepresented in our panel rather than geographies. We can use our existing demographic data on our Contributors to determine whether they live in a household that includes the underrepresented population (e.g. respondents aged 55+). Our partners can then publish face-to-face survey tasks just to those Contributors, now enumerators, who live in households with members of the underrepresented population. Since such face-to-face interviews do not involve performing a random walk or other standard enumerator duties, training is usually not necessary. They will be tasked with conducting the interview and taking a photo of the interviewee’s identification card to prove they are in the target population. 

When To Use Mixed-Methodology Surveys 

Mixed methodology surveys in low and middle income countries are useful in several scenarios:

• • When certain demographics are not well-represented in our panel. Older demographics, women, and certain geographies are sometimes underrepresented in our panel—even in countries with high smartphone ownership. If you need a nationally representative sample but are on a limited budget or timeline, a mixed methodology survey is a great way to improve representation without the time and expense of only face-to-face interviews.
  • Programs targeting youth or children. Premise does not allow people under 18 on our app, however, we do allow our Contributors to interview people under 18 in their household. If your program exclusively targets these groups or If your audience includes ages under 18 then mixed methodology surveys are a good option.

For more information, check out our full PDF guide, How to Use Premise for Mixed-Mode Surveys.