Random Sampling for Online Surveys for Public Policy in Canada


Advanis has developed its own proprietary General Population Random Sample (GPRS) in Canada. We use a combination of random digit dialing (RDD) to both wireless and landline phone numbers and recruited individuals using an IVR-to-Web methodology. This method is probability-based; that is, every recruit has an equal and known chance of being invited to participate in the survey. The only way a potential respondent can be included in GPRS is through random selection; people can't "join" GPRS. So, unlike using traditional online panel sample, most of which is not randomly recruited, researchers can calculate the representativeness of the data collected with associated margins of error and can perform statistical testing. To-date, Advanis has recruited over 270,000 Canadians who are interested in completing surveys occasionally about public policy and not-for-profit topics.

 

The data quality is higher among those recruited from our probability web sample when compared to panelists from a global online panel company.

 

For example, in 2019, Advanis conducted a longitudinal study among Canadian citizens who were eligible to vote in the 2019 General Election using both an online panel and probability web sample recruited from Advanis’ GPRS sample. Over 52,000 responses were collected across the three waves using the GPRS panelists and over 37,000 responses were collected using the online panel.

It was clear that the data quality was higher among Advanis’ GPRS respondents than the online panelists due to a number of factors:

  • First, they spent longer completing the survey (from 1.3 to 3.9 minutes longer on average per survey wave) and fewer fell below the minimum time threshold established for each survey (1% vs. 5% for online panelists across each wave of the survey).
  • A greater proportion provided an answer to both open-ended (79.2% vs. 72.2%) and closed-end questions (96.3% vs. 94.6%) rather than selecting the “Don’t know” response.
  • After weighting for age, gender, region, and Indigenous status, the GPRS respondents more closely aligned to other Statistics Canada demographic metrics than the online panelists. 

Our GPRS sample is used exclusively for public policy topics. All of our surveys using this sample are for governments (municipal, provincial, federal) and non-profits. More information can be found at:  http://www.tellcityhall.ca/index.html or by contacting Gary Offenberger (gary_offenberger@advanis.net).