TL;DR:

• ESRA 2023 conference took place in Milan from the 17th to the 21st of July after a 2-year wait since the last edition: It had a rich program throughout all the days with seven parallel sessions in each time block panel discussions, keynote speaker, and well-organized coffee breaks.
• Lifepanel made its debut at its first social research industry event by exhibiting and presenting two research papers
• Enforcing a push-to-web approach for all respondents represents an imminent obstacle researchers need to solve. It encourages us to move beyond the assumption that web-based surveys are a one-size-fits-all solution.
• Mode Effects in Probability Panels: Moving from CATI probability projects to probability panels introduces not encountered mode effects, which vary in magnitude based on the country.

Earlier this month, Milan hosted the long-awaited European Survey Research Association (ESRA) 2023 conference, where survey researchers from academia and the private sector met to network and present their most recent research findings. The event took place at the University of Bicocca in a week where temperatures soared to 40 degrees Celsius and multiple heavy thunderstorms surprised both attendees and locals. 

Despite these weather extremes, the conference kept all participants indoors with seven parallel sessions in each time block and well-organised coffee breaks, which provided sufficient time for exhibitors and attendees to meet, mingle and promote their novelties, work, and research papers. 

The ESRA conference acted as a bridge between academia and commercial enterprises, offering a platform for sharing research findings and methodologies. This event showcased the fruitful dialogue that arises when these two worlds with sometimes clashing views come together.

Lifepanel was also among the many exhibiting organizations at the conference – making its first live debut at a social research symposium. Additionally, Lifepanel also made it on the official session program by presenting its two research papers:

• “Where are my completes? Leveraging different response modes in a German online probability panel” by Nadica in “Using Internet Surveys in Cross-National Research”
• “Comparing effectiveness in panel recruitment via piggyback on existing surveys versus direct recruitment in building an online European Probability panel” by Nikola in the sessions for “Cross-national probability-based web or mixed mode panels and Using Internet Surveys in Cross-National Research”

Unfortunately, both presentations shared the same time block on the last day of the event, at completely opposite classrooms, thus leaving both the Lifepanel team and interested attendees divided and not able to indulge in both presentations. 

Regardless of this drawback, with this article, we hope to share the key figures and conclusions of the two presentations to the wider public, with hopes that it may pave the way to more developments and novelty in the field of Online Probability Panel Surveying in Europe. 

Key Takeaways From Lifepanel’s ESRA presentations:

1. Where are my completes? Leveraging different response modes in a German online probability panel:

In this paper, we delve into the nuances of response modes by examining data from a push-to-web study conducted through Lifepanel across Germany in addition to smaller countries such as Malta, and North Macedonia.
The analysis focuses on response patterns across different contact modes used in the panel: SMS, Email, and WhatsApp in each country.
We also explored how these response trends vary based on key demographics like age groups and education levels.
Most notably, our findings reveal that non-conventional online contact methods like WhatsApp demonstrate significantly higher completion rates among younger demographics (16-29) in contrast to the age groups of 50-64 and 65+ which yielded higher completion rates for SMS and/or  Email. While on the other hand, reaching less-educated populations was more effective through SMS push to web. 

Interestingly, individuals aged 30-49 and those with higher education levels showed comparable response rates for both SMS and Email.

Additionally, we see differences across countries, mainly in Malta for the 65+ age group the highest number of completes were achieved when the survey invitation was sent via Email, as opposed to North Macedonia and Germany for which the most successful method was SMS for the same age range, as well as in this country for the lesser educated WhatsApp had almost double the responses when compared to the two. For North Macedonia, in the 16-29 age group, SMS is the leading response mode, whereas for Germany and Malta is the WhatsApp push message, which is expected as WhatsApp usage in N. Macedonia is half of that in the other two countries, as well as Email responses, seem to be on the lower end for the higher educated population.

2. Comparing effectiveness in panel recruitment via piggyback on existing surveys versus direct recruitment in building an online European Probability panel

Table 1: Setup of both strategies for recruitment of panel members in Germany

Between late December 2022 and February 2023, Lifepanel engaged in an extensive probability panel recruitment experiment. The recruitment strategy encompassed various campaigns employing RDD sampling and CATI contacting methods. Leveraging the approach of piggyback recontacting from a prior National Representative survey, as well as direct recruitment via telephone, for this research we measured the panel representation of both strategies and their productivity. Our analysis unveiled the following insights: the direct recruitment method demonstrated a commendable alignment with demographic distributions. However, it’s noteworthy that this approach incurred significantly higher costs in comparison to the more cost-effective piggyback recontacting strategy.

Additional Takeaways on Online Probability Panels from ESRA 2023 Presentations:

The post highlights the complexity of enforcing a push-to-web approach for all respondents. This strategy might not always be the most cost-effective or efficient in the short term. This notion encourages us to move beyond the assumption that web-based surveys are a one-size-fits-all solution.

Tailoring Probability Panels:

The challenge of establishing a uniform approach for probability panels in Europe becomes evident. Ensuring quality and coverage standards while adapting to specific contexts is a necessity. The conference discussions emphasized the need for panel building and management to be adaptable to diverse requirements. This is showcased in the presentation “KnowledgePanel Europe – A comparison of recruitment strategies and challenges” by Femke De Keulenaer from Ipsos European Public Affairs in their European Probability Panel developments at Knowledge Panel.

Mode Effects in Probability Panels:

Transitioning from CATI (Computer-Assisted Telephone Interviewing) probability projects to probability panels introduces not encountered mode effects, which vary in magnitude based on the country. Patrick Moynihan’s presentation “Consequences of Moving from RDD to Web-based Probability Panels” from the PEW Research Center highlighted data from Kantar Public’s Probability panel, showcasing these mode effects. This indicates that the journey toward probability panels requires understanding and managing these potential effects.

Looking Ahead:

We are in anticipation of the future of European probability panels. With the promise of reconnecting with the same stakeholders in Utrecht for ESRA 2025, the survey research community eagerly awaits the developments, innovations, and insights that will shape the landscape of probability panels over the next two years.

In summary, the ESRA 2023 conference in Milan served as a nexus where academia and industry converged to explore the intricacies of survey research methodologies. More and more focus has been provided to the developments and adaptation of Online and Mix mode Probability Panels in this edition – signaling to thought leaders that this may very well be where survey research is headed as telephone and face-to-face response rates continue to decline and are becoming more costlier for researchers to conduct their studies.

The findings shared from our presented papers provide a glimpse into the multifaceted considerations and discussions that unfolded with affected parties that range from clients, competitors, and suppliers, during this event. 

As Online Probability Panels are still evolving in the field of survey research, the collaboration between diverse probability panel providers and stakeholders promises to yield innovative solutions and a deeper understanding of the dynamics at play.