What Leads to Bad Quality in Research Panels? Part II

Ensuring the quality of research insights begins with the integrity of your data, which is largely dependent on the reliability of your panelists. Disregarding due diligence in the recruitment process and neglecting comprehensive checks can open the door to fraud and various respondent issues that jeopardize the accuracy of your findings.

In this three-part series, we delve into the leading causes contributing to poor quality in online research panels and how to avoid them. Part I highlighted the pitfalls of rushed recruitment.

In this installment, we turn our focus to the critical issue of disregarding due diligence to mitigate risks and enhance the credibility and reliability of your research insights.

Due Diligence in research

Tools and algorithms designed to assess and validate panelist data can greatly enhance efficiency and accuracy. However, an overreliance on these technological solutions and panelist trust can lead to significant shortcomings if not complemented by human oversight.

Here’s why…

Bad Panelist Behavior Is More Common than You Think

Dishonesty in relation to panelists’ expertise, experience, job title is unfortunately too common in online and survey research. Additionally, the lure of incentives often leads panelists to rush through surveys, submit poor-quality responses, or attempt to take the same survey multiple times.

Also, it is important to be aware that many B2B research panelists change jobs or move out of their industry.  This may cause their profile information to be out of date and, even if not intentionally misleading, this can still negatively impact your research.

To combat panelist misconduct and ensure high-quality data, consider implementing the following strategies:

• Design your survey with engaging and thoughtful questions to discourage hasty responses and improve the overall quality
• Offer moderate incentives and align them with survey completion milestones rather than participation alone can reduce the temptation to rush through or repeat surveys
• Employ comprehensive verification techniques, such as cross-referencing responses with other data points and regularly updating panelist profiles to reflect current job roles and industry status
• Incorporate quality control mechanisms like attention checks and response time analytics to identify and filter out low-quality submissions

How Quality Measures Enhance Data Integrity

Implementing effective quality measures and technology significantly improves your ability to identify and screen out problematic panelists. At IRB, we use a comprehensive, scientifically rigorous approach for recruiting, engaging, and managing panelists. During recruitment, our team validates each B2B panelists through their LinkedIn profile to ensure authenticity.

Additionally, our proprietary Know Your Respondents (KYR) technology leverages advanced data collection methods to provide real-time insights into respondent quality.

Limitations of Automated Systems

While automated systems and AI-driven tools offer powerful capabilities, they are not infallible. Algorithms can sometimes miss subtle inaccuracies or inconsistencies in respondent data that might be detected through manual review. For instance, automated systems may fail to detect nuanced discrepancies in job titles or changes in employment that could impact the relevance of the data provided. Additionally, technology may not always accurately assess the sincerity of respondent answers or detect cases of deliberate misinformation.

By integrating both technological and manual verification methods, researchers can better safeguard against inaccuracies and ensure the credibility of their data. Leverage advanced tools for initial screening and data validation, and be sure to also implement periodic manual checks and audits. This hybrid approach allows for a thorough evaluation of respondent information, enhancing the overall quality and reliability of the research data.

On a final note:  It’s essential to familiarize yourself with and follow industry guidelines such as the ESOMAR 37, in addition to privacy guidelines in the countries where there are panelists.

For more information on online research panels, contact IRB today!