“What is research but a blind date with knowledge?” – Will Harvey
In the market research industry, data quality is everything. Data quality needs to be a top priority; everything relies on it. Putting your energy into monitoring the quality of your data will help ensure the quality of your results, quality findings, and quality recommendations; thereby ensuring you meet the needs of your customers.
Quality Control (QC), simply put, refers to the efforts and procedures survey researchers put in place to ensure the quality and accuracy of data being collected using the methodologies chosen for a study.
It is highly problematic for client-side researchers/research requesters to have a laid-back attitude and just wait to accept any research results from agencies as accurate and true. It is not easy to check the quality of a market research study. Whether the figures in a Report clearly depict results from the fieldwork or not, outright fraud may not be easily detected.
A few measures put in place would, however, guarantee that quality is assured in the full value chain of the research process. First and foremost is the choice of research agency to assist in delivery of the survey. It is always best practice to have more than three agencies bid for the project in question, and based on their respective proposals and key specialties choose the agency that best fits the purpose.
One key area to look out for is the agency’s Quality Control processes as specified in their proposal. Look out for how they intend to Deploy, Execute and Analyse the Quality Control processes for the study. In Deployment, ensure your project is handled by senior executives and experienced interviewers/supervisors, one or two pilot studies to test methodologies, survey instruments, length of survey etc., as well as your ability to participate in briefing and pilot processes.
In executing the QC process, 100% logic check/editing of questionnaires by the agency will help weed out inconsistencies; and if need be, interviews carried out again via Computer Assisted Telephone Interviewing (CATI) or otherwise. Also, Survey Coordinators to maintain close surveillance on both interviewers and supervisors, to ensure interviewers do not ‘sit under some trees’ or in their homes by themselves to fill out the questionnaires. Calling back respondents on selected questions via CATI should be carried out; this process can be done by both parties (client and agency), though separately.
Analysis in the Quality Control process include standard protocols such as Verification and cleaning of data synchronisation, data outputs and visual tabulation checking, as well as extra analysis based on data findings to add more value.
Another key ingredient to ensure Quality Control is to Agree with the Agency to have access to raw data from fieldwork. This has the advantage of you, the client, not only being able to reach out to respondents to seek clarification or inconsistencies you might have discovered, but also keep the agency on their toes bearing in mind that ‘big brother is watching’. By this, you get to have an in-house Data warehouse of some of your customers who come in handy anytime you need to reach out to them for in-house surveys.
For Online or Telephone surveys in particular, ensure you have access to recordings of at least 75% of the interviews. Listening in to some of these recordings ensures that interviewers are following the right methodologies, interview flow is smooth, choice of words in the questionnaire are simplified and explained to respondents, and help weed out or re-train some interviewers where necessary.
Access to these recordings helps re-design methodologies for subsequent projects if need be. Some of these recordings can be played out to various functions or departments of interest to listen to issues pertaining to them, from the ‘horse’s own mouth’. For instance, remarks made regarding advertising or the business’ digital channel/app can be played out for the Communications or Digital team to have first-hand opinions about the perception customers have about advertisements and digital channels/apps respectively.
In the line of Qualitative Research, look out for Quality Control measures agencies have spelt out in their proposals as processes to ensure quality. After recruitment is done by field workers, will there be call-backs to validate respondents before coming in to be screened for the groups? Will the agency submit to you a full list of participants on demand? How many members of your team (from client-side) will be allowed for observation?
Primarily, the use of an experienced Focus Group Moderator cannot be underestimated. Years of experience, background and industry experience are all great, but they don’t mean anything if the moderator cannot connect with the consumers in the room (or online) and uncover insights that matter to you, the client. Have an engagement session with the potential moderator on how s/he intends to get consumers to open up and get at what they are really feeling. How can they get consumers to say what they normally wouldn’t say, as well as respective Projective techniques to be adopted based on the objectives? Also, make sure to sit in and observe Focus Group proceedings; and have debriefing sessions with the moderator at the end of each day to summarise and synthesise key findings and address the project objectives.
It is always best practice to commission agencies with memberships to prestigious Market Research societies such as ESOMAR, Marketing Research Association (MRA), African Market Research Association (AMRA), Market Research Society (MRS) etc. to conduct external research surveys. These societies have laid-down principles and best practices for member agencies to follow in executing research surveys. These societies also require agencies to design research to the specification and quality agreed with the client, to ensure findings and interpretations are clearly and adequately supported by data, and ensure that the research is carried out in accordance with appropriate scientific research principles, methods and techniques.
Send in your comments and Market Research topics of interest to [email protected] or Whatsapp to 0244537533
