“Quality Control is applicable to any kind of enterprise. In fact, it must be applied in every enterprise” – Kaoru Ishikawa
Last week, we dealt with modern Quality Control measures on the Client-side. This week, we will take a look at Quality Control from the Agency-side.
In the market research industry data quality is everything. Data quality needs to be a top priority; everything relies on it. Putting your energy into monitoring the quality of your data will help ensure the quality of your results, quality findings, and quality recommendations, thereby ensuring you meet the needs of your customers.
Quality Control (QC) simply put, refers to the efforts and procedures that survey researchers put in place to ensure the quality and accuracy of data being collected using the methodologies chosen for a study.
Ensuring quality control in the various categories of research has the main advantage of trust and goodwill between agency and the client. As a full-functional market research agency, ensure Quality standards are applied in the categories of Computer-Assisted Telephone Interviewing (CATI), Face to Face, Mystery Shopper and Qualitative methodologies of a project.
Computer-Assisted Telephone Interviewing can be complex. Unlike online interviewing, CATI requires staff trained to deal with the complexities of speaking directly with people over the phone and gaining their trust, all the while adhering to a rigorous methodology to ensure that questions are asked—and answered—correctly. As an Agency it is required you train field workers adequately and have some level of internal quality assurance practices to every project.
Initial training involving briefing and pre-screening for voice-quality and enunciation could span two days and followed by a test. Only those prospects receiving a passing grade become interviewers for the project. Project managers or executives should monitor all interviewers, coach them, and give them ongoing assessments. Also, randomly monitor and play recorded interviews and if possible allow remote client monitoring or make recordings available to client (with respondent consent, of course).
Warranting you have a zero-tolerance policy for falsification is key. The least detection of data falsification by a client can not only extinguish your business relationship with your client but black-list you completely in the business sector. Ensure respect for respondent is maintained throughout the duration of the interview, as lack of this may lead to incomplete interviews and destroy client’s reputation in the eyes their customers. Call back 10% of all interviews to perform validations for key questions in the questionnaire. If any inconsistencies exist between the answers, then those interviews must be replaced.
Quality standards in Face to Face interviews are quite straight-forward. They involve a three-step process: Mobilization, Implementation and Analysis. Mobilization typically involves having dedicated Senior Executives and Senior Field Staff that are IT Savvy to handle project. IT savvy is critical here because modern market research practices have moved away from the usual paper and pencil format to using tablets to record responses for surveys.
Conduct a Pilot Study to test survey instrument, methodology and devices prior to commencement of fieldwork. Perform rigorous role plays to ensure that all enumerators and interviewers are on top of the manual and electronic questionnaires. Encourage client Participation in Briefings/Pilots. This allows for any queries by the interviewing team to be answered.
Implementation involves Standard Quality Control, thus a 100% logic check and 20% back check on filled questionnaires. Submit Fieldwork reports and regular project updates to Client, ideally on a weekly basis. This process does not only offer the advantage of ensuring everything progresses as expected, but also highlight some key issues emanating from responses from ongoing fieldwork so client can make some strategic and timely decisions on their products or services. Install Trackers on respective interviewer tablets from QC Base Station to track interviewer locations, interviewing time and date and synching regimentation. If possible, grant client Access to platform information while in field so they could monitor fieldwork remotely.
The Analysis stage simply involves verification and cleaning of data synchronization, mostly to ease data entry, data outputs & visual tabulation checking, and extra analysis based on data findings mostly to add value to data collected.
When determining how to improve customer experience, Mystery Shopping (customer data collected undercover to improve customer experience) can be a powerful tool. Worthy of note here is the fact that Mystery Shopping is the most dynamic methodology of market research surveys. It is therefore imperative for a market research agency to be constantly abreast with modern Quality control measures for successful Mystery Shopper surveys. This involves Deployment, Execution and Analysis.
The Deployment stage involves dedicated Senior research executives & experienced senior field interviewers/staff who are tech-savvy to handle monitoring gadgets and the project. Camera enabled devices for video, audio and pictures can be tested for efficacy and usability. Pilot study to test survey instruments and methodology are very critical at this stage. Client participation in briefings and pilots are also necessary to ensure best practices are adhered to.
Regarding Execution, standard quality control here involves 100% logic check and editing of questionnaires (more like a fact sheet to document experience during an interaction with a sales person or customer care representative). Survey coordinators are to maintain close surveillance on both interviewers and supervisors, to ensure locations required are actually visited. This can be verified using evidence captured in pictures and videos. In the service industry such as telecoms, banking, insurance, health, etc mystery shopping can be carried out using monitoring devices such as cameras imbedded in pens, wrist watches, spectacles, baseball caps, etc. For the retail industry, apps such as the iAuditor mystery shopper app (or secret shopper app) can be used.
The Analysis stage of QC for mystery shopper involves verification of visits captured in pictures and videos, data outputs & visual tabulation checking as well as extra analysis based on data findings to add more value.
Qualitative market research is all about understanding people’s beliefs and point of views and what they feel about the situation and the deciding factors that influence their behavior. Quality Control measures for Qualitative studies mostly focus on the execution of the survey. As the project manager/senior research executive it is your primary responsibility to be involved in the full process from briefing, recruitment, moderation of the focus groups to analysis and reporting of findings. Ensure the right respondents/product users are recruited, be actively involved in re-screening respondents to weed out outliers. If possible, conduct one or two pilot groups prior to commencement to pinpoint problem areas, reduce respondent burden, determine whether or not respondents are interpreting questions correctly, and ensure that the question flow is not influencing the way respondents answer questions during the discussion.
In essence, practicing Research agencies should have their in-house Quality Control Manuals that describe their quality control systems in detail. These manuals should vary depending on the category of research (CATI, Face to Face, Online surveys, Mystery Shopper and Qualitative). Project executives must ensure content of these manuals are spelt-out, read and understood by both supervisors and interviewers alike, and not only in Proposals sent to clients for project sourcing. Adherence to ESOMAR, Market Research Society and other market research affiliations’ Code of Marketing Research Standards should be maintained always.
Send in your comments and Market Research topics of interest to [email protected] or Whatsapp to 0244537533
