Review of continuous Customer Survey Program

The brief for the HMRC Customer Satisfaction audit

We were commissioned as part of an Agency team with IFF Research to audit HMRC’s continuous Customer Survey program, consisting of three stratified random probability surveys covering individual taxpayers, businesses, and tax advisors. The project covered the design and development of the survey before re-tendering the contract. We also had to learn from the previous survey waves and improve them to create customer insights that were more actionable and better-suited changes in HMRC strategy and service delivery.

The scope of the project

We were charged with a review of the Sampling methodology, including modular design and weighting of each survey. Our role encompassed:

• A full review of the Technical Documentation, Sources of Sample, Survey Data sets in SPSS and Design Effects.
• A face-to-face and follow-up discussions with the Statistical Lead of the incumbent agency.
• Production of a report reviewing the current methodology and making recommendations for change.

The steps involved

The role involved face-to-face meetings with the incumbent agency and on-going dialogue with our agency partners and the HMRC team.

The challenges we faced

• Establishing a detailed conversation with the incumbent agency’s statistical lead was key to the success of this project and helped us to fully understand the methodology employed, beyond the summary in the technical report.
• The politics of this assignment were complicated by the fact that the incumbent agency was a competitor of our partner agency, which meant we needed to approach the task sensitively.
• Establishing a conversation with the HMRC team, working with the partner agency, was also crucial. This involved asking challenging questions, questioning assumptions and asking the team to prioritise their requirements.
• We also conducted our own review of weighting and design effects in-house with detailed reference to the survey SPSS files. We were able to derive estimates of design effects under proposed alternative sampling plans.

The final outcome

Through our recommendations, we were able to offer an alternative design to:

• Simplify the survey methodology.
• Maintain the criteria for Statistical Confidence.
• Reduce the sample size, wastage and hence budget.
• Eliminate duplication and unnecessary processes, therefore simplifying survey logistics.

Our approach was to find the middle ground between accommodating sensible changes, reducing pressure on the design without compromising the trend data.