FCA selects the Stats People to audit the methodological design for the Financial Lives Survey Wave 1
The brief for the Financial Lives Survey Wave 1 methodological design audit
We were commissioned to undertake a full review of wave 1 of the Financial Lives Survey for the FCA to audit the existing methodological design and technical reporting. The survey is one of the largest scale consumer surveys in the UK and is designed to collect a range of indicators relevant to the financial services and pensions industry. We undertook the review in parallel with the retendering process to ensure that learnings from the first wave were considered when designing wave 2.
Our approach
The review covered issues such as sources of remaining non-response bias after weighting, design effects of clustering and weighting, the rationale for the complex weighting, survey mode effects, survey modularisation, and technical reporting.
The process involved:
- Reviews of survey documentation and data sets.
- Detailed discussions with the project team and with the incumbent survey agency.
- Undertaking additional analysis on the survey data.
The challenges we faced
Due to the politically sensitive nature of the role, it was important for us to build trusting relationships with the incumbent agency. It was not always possible to obtain the full supporting information we needed to validate survey design decisions and so one of the challenges was to get the most out of the information that was made available.
The outcome
The results were presented to the FCA and the project team of the new agency appointed to manage the second wave of research.
Challenges
To get the most out of the information that was made available.
Outcome
The results were presented to the FCA and the project team of the new agency appointed to manage the second wave of research.