Reducing manual review through automated data cleanup in the Tuberculosis Screening Programme
Publicly sourced
The challenge
The UK's Tuberculosis Screening Programme invites new migrants for testing based on their country of birth, using data from GP registration records. However, these records often include incomplete, misspelled, or ambiguous place names, resulting in messy data that requires extensive manual review. On average, around 40,000 records must be reviewed each year to ensure accurate screening.
The solution
The team used AI to automatically match inconsistent place name data to countries, significantly reducing the need for manual checks. The solution is now fully integrated into the Tuberculosis Screening Programme team's regular workflow.
NOTE: An Algorithmic Transparency Recording Standard (ATRS) record is currently being prepared.
The results
Achieved 90% accuracy in correctly identifying country of birth from inconsistent data.
Reduced manual review workload by 85%, from 40,000 to 6,000 records per year.
Details
Organisation name
Department of Health and Social Care (DHSC) / UK Health Security Agency (UKHSA)
Data Disambiguation Assistant
Reducing manual review through automated data cleanup in the Tuberculosis Screening Programme
The challenge
The UK's Tuberculosis Screening Programme invites new migrants for testing based on their country of birth, using data from GP registration records. However, these records often include incomplete, misspelled, or ambiguous place names, resulting in messy data that requires extensive manual review. On average, around 40,000 records must be reviewed each year to ensure accurate screening.
The solution
The team used AI to automatically match inconsistent place name data to countries, significantly reducing the need for manual checks. The solution is now fully integrated into the Tuberculosis Screening Programme team's regular workflow.
NOTE: An Algorithmic Transparency Recording Standard (ATRS) record is currently being prepared.
The results
Details
Links