Implementing a Machine Learning Entity Resolution System for a Global Biotech Leader

Client

Global Biopharmaceutical Organization

Engagement Type

Machine Learning, Data Engineering, and Entity Resolution

Business Need

The client required a scalable and highly accurate method to match healthcare entities across multiple internal and third-party datasets. Existing processes relied heavily on manual review and legacy matching logic, which struggled to handle inconsistencies in naming conventions, formatting, and incomplete data.

The organization needed a solution that could reliably identify whether two records represented the same real-world healthcare entity – while significantly reducing manual effort and improving data quality across critical business systems.

• Matching entities across disparate datasets with inconsistent and unstructured data
• Handling variations in naming, abbreviations, and formatting for healthcare providers and organizations
• Integrating multiple identifiers (e.g., NPI, DEA) with varying levels of completeness and accuracy
• Converting unstructured text fields into structured inputs suitable for machine learning
• Reducing reliance on manual review without sacrificing match accuracy

Osprey designed and implemented a machine learning–based record linkage system to accurately match healthcare entities across datasets.

The solution leveraged supervised learning techniques to predict whether two records represent the same real-world entity. It combined advanced feature engineering, natural language processing (NLP), and domain-specific logic to significantly improve match quality.

Machine Learning Approach

Feature Engineering and Similarity Modeling
Osprey engineered a robust set of similarity features across key attributes, including: Entity names, Addresses, Medical specialties, Facility types, Unique identifiers such as NPI and DEA.

Advanced comparison techniques included:

• Token-level similarity scoring
• String distance and fuzzy matching algorithms
• Address component-level matching

Natural Language Processing (NLP)
To improve consistency and comparability across records, NLP techniques were applied to unstructured text fields:

• Tokenization and normalization of names and addresses
• Standardization of abbreviations and formatting
• String similarity analysis on normalized text

This enabled the system to detect equivalent entities despite variations in spelling, structure, and data entry practices.

Model Development
A supervised machine learning model was trained using labeled match and non-match outcomes derived from the client’s legacy matching process.

Unstructured data fields were transformed into structured similarity scores, allowing the model to make accurate, data-driven match predictions.

• 90% reduction in cases requiring manual review compared to the legacy process
• Significant improvement in entity matching accuracy across internal and external datasets
• Scalable, automated record linkage system supporting ongoing data integration efforts
• Enhanced data quality and consistency across downstream analytics and operational systems