GT was founded in 2019 by a former Apple, Nest, and Google executive. GT’s mission is to connect the world’s best talent with product careers offered by high-growth companies in the UK, USA, Canada, Germany, and the Netherlands. On behalf of the client, GT is looking for a Senior ML Engineer / Applied ML Engineer who is interested in solving complex matching and search problems at scale, applying ML and algorithms to hundreds of millions of real-world data records.
🏢 About the Client
Our client is a leading global management consultancy known for tackling some of the world’s most complex business challenges. With a focus on strategy, transformation, and performance improvement, the firm partners with major organizations across industries to drive lasting impact. Recognized consistently as a top workplace, it combines deep industry expertise with a collaborative, innovative culture. Its centralized European hub plays a key role in supporting operations across the EMEA region, ensuring excellence and efficiency at scale.
🎯 The Role
We are looking for a Senior ML Engineer / Applied ML Engineer to build and evolve the client’s internal entity resolution system — a core part of the data platform that uses machine learning, LLMs, and search and matching algorithms to turn complex, noisy data into trusted, unified entities. The system operates at significant scale, processing hundreds of millions of records, and the role will focus on developing and improving the ML models, matching logic, algorithms, and service capabilities behind it. This is a hands-on engineering role requiring strong Python and SQL, practical ML engineering experience, experience with large-scale data pipelines, and strong algorithmic and problem-solving skills. This is not a traditional Data Engineering role. The main focus is on building intelligent, production-ready ML systems and solving complex matching and algorithmic problems, with Data Engineering technologies such as Spark/PySpark used to support scalability and productionization.
✅ Key Responsibilities
Entity Matching & Distributed Algorithms
Design and improve entity matching, clustering, and deduplication algorithms at scale
Implement distributed matching approaches such as blocking strategies, multi-pass matching pipelines, nearest-neighbor and similarity-based methods
Apply and combine rule-based, statistical, and ML-assisted techniques (including embeddings where relevant)
Optimize candidate generation and scoring to balance accuracy, recall, performance, and cost
Translate algorithmic ideas into scalable implementations using Spark and SQL transformations
Continuously experiment with different approaches and iterate based on performance metrics and results
Large-Scale Analytic Engineering
Design, build, and operate a large-scale analytic system processing hundreds of millions to billions of records
Implement and optimize Apache Spark pipelines for entity matching, deduplication, and clustering
Build and maintain complex workflows using Airflow and DBT
Ensure pipelines are fault-tolerant, observable, and cost-efficient in distributed environments
SQL & Data Modelling
Develop and maintain analytical SQL models using Snowflake or a similar cloud data warehouse
Optimize large joins, aggregations, and window functions over very large datasets
Design data models that support both matching pipelines and downstream consumers
Data Quality, Validation & Iteration
Build validation logic and metrics to measure match rate, precision, recall, and accuracy
Support continuous improvements to the matching engine through iterative releases
Debug and resolve data quality issues across heterogeneous and imperfect data sources
📌 Required Qualifications
Core Requirements:
5+ years of software development using Python, ideally in a team lead capacity
Solid understanding of distributed systems and algorithms (partitioning, shuffles, joins, scalability trade-offs)
Experience building & working with complex data pipelines or data systems
Strong SQL skills, ideally with Snowflake or similar analytical databases
Strong hands-on experience with Apache Spark – Databricks (PySpark or Scala) in production
Experience with AI/ML-assisted systems (embeddings, inference, re-ranking)
Matching, Search & Similarity (Required or Willingness to Learn)
Experience with, or strong interest in, fuzzy and semantic matching techniques, such as Levenshtein / edit distance, Token-based similarity, BM25 or other lexical ranking methods, Vector embeddings and cosine similarity, Approximate nearest-neighbor or vector search concepts
Strong willingness to learn and apply advanced semantic matching techniques if not already experienced
⭐ Desirable Experience
Experience with data orchestration tools such as Airflow (or equivalents)
Experience building entity resolution, deduplication, or record linkage systems
Familiarity with search or retrieval systems (e.g., Elasticsearch, OpenSearch, vector databases)
Experience operating data pipelines at very large scale (100M+ records)
Background in data quality frameworks, validation automation, or QA at scale
Experience with Kubernetes or Containerized Functions (e.g., Azure Container Apps, AWS Fargate)
