Identity Matching: Getting Through the Numbers While Keeping the Hits Right


A huge challenge for organisations is the sheer quantity of data that needs to be processed now. Plus, it needs to be processed quickly and accurately. Accuracy is critical as organisations don’t want the situation whereby hits are missed that should have been picked up or lots of irrelevant hits are triggered.

The volume of data that has to be sifted through has grown exponentially in the past ten years and will keep on growing. Back in the nineties when electronic identity matching first started, there were a few hundred names on sanctions lists. Now those numbers have stretched to millions of listed names with millions of counterparties. Organisations have to scan full client databases that potentially have tens of millions of counterparties with Political Exposed Persons lists that encompass anything from 1 to 3 million names.

This is no mean feat and requires IT systems with considerable processing power. It also requires organisations to have the right technology in place to ensure the right search/hit ratio.

To help organisations manage this volume of data, specialised search techniques have been developed, coupled with parallel processing and efficient false positive hit reduction mechanisms. What’s a false positive hit reduction mechanism? It’s the ability to limit the number of incorrect alerts that are raised.

In order to filter data effectively and quickly, organisations need to combine the advanced search methods with hit reduction mechanisms, right through the process. That produces the best, most accurate results. It enables organisations to much more effectively control their hit rate so that they don’t miss an important hit and are not overwhelmed with false positives. That is the holy grail of identity matching.

Organisations can choose from generic hit reduction mechanisms, such as whitelist application or score-based categorization. Or they can opt for more advanced techniques that employ optimized processes and additional criteria.

What’s next? We will look at those different approaches in next week’s post, highlighting the various options, what they mean and what they deliver.

Also in our series on identity matching

  1. What is name screening and why do we need it?
  2. What’s in a name?