AI fraud detection with ML & NLP: How Ravelin does it

Ravelin is proud to call itself an AI-native fraud prevention company. We’re data-focused, employ strong engineering practices, and machine learning has been at the heart of our offering ever since we were founded, in 2014.

Today, we’re going to take a deeper dive into how we handle AI fraud detection at Ravelin – including how we optimize working with clients, bespoke client models, consortium features and more.

Some of what we’ll examine is more on the technical side, but we will also look at the bigger picture, and the philosophy behind our approach.

Setting the scene

There are several reasons why Ravelin prioritizes custom machine learning models to tackle fraud, as well as making use of additional AI fraud detection techniques:

• Everyone’s fraud is different. Every company is different, and so is their fraud landscape. Even within the same vertical, attack vectors differ, and so do company policies, risk tolerance, and even business goals. So, even though we take our sector-specific expertise and experience into account, we build a model for each of our customers – and continue improving them. We will never provide a cookie-cutter solution.
  
  

• A customer-centric approach is best. We’ve chosen to focus on customers rather than individual transactions, to empower our merchants’ decision making and deliver more nuance and accuracy, also making use of link analysis to catch more fraud. This in turn allows our clients to understand risk vs reward better, estimate the lifetime value of their customers, get in-depth information to drive manual reviews, or even adjust their policies based on data and KPIs.
  
  

• Human expertise is still valuable. Ravelin’s machine learning recommendations are fully transparent and explainable – clearbox or whitebox, in other words. This means that a human analyst can look at any of the decisions and know which specific features, as well as types of features, led to them. Moreover, Ravelin’s models learn from manual overriding and manual reviews, improving in the future according to each company’s preferences.
  
  

• Companies want to grow securely. The goal of fraud teams is to drive growth, not merely to eliminate fraud. Our goal is to enable them to do so, fuelling the growth with a wealth of data and ML features that allow them to make more confident decisions in accepting payments and other customer actions.

What makes Ravelin AI-native?

We’ve had machine learning at our core since 2014, when we started our journey. Ravelin also uses LLM-powered natural language processing (NLP) for rule-setting, as well as NLP to make our models better understand fraud signals.

1. Ravelin’s solutions make use of ML models trained on clients’ own data and risk landscape, allowing you to scale and grow with consistency and confidence. And they grow along with our clients, constantly learning and improving.

2. Using LLM technology and working alongside the ML model to empower fraud analysts, Ravelin’s AI fraud rule generator understands requests in any language to save time and effort, generating complex fraud rules from a simple text prompt.

3. We’ve deployed natural language processing (NLP) in our machine learning models, to allow them to better understand non-numerical signals such as text within context. As a result, they can catch more fraud with greater precision.

Unquestionably, the biggest area of focus for Ravelin is the first of the above points, but we continue to look for ways to leverage the latest technology to improve our products.

How is AI used in fraud detection?

Compared to rules-based approaches, artificial intelligence in fraud detection allows operations to scale, freeing up fraud analyst time and allowing for improved results. Fraud rings are more easily detected, as are fraudsters employing various spoofing techniques to appear more legitimate.

In 2024, AI may be a buzzword across sectors and industries – but in fraud, machine learning is already well-established as a key technology. And it’s important to explain and showcase its capabilities in more detail, including by going through use cases.

For Ravelin, supervised machine learning is integral to a successful overall fraud strategy.

The data scientists from the Payments Fraud team use historic labeled data to uncover patterns that indicate fraud or abuse, in order to be able to extrapolate into the future. The labels come from various signals, such as cardholder disputes, subscription terminations, and human analysts conducting manual review.

This training data is used by the algorithm to point to the differences between genuine customers and fraudsters, then used to build the ML model.

Once the model has been set up, it can work with real-time transactions:

• Every time someone makes a purchase, the company makes a risk decision via machine learning. The transaction can either be approved or blocked

  • If it’s blocked, the shopper’s journey ends.

  • If it’s approved, the transaction continues the payment flow and is completed.
    

So, what is happening under the hood?

Using information from a huge number of transactions that have happened in the past, the machine learning model can assess whether the current behavior is indicative of fraud, and return a recommendation.

For example, it will consider questions such as:

• Has this user attempted this action before? How many times?

• How many different customers have used the exact same card?

• Does this shopper’s behavior diverge from the behavior of known good customers?

The ML model will also take into account some contextual information.

For example, whether the IP address matches the issuer country of the card, or what type of payment it is looking at – trial pre-authorization or actual payment.

Ravelin’s ML for fraud prevention pipeline at a glance

The machine learning models used are built by Ravelin in five steps:

1. Data extraction: Historic data, including risk decisions, transactions and labels, is extracted from previous transactions – outlining how the client’s fraud, as well as this client’s genuine customers, have looked like and behaved in the past.

2. Feature calculation: All possible ML features are calculated on historic data. We’ll look at features in more detail in the next section.

3. Selection: After considering different features and hyperparameters a model should use, the selection is made as to which ones are best fit for purpose and lead to the best possible results.

4. Training: The selected model is trained on the complete data set.

5. Evaluation: Does the model work well? The resulting predictions are evaluated by human analysts, taking into account the most recent fraud trends, policies and data.

At any point, a new model may be in development, with different or additional features and different or additional training data and labels.

It will then compete with the currently deployed model on efficacy, and the most useful model will be selected for deployment. We also calibrate models when we deploy them, so that our clients generally should not need to change thresholds or rules when a new model is introduced.

What are ML features in fraud prevention?

When working to prevent card-not-present (CNP) fraud and other online merchant pain points, a ML feature refers to a measurable characteristic of a user action. Features are fed into the machine learning model in order to allow it to make predictions.

For each model, there are hundreds of features taken into consideration when assessing how likely a customer is to be a fraudster.

Generally speaking, historic training data for the machine learning model is split into these types:

• Payment-related data – for example BIN, card type and issuer country

• Device and IP data – such as IP country, device type, the presence

• User data – this can be account creation date, user name, etc.

• Consortium data – there are 9+ billion identity elements in Ravelin’s databases, pulled when relevant to help decision making without causing false positives

To build a feature, you turn these initial fields into a calculation.

Note that not all features are going to be useful, or as useful as others. It is important to select those features that are relevant to one’s specific fraud landscape and fraud appetite.

These features are then mapped into wider feature families – megafamilies – with each family made up of a dozen or more features. Examples include:

• Order SKUs

• Account age

• Distance between orders

• Number of connected devices

• Network age and growth

• Order velocity

• etc

Ravelin’s data scientists work closely with our clients to better understand their priorities, goals and pain points.

However, it’s the models themselves that determines the importance of the various available features and megafamlies for each client, based on historical and current data.

Decision trees, forests and gradient boosted trees

For those hoping to go deeper into how Ravelin’s anti-fraud AI provides decisioning, we’ll now look at the technical details of the machine learning models.

The vast majority of models we use at Ravelin are tree-based. We use gradient-boosted tree models for most of our products, including our Payment Fraud prevention, Account Takeover solution, and Refund Abuse products. However, we use anomaly detection isolation forests for our Supplier Fraud Prevention product.

Simply put, a decision tree model asks certain questions at certain times in order to come up with one of several decisions. For example, when Ravelin’s ML sees a customer and tries to determine if they are genuine, it may ask “Is this a new customer?”. If they are not new, it may ask, “Have they logged on from this device before?”. If they are new, we might ask, “Does their shipping address match their registered card address?” and so on. The outcomes may then be, according to the answers to the above, that this person is fraudulent or genuine.

Combining the results of many decision trees created using slightly different datasets tends to give much better performance than a single decision tree. A single decision tree may be good at identifying fraudsters in the data it was created with, but it may not perform well on new data.

Gradient-boosted tree models are built differently to what’s known in data science as a random forest. Each decision tree in a random forest is built independently, while gradient boosted trees are built sequentially, with each new decision tree attempting to correct the errors and catch the faults of the previous one.

After each tree has been built, we use it to make predictions, calculate the residuals and then use these residuals to build the next tree. In other words, we’re looking to learn from any patterns in the errors of each tree.

The final recommendation that the fraud model comes out with is essentially a combination of the predictions of each sequential decision tree. At Ravelin, it’s a score from 0 to 100 – the highest it is, the more likely the customer is to be fraudulent.

How link analysis works with machine learning at Ravelin

To make better use of link analysis and harness the full power of Ravelin’s graph network, Connect, we feed the characteristics of the network that a shopper is part of into the ML model.

In short, the graph network analyzes relationships between different users, who might share characteristics such as their device or payment card, providing a visual representation of what they have in common – if anything.

For example, how many individual cards, devices or addresses appear in the user’s network? The network’s size and growth is another useful indicator of potential fraud.

It thus becomes possible to not just spot one device that has logged into a few dozen accounts, or used a suspicious number of cards, but also to quickly digest and communicate this information, and make decisions not just about that one device but various data points that are linked to it.

When the graph network link analysis combines with machine learning, we can create complex features from characteristics of networks, and then add them to the ML model to increase the quality of predictions.

Note: Ravelin’s graph network can also be used as standalone, to discover networks and block abusers and fraudsters using rules or human decisioning. This is in addition to using it predictively in our ML models. Learn more in our deeper look at Ravelin’s Connect feature.

AI fraud detection challenges to overcome – and how

Some of the challenges of using machine learning to prevent fraud include:

• Unbalanced data: Most of any company’s historic data isn’t fraudulent. Only small proportions of the data can help show the ML algorithm what is fraud, and we have to extrapolate from that onto the general population.
  
  

• Real-time: Decisioning takes place in real time, and we have a very short window to make these decisions – otherwise, the customer would face delays and is more likely to churn.
  
  

• Time-lagged labels: Hindsight may be 20/20 but it takes a while to find out what was fraud and what wasn’t. Chargeback requests come in later, but also other signals are revealed later.

In order to overcome such challenges often associated with ML, Ravelin’s solutions also allow for the use of additional, human-set rules. These can be deployed right away to stop any new patterns as they occur, before a new model is trained to pick up the newly fraudulent behavior.

We’ve highlighted before on the Ravelin blog that rules are still useful in ML-first fraud prevention, and this is an example.

Fraud scores and thresholds – what do these recommendations mean?

We’re confident in the power of our models, but ought to point out that the recommendations that the ML model returns are simply that: a non-binding fraud score, which needs to be translated into action. For that, we have thresholds.

We know that no two companies are the same – and this extends to their risk appetite.

In addition to being transparent, we want to enable our partners to enjoy control over their fraud strategy and shape it according to their objectives and circumstances.

By allowing you to change model thresholds, Ravelin’s platform gives you control over the actions taken based on the fraud score. You can also set different thresholds by market, and for each type of fraud.

Conveniently, we will also automatically calculate how changing your review and prevent thresholds would affect your customers. You’ll see:

• How many of your customers would have been allowed, reviewed and prevented in the past week, if those new thresholds were already active

• How many disputes would have been prevented under the new threshold

This serves to give you a clearer idea of the consequences, were those thresholds to change. In effect, you get to optimize your fraud strategy. You’re deciding how much risk to take on and what proportion of customers should face friction.

For example, one company may have a very high fraud tolerance, only blocking/preventing customers whose fraud score is over 80, and perhaps sending to manual review those between 70 and 80.

Those with a lower tolerance for fraud might decide to set a low Review threshold, sending everyone over 40 to manual review and over 80 to be prevented.

It is up to each individual company and fraud team to consider what works best for their risk strategy and overall goals. Ravelin’s dedicated support teams are always at hand to advise, however.

In summary

Artificial intelligence fraud detection and especially machine learning is incredibly powerful in the risk mitigation space.

But it also presents some interesting challenges, which can throw a spanner in the works of those teams not being thorough in their strategy. Fraudsters constantly innovate, and we as an industry need to stay well ahead of them.

Importantly, machine learning is a scalable, dynamic and effective system of fraud detection for firms with large volumes of payments.

Ravelin builds AI-native fraud solutions, testament to our belief in the potential and power of machine learning, while the Connect Graph Network boosts protection, and fraud rules help where ML might fall short on occasion.

For a more detailed breakdown of how Ravelin’s AI fraud detection prevention solutions can power your secure growth, book a call with our team today.

Related resources

• On-demand video: How Spotify uses machine learning to prevent fraud

• Machine learning and fraud prediction: What data is required to make it work?

• On-demand webinar: Demystifying machine learning

• How to showcase the business impact of machine learning for fraud

• On-demand video: Why a bespoke model matters

• Ravelin Insights: Machine learning

• Ravelin Insights: Online payment fraud