AI agents are bringing an old banking problem into a new era

Banks are right to be cautious about giving AI agents access to customer accounts.

Giving an external AI agent permission to access someone's bank account is risky, especially if that agent can move money.

On the flip side, I also think it's only a matter of time before people expect AI agents to handle everyday financial tasks, like paying their bills. Banks will eventually have to support that saely.

Right now, a lot of the discussion is about how to identify AI agents, verify who's behind them, and control what they're allowed to do.

I agree that these are important problems to solve.

But there's another risk I'm much more concerned about. And banks are already struggling with it today.

The worst thing an AI agent can do with your bank account

Let’s say someone gives an AI agent permission to pay their monthly bils.

One day, the agent receives an email that appears to come from their internet provider. The email says the provider has changed its payment details and asks for future payments to go to a new account.

Except the email is from a scammer.

The agent follows the instructions and sends the money.

As far as the agent is concerned, it was doing exactly what you asked it to do. It had permission to pay your bills, and the request appeared legitimate.

Now consider what the bank processing that payment sees.

The customer is legitimate. The agent is authorized. The payment falls within its permissions.

Everything could look perfectly normal, even though the money is going to a scammer.

This is essentially the same challenge we face with authorized payment scams today.

A scammer convinces a legitimate customer to send money to a fraudulent account. The bank authenticates the customer and processes a payment that appears legitimate.

A legitimate, verified AI agent can have all the right permissions and still send a customer's money to a scammer.

That’s the bigger issue.

Know Your Agent won't solve this problem

Agent detection is an important part of managing fraud risk as AI agents become more common in banking. But it's not a silver bullet.

New agents will keep appearing. Fraudsters will find ways to misuse them, and banks will have to keep updating their controls.

And even if we get really good at verifying agents, we still haven’t solved the bigger problem.

The agent is legitimate. The customer authorized it to pay bills. The payment falls within its permissions.

But the money is still going to a scammer.

Banks needs to be able to recognize that risk, regardless of who's initiating the transaction.

That's where we should focus more of our attention.

We need to get better at understanding who receives the money

Banks generally have a lot more information about the person sending money than the person receiving it.

They can recognize the customer's device, authenticate their identity, and evaluate their account activity.

But with authorized payment scams, the person sending the money is often the legitimate account holder. All those checks can come back clean, and the bank can still end up processing a fraudulent payment.

I've argued before that we need to get much better at understanding the receiving side of these transactions.

One way to do that is to look for evidence of an established relationship between the sender and recipient.

This is something we've been working on at Incognia.

We've built a Physical Social Graph that uses historical location behavior and persistent device intelligence to identify evidence of relationships between people in the physical world.

For example, two people might have an established relationship that isn't obvious from their account or transaction history. But their historical location patterns may show that their devices have repeatedly visited the same precise places over time.

That can provide evidence of a relationship that wouldn't necessarily be visible through banking data alone.

And we've found that this information can be particularly useful for assessing scam risk.

In our analysis of more than 300 million transactions, we found that:

  • 79.2% of legitimate transactions showed evidence of a physical-world relationship between the sender and recipient
  • 86.2% of reported scams showed no observed physical-world relationship

These findings are important because they give banks another way to assess a payment when the sender appears legitimate.

Of course, people send money to new recipients for legitimate reasons all the time. The absence of an observed relationship doesn't mean the payment is fraudulent.

But it can be a useful risk signal when combined with other information.

For example, a bank might also see suspicious activity associated with the receiving account. Or it might detect that the sender is on an active phone call or has someone remotely accessing their device while making the payment.

By evaluating those signals together, banks can make a more informed decision about whether a payment needs additional scrutiny, even when the person initiating it is legitimate.

And that same approach can help assess payments initiated by authorized AI agents.

The fraud controls we build today should also protect payments made by AI agents

I wouldn't wait for AI agents to become widely adopted before addressing this problem.

Authorized payment scams are already difficult to detect.

As AI agents start making more payments on behalf of customers, I expect that challenge to become even harder to manage.

Whether a customer sends money directly or asks an AI agent to do it, banks still need to evaluate the transaction and understand where the money is going.

We'll need controls around agent identity and permissions. I agree with that.

But if I were running fraud at a bank today, I'd put more resources into improving our ability to detect suspicious payments, especially when the person or agent initiating them appears legitimate.

That's a problem worth investing in now.

And the better we get at detecting these scams today, the better prepared we'll be as AI agents start making more financial decisions on behalf of customers.

 


 

I've written more about how we're using physical-world relationships and other risk signals to help banks assess suspicious payments. Read about our approach to Real-Time Scam Prevention.

We're also looking at how banks can assess the risk of transactions initiated by AI agents, even when those agents are verified. Read more about Intent Intelligence for Banks.

Subscribe to The Signal: FinServ for more on fraud prevention and what's changing in financial services.

By clicking "Accept" or continuing to use this Website after this notice, you agree to our Terms of Use - including your rights, responsibilities, and how we handle disputes.