The Worst Thing an AI Agent Can Do With Your Bank Account

What’s the worst thing an AI agent can do with your bank account?

Send your money to someone who should never get it.

That’s where I think banks should start. Whether you’re using the account, your agent is helping you, or a scammer’s agent has broken in, the bank needs to look at who’s getting the money.

At Incognia, we built our Real-Time Scam Prevention solution to help banks catch payments to scammers before the money leaves. I believe this is the most important part of “intent intelligence” for AI agents: spotting when a payment may send your money to a scammer, even when someone has given permission to make it.

In an analysis of transactions from a bank with more than 80 million active users, we identified approximately 70% of scam transactions.

Imagine asking your agent to pay your bills. It gets an email that looks like it came from your internet provider: “We’ve changed banks. Please use these new account details.” The email is fake. Your agent believes it and pays.

You gave the agent permission. It followed your instructions. But your money went to a scammer.

Know Your Agent (KYA) is important. Banks need to know who controls an agent and limit what it can do. I also think KYA will be a game of whack-a-mole for a long time, as new agents and ways to misuse them keep appearing.

Knowing which agent made the payment wouldn’t have fixed this. Neither would checking that it had permission to pay your bills. The bank also needed to check the account receiving the money.

Banks already deal with this when people make payments themselves. A scammer convinces someone to send money from their own phone, using their own bank account. The bank recognizes the customer and sees them approve the transfer. Everything can look normal.

That’s why asking the customer to approve the payment again may not help. If they still believe the scammer, they’ll say yes again. An agent can fall for a fake request too, or follow an instruction from a customer who is being tricked.

Look at who’s getting the money

Incognia looks for clues about the person receiving a payment and their connections to other accounts. It also looks for signs that the sender and receiver may know each other in the real world.

We call this the Physical Social Graph. It uses the places devices have visited over time to help reveal those connections. Our indoor location technology is precise enough to tell apart neighboring apartments.

For example, several accounts might look unrelated to a bank. But if the phones using them keep showing up in the same apartment, that could be a clue that the accounts belong to one scam operation. Some may be “mule accounts,” used to collect and move stolen money.

The same history can help explain a payment that looks unusual. Two people may live far apart but regularly visit each other. Being in different cities doesn’t mean they’re strangers.

We found that nearly 79.2% of legitimate payments were between people with a real-world connection. By comparison, 86.2% of reported scams involved people with no such connection.

Of course, sharing a place doesn’t prove that someone is trustworthy. And people pay strangers for perfectly good reasons. We combine these clues with other warning signs, such as someone controlling the customer’s phone remotely or being on a call while making the payment.

The bank gets an assessment of the risk and the clues behind it. It can use these alongside its own checks to let the payment through, warn the customer, ask for more information, or stop it. These checks depend on having the relevant device and location history.

A suspicious receiving account stays suspicious whoever is sending the money. It could be you, your agent, or an agent working for a fraudster. The same evidence can help the bank spot the danger.

Banks still need to verify agents and set limits on what they can do. But we don’t need to wait for agents to become common to tackle the scam problem. The work banks do to stop these payments today will also help protect customers when agents make more of their payments.

If an agent is about to send your savings to a scammer, knowing which agent it is won’t be much comfort. You want your bank to stop the payment.