- The Cautionary
- Posts
- Learning AI #37
In January 2024, a mid-level finance employee in the Hong Kong office of Arup, the British engineering firm, received an email that appeared to come from the company's chief financial officer in London. It described a secret transaction that needed to be handled.
The employee, let’s call him Mr. Smith, was suspicious about the email and thought it was phishing. He was right and he ignored the email.
When the email didn't work, the scammers invited Smith to a video conference. Several people were on it. He recognized the CFO, and he recognized other colleagues. They looked and sounded right.
The people behind the scam did their preparation in public. According to Hong Kong police, they collected publicly available video and audio of the CFO and several other Arup staff, including footage from YouTube and recordings of past online conferences. The investigation found that they used deepfake technology to imitate the targets' voices.
While one deepfake might be easy to identify, many at once makes the task much harder.
They also targeted Mr. Smith, who had the authority to move money, but was located a half a world away from the CFO in London.
In a multi-person video conference, it turns out that everyone he saw was fake.
Police described the participants as prerecorded. The fake executives gave instructions, and nobody held a real conversation. He was asked to introduce himself, so the one live voice on the call belonged to Smith.
The design solved the scammers' hardest problem. A live deepfake that has to answer unexpected questions can slip. A scripted one only has to sit there and give orders, and in a meeting with senior executives, a junior finance employee doesn't usually interrupt.
Because the people on the call looked like the real people, Smith followed what he thought were his boss's instructions and transferred $25 million to five local bank accounts controlled by the scammers.
About a week after the group video call, Smith checked with the company’s head office. That was the first time anyone at the company heard about the transaction and it was immediately reported to the police.
The company confirmed "fake voices and images were used" and said its "financial stability and business operations were not affected and none of our internal systems were compromised." So far, no arrests have been made.
The employee did the right thing at the start. He doubted the email. The scammers answered his doubt with more fakes, including the elaborate group video call.
The check that worked was a call through a channel the scammers didn't control: contacting head office directly. That one call ended the fraud.
Things I think about
The fax machine is older than the telephone. Alexander Bain patented an early fax machine in 1843, 33 years before Alexander Graham Bell patented the telephone in 1876.