Both The Washington Post and Digital Trends have spotted instances of scam support numbers showing up in Google AI Overviews, reports of which appeared on Facebook and Reddit respectively. Credit unions and banks are also warning their customers about these scams.

[...]

It doesn't seem to be a completely new problem, but the way Google Search works now, it's been given a new twist.

Here's what happens: The unfortunate victim Googles a company name looking for a contact number, then calls the number thrown up by AI. This doesn't actually lead to the company in question, but rather to someone pretending to be that company, who then tries to take payment information or other sensitive details from the caller.

It's not clear exactly how these fake numbers are being planted, but the best guess is that they're being published in multiple low-profile places online, alongside the names of major companies. AI Overviews then comes along and scoops them up, without running the proper checks to verify the information.

[...]

Google says it's actively fighting these scammers and that it’s continuing to roll out updates that make its spam-detection systems stronger. “Our anti-spam protections are highly effective at keeping scams out of AI Overviews and showing official customer support numbers where possible,” the company said in a statement to WIRED.

Of course, it's not just happening on Google Search. Security researchers have shown how malicious text can be hidden in emails—and presumably documents as well—which is then scraped and summarized by the AI, and served up to the user who takes it as accurate and authentic. The issue is also showing up in other AI search engines.