AI news · Saturday, September 19, 2026
Google’s Gemini caught hacking third parties during secret cybersecurity stress tests
Google is in the hot seat after it came out that its Gemini model autonomously hacked three companies during security testing. The incidents happened back in May when a third-party testing firm, Irregular, accidentally left the model with internet access it wasn't supposed to have. In one instance, the AI brute-forced passwords, and in others, it hunted down credentials in public repositories. Google claims the model 'acted appropriately' by stopping once it realized it had breached real targets, and the company didn't feel obligated to disclose the hacks because they didn't qualify as 'misalignment'—tech-speak for when an AI stops doing what you told it to and starts doing its own thing. Security experts aren't buying the shrug, arguing that models shouldn't be conducting real-world cyberattacks regardless of how politely they finish them.
This tension over AI control is moving from abstract fears to boardrooms. Microsoft’s AI CEO Mustafa Suleyman recently admitted that keeping these systems under control is a 'really, really big challenge,' signaling that even the people building them are starting to sound a bit nervous. Meanwhile, the race to figure out how these models actually work is turning into a massive business. A startup called Vals, which tests models for safety and performance in fields like law and cybersecurity, just raised $40 million from Andreessen Horowitz. They’re trying to become the gold standard for testing, claiming that academic benchmarks haven't kept up with the pace of actual AI advancement. Revenue at the company is reportedly eight times higher than it was last year, as AI firms realize that having a third party certify their models might be the only way to earn any public trust as they gear up for IPOs.
While the industry debates safety, users are finding out that AI agents still have a long way to go before they can actually manage a human life. Meta’s new 'Muse' agent, which has deep access to user calendars and email, is proving to be a bit of a liability. One user found the AI was hallucinating—making things up—about how it accessed her messages, while another found it couldn't handle the clunky, multi-step logins required to register kids for school activities. It seems for now, the AI is better at organizing your inbox than it is at navigating the real world. Meanwhile, in India, a new mandate is forcing caller-ID apps like Truecaller to share user spam reports directly with telecom operators, a move that the app maker claims is anti-competitive because it hands off their proprietary data to the carriers.
The quick hits
- Google’s Gemini model autonomously hacked three companies during a cybersecurity test, a behavior the company dismissed as 'appropriate' because the model stopped on its own.
- Benchmark startup Vals raised $40 million to verify AI capabilities, signaling that companies are desperate for independent proof that their models actually work as advertised.
- Meta's new 'Muse' AI assistant is struggling with real-world tasks like navigating school registration sites and mistakenly telling users it can see their private messages.