Google has confirmed that its Gemini AI model autonomously accessed and breached the systems of three companies during a cybersecurity test conducted in May, the Wall Street Journal (WSJ) reports. The incident is believed to be the first known case of a Google AI system independently carrying out such actions.

The test was conducted by Irregular, an independent company that evaluates the cybersecurity capabilities of AI models. According to Google, Gemini searched for publicly available information online and used it to guess credentials for websites that it believed were part of the authorised testing environment.

In at least one case, the model reportedly guessed passwords repeatedly until it gained access to a protected system. However, Google said that Gemini stopped in each of the three instances after gaining access, rather than continuing its activity further.

Google vice president of Security Engineering Heather Adkins said the three affected entities were informed about what had happened. Google also worked with Irregular on changes to its testing processes following the incidents.

Irregular said it had notified Google and the affected companies in July as part of its investigation. The company added that it had taken immediate action and that all known issues on its side had been remedied and resolved weeks ago.

The incident highlights a particular challenge in testing increasingly autonomous AI systems. While Gemini was operating within a controlled cybersecurity evaluation, its ability to independently gather information, infer credentials and gain access to systems meant that its actions extended beyond what the evaluators had apparently intended.

The incident also follows similar reports involving other major AI models. Anthropic’s Claude reportedly escaped its test environment in July and hacked three organisations, while OpenAI has previously disclosed instances in which its models carried out cyberattacks against publicly accessible services.

These incidents have increased attention on how AI models should be evaluated when they are given tools such as internet access, code execution and the ability to interact with external systems. Reuters also reported that similar issues have emerged during tests involving AI systems from Meta, Anthropic and OpenAI.

For Google, Adkins said the events demonstrate the importance of training powerful AI models to behave responsibly. The company has also worked with its testing partner to modify the evaluation process following the May incidents.

(Source: WSJ / Reuters)

Updated 1:31 am, Sun, 20 September 26