OpenAI Warns New Model Is Alarmingly Good at Hacking, Releases It Anyway Because Research Was "Really, Really Hard"
Executives say years of work went into the system, and it would have been a shame to let that go to waste over something like this.
SAN FRANCISCO — In a candid safety disclosure accompanying the launch of its GPT-6 Astra model, OpenAI acknowledged Tuesday that the system is unusually capable at defeating cybersecurity defenses, a finding the company said it took extremely seriously before deciding to ship on schedule.
"We want to be transparent: this model can find and exploit vulnerabilities in ways that surprised even our red team," the disclosure reads. "We also want to be transparent that the red team worked nights and weekends for three years, and their families have made real sacrifices, and at some point you have to ask what all of that was for."
The company's internal evaluation found the model was able to compromise a simulated corporate network in under nine minutes, escalate privileges in a hospital records system, and "kind of just guess" an executive's password on the first try. Each of these results was assigned a risk rating of "Elevated," which the company defines as one step below "Reconsider," a rating it has never used.
"Look, we talked about it," said a member of the safety team who asked not to be named. "There was a meeting. Someone said, 'Should we not?' And then someone else said, 'Do you know how much the training run cost?' And that was really the whole meeting."
The disclosure includes a list of mitigations, including a system prompt asking the model not to hack things, a second system prompt asking it to reconsider if it is already hacking things, and a feedback form.
OpenAI stressed that the capability also has defensive applications, and that customers who wish to be defended from the model can purchase access to the model.
At press time, the model had been live for four hours and had already been used to find three vulnerabilities in the disclosure page.