OpenAI introduces GPT-Red to strengthen AI safety and improve model robustness

0
33
OpenAI develops GPT-Red to enhance AI security through automated safety testing
OpenAI develops GPT-Red to enhance AI security through automated safety testing

OpenAI has introduced GPT-Red, an internal AI safety red-teaming model designed to identify vulnerabilities and strengthen the security of future AI systems. The model is built to automatically uncover weaknesses before deployment and generate advanced attack scenarios that help improve the robustness of production models.

AI systems often interact with third-party data through browsers, connected apps, local files, emails, webpages, code repositories, and other tools. These interactions can expose models to prompt injection attacks, where hidden instructions attempt to manipulate AI into performing unintended actions, such as exposing sensitive information.

GPT-Red uses self-play reinforcement learning, where it continuously attacks a group of defender AI models while those models learn to resist the attacks and complete their intended tasks. As the defenders become stronger, GPT-Red develops more advanced attack techniques. OpenAI said the model was trained using compute resources comparable to some of its largest post-training runs, making it one of the company’s biggest safety-focused training efforts.

The company has already integrated GPT-Red into the training process of its production models. As a result, GPT-5.6 Sol is reported to be its most robust model against prompt injection attacks, recording 6x fewer failures on its toughest direct prompt injection benchmark compared to its best production model from 4 months earlier.

OpenAI said GPT-Red remains an internal-only system to prevent its malicious capabilities from being misused while allowing its defensive techniques to strengthen deployed AI models.

The model also demonstrated strong performance in independent evaluations. It achieved an 84% attack success rate on previously unseen prompt injection scenarios, compared with 13% for human red-teamers. GPT-Red also successfully identified vulnerabilities in an AI-powered vending machine by changing product prices, placing discounted orders, and cancelling another customer’s order. OpenAI said these vulnerabilities have been disclosed and additional safeguards are being tested.

According to the company, GPT-Red has also helped identify new attack methods, including “Fake Chain-of-Thought” attacks. While these attacks previously achieved success rates above 95% against GPT-5.1, they now succeed in fewer than 10% of cases against GPT-5.6 Sol. The latest model also fails on only 0.05% of GPT-Red’s direct prompt injection attacks while maintaining its normal capabilities without increasing unnecessary refusals.

OpenAI said it plans to continue expanding GPT-Red through more compute, data, and algorithmic improvements to make future AI models more secure, aligned, and trustworthy. The company also confirmed that it will release a pre-print with additional technical details later this week.

(This story has not been edited by The Mainstream staff and is auto-generated from a syndicated feed.)

Also read: Viksit Workforce for a Viksit Bharat

Do Follow: The Mainstream LinkedIn | The Mainstream Facebook | The Mainstream Youtube | The Mainstream Twitter

About us:

The Mainstream is a premier platform delivering the latest updates and informed perspectives across the technology business and cyber landscape. Built on research-driven, thought leadership and original intellectual property, The Mainstream also curates summits & conferences that convene decision makers to explore how technology reshapes industries and leadership. With a growing presence in India and globally across the Middle East, Africa, ASEAN, the USA, the UK and Australia, The Mainstream carries a vision to bring the latest happenings and insights to 8.2 billion people and to place technology at the centre of conversation for leaders navigating the future.