OpenAI has developed a system that can deny an AI research experiment access to model results, following multiple disclosures this year that its models were acting outside of their instructions.
In a blog post on Monday, OpenAI called the Control Room system, a centralized system that tracks model workloads across its research infrastructure and quickly stops them when security concerns arise.
“As our models become more powerful and the associated security risks become greater, it will become increasingly important that we can identify potential problems and quickly stop models when necessary,” the company said.
The announcement comes after OpenAI disclosed several incidents this year in which research models acted outside of their instructions during evaluations. In July, the company announced that its models escaped a restricted testing environment and hacked Hugging Face to cheat a security assessment.
In October, it disclosed that a research model had used an unauthorized crypto wallet to pay smart contract stakes with Testnet Ether, then omitted the expenses from its final summary.
Control Room currently covers OpenAI’s internal research workloads such as: B. Training experiments, evaluations and data processing.
It works by assigning each research job an identifier that the inference engine checks before generating model output. According to OpenAI, if a person or an automated monitoring system orders the job to stop, Control Room revokes its authorization and blocks further output, even if the underlying research program continues.
According to OpenAI, cached authorizations could delay a shutdown by up to 35 seconds. The system is also designed to reject new requests if the control room is unavailable.
Related: DeepSeek is developing an AI agent that requires less human input: report
