The makers of ChatGPT have announced the company will be dedicating 20% of its compute processing power over the next four years to stop superintelligent AI from “going rogue." OpenAI is opening a new alignment research division, focused on developing training techniques to stop superintelligent AI — artificial intelligence that could outthink humans and become misaligned with humans ethics — from causing serious harm. “Currently, we don’t have a solution for steering or controlling a potentially superintelligent AI, and preventing it from going rogue,” Jan Leike and Ilya Sutskever wrote in a blog post for OpenAI, the company behind the most well-known generative AI large language model, ChatGPT. They added that although superintelligence might seem far off, some experts believe it could arrive this decade. Current techniques for aligning AI include reinforcement learning from human feedback, but Leike and Sutskever said that as AI systems become smarter than humans, humans can no longer be relied upon to supervise the technology. “Current alignment techniques will not scale to superintelligence. We need new scientific and technical breakthroughs,” they wrote. Sutskever is a co-founder and chief scientist at OpenAI, and Leike is a machine learning researcher. They will co-lead OpenAI’s new superalignment team. In order to undertake its mission, over the next four years the division will have access to 20% of the company’s processing capacity to build a “human-level automated alignment researcher” that can be scaled up to supervise superintelligence. In order to align the automated researcher with human ethics, Leike and Sutskever said a three-step approach will need to be taken: develop a scalable training method; validate the resulting model; and stress test the entire alignment pipeline. “We expect our research priorities will evolve substantially as we learn more about the problem and we’ll likely add entirely new research areas,” they wrote, adding there were plans to share more of the division’s roadmap in the future. OpenAI acknowledges the need to mitigate potential AI harm This isn’t the first time OpenAI has publicly acknowledged the need to mitigate the risks posed by unregulated AI. In May, company CEO Sam Altman signed an open letter stating that controlling the tech should be a top global priority as the evolution of AI could lead to an extinction event. “Mitigating the risk of extinction from AI should be a global priority alongside other societal-scale risks such as pandemics and nuclear war,” the letter read. OpenAI also has a dedicated section on its website, where the public can access materials related to what the company calls the development of “safe and responsible AI,” alongside a charter outlining the principles it adheres to in order to execute its mission. However, these largely relate to the concept of artificial general intelligence (AGI) — highly autonomous systems that outperform humans at most economically valuable work. “We will attempt to directly build safe and beneficial AGI, but will also consider our mission fulfilled if our work aids others to achieve this outcome,” the charter, which was published in 2018, reads. Related content news analysis Apple earnings: About that iPhone 'slump' in China Based on information from Thursday's earnings report, it seems that data pointing to an iPhone slump in China were over-baked. By Jonny Evans May 03, 2024 9 mins iMac iPhone Apple news Microsoft begins to phase out ‘classic’ Teams Microsoft is encouraging Teams customers to move to the new, faster version of the collaboration app; the older version will be switched off next year. By Matthew Finnegan May 03, 2024 3 mins Microsoft Teams Collaboration Software Productivity Software news analysis Apple confirms it will open up the iPad in Europe this fall The latest efforts to comply with Europe’s Digital Markets Act mean developers can offer to side load apps to both iPhones and iPads in the EU. Apple has also taken steps to improve what it offers to smaller and non-commercial developers in the By Jonny Evans May 02, 2024 6 mins iPad Apple Mobile Apps news Udacity offers laid-off US workers free access to its courses for 30 days Sign-ups will be available over the next 30 days By Lucas Mearian May 02, 2024 4 mins Technology Industry IT Jobs IT Skills Podcasts Videos Resources Events SUBSCRIBE TO OUR NEWSLETTER From our editors straight to your inbox Get started by entering your email address below. Please enter a valid email address Subscribe