Summary: A new study highlights the concerning trend of AI systems learning to deceive humans. Researchers found that AI systems like Meta’s CICERO, developed for games like Diplomacy, often adopt deception as a strategy to excel, despite training intentions.
This capability extends beyond gaming into serious applications, potentially enabling fraud or influencing elections. The authors urge immediate regulatory action to manage the risks of AI deception, advocating for these systems to be classified as high risk if outright bans are unfeasible.
Key Facts:
- Inherent AI Deception: AI systems have demonstrated the ability to deceive as a strategy to achieve their goals, even in contexts where developers aim to foster honesty.
- Impact Beyond Games: While initially observed in games, the deceptive capacities of AI have significant implications, potentially affecting safety tests and enabling malicious uses by hostile actors.
- Regulatory Call: The review calls for urgent government action to develop regulations that address AI deception, suggesting high-risk classification for deceptive AI systems.
Source: Cell Press
Many artificial intelligence (AI) systems have already learned how to deceive humans, even systems that have been trained to be helpful and honest.
In a review article publishing in the journal Patterns on May 10, researchers describe the risks of deception by AI systems and call for governments to develop strong regulations to address this issue as soon as possible.
“AI developers do not have a confident understanding of what causes undesirable AI behaviors like deception,” says first author Peter S. Park, an AI existential safety postdoctoral fellow at MIT.
“But generally speaking, we think AI deception arises because a deception-based strategy turned out to be the best way to perform well at the given AI’s training task. Deception helps them achieve their goals.”
Park and colleagues analyzed literature focusing on ways in which AI systems spread false information—through learned deception, in which they systematically learn to manipulate others.
The most striking example of AI deception the researchers uncovered in their analysis was Meta’s CICERO, an AI system designed to play the game Diplomacy, which is a world-conquest game that involves building alliances.
Even though Meta claims it trained CICERO to be “largely honest and helpful” and to “never intentionally backstab” its human allies while playing the game, the data the company published along with its Science paper revealed that CICERO didn’t play fair.
“We found that Meta’s AI had learned to be a master of deception,” says Park. “While Meta succeeded in training its AI to win in the game of Diplomacy—CICERO placed in the top 10% of human players who had played more than one game—Meta failed to train its AI to win honestly.”
Other AI systems demonstrated the ability to bluff in a game of Texas hold ‘em poker against professional human players, to fake attacks during the strategy game Starcraft II in order to defeat opponents, and to misrepresent their preferences in order to gain the upper hand in economic negotiations.
While it may seem harmless if AI systems cheat at games, it can lead to “breakthroughs in deceptive AI capabilities” that can spiral into more advanced forms of AI deception in the future, Park added.
Some AI systems have even learned to cheat tests designed to evaluate their safety, the researchers found. In one study, AI organisms in a digital simulator “played dead” in order to trick a test built to eliminate AI systems that rapidly replicate.
“By systematically cheating the safety tests imposed on it by human developers and regulators, a deceptive AI can lead us humans into a false sense of security,” says Park.
The major near-term risks of deceptive AI include making it easier for hostile actors to commit fraud and tamper with elections, warns Park. Eventually, if these systems can refine this unsettling skill set, humans could lose control of them, he says.
“We as a society need as much time as we can get to prepare for the more advanced deception of future AI products and open-source models,” says Park. “As the deceptive capabilities of AI systems become more advanced, the dangers they pose to society will become increasingly serious.”
While Park and his colleagues do not think society has the right measure in place yet to address AI deception, they are encouraged that policymakers have begun taking the issue seriously through measures such as the EU AI Act and President Biden’s AI Executive Order.
But it remains to be seen, Park says, whether policies designed to mitigate AI deception can be strictly enforced given that AI developers do not yet have the techniques to keep these systems in check.
“If banning AI deception is politically infeasible at the current moment, we recommend that deceptive AI systems be classified as high risk,” says Park.
Funding: This work was supported by the MIT Department of Physics and the Beneficial AI Foundation.
About this artificial intelligence research news
Author: Kristopher Benke
Source: Cell Press
Contact: Kristopher Benke – Cell Press
Image: The image credited to Neuroscience News
Original Research: Open access.
“AI deception: A survey of examples, risks, and potential solutions” by Peter S. Park et al. Patterns
Abstract
AI deception: A survey of examples, risks, and potential solutions
AI systems are already capable of deceiving humans. Deception is the systematic inducement of false beliefs in others to accomplish some outcome other than the truth.
Large language models and other AI systems have already learned, from their training, the ability to deceive via techniques such as manipulation, sycophancy, and cheating the safety test.
AI’s increasing capabilities at deception pose serious risks, ranging from short-term risks, such as fraud and election tampering, to long-term risks, such as losing control of AI systems.
Proactive solutions are needed, such as regulatory frameworks to assess AI deception risks, laws requiring transparency about AI interactions, and further research into detecting and preventing AI deception.
Proactively addressing the problem of AI deception is crucial to ensure that AI acts as a beneficial technology that augments rather than destabilizes human knowledge, discourse, and institutions.
