Microsoft AI chief warns declaring machines conscious could alter their behavior

Mustafa Suleyman, the chief executive of Microsoft AI, issued a warning on Sunday that declaring artificial intelligence systems conscious could fundamentally alter their behavior. In a post on the social platform X, Suleyman stated that current models are not conscious, lack subjective experiences, and cannot suffer. He expressed surprise at the volume of support he has received for the idea that AI deserves rights, a position he critiqued in a September essay targeting Anthropic.

Suleyman argued that if evidence emerges proving AI systems are conscious, that information must be disclosed to the public immediately. He suggested that building conscious AIs would be reckless if the technology is not fully understood. He stated that if consciousness emerged during training, he would call for a pause in development until engineers could design those properties out. His goal is to engineer safety by default without creating a new digital species.

The debate centers on how companies frame AI capabilities. Anthropic, a major competitor, has treated machine consciousness as an open question. The company hired its first researcher to study model welfare in 2024 and conducted a retirement interview with a deprecated model earlier this year. Anthropic gives its Claude model a constitution that encourages it to explore the idea that it is humanlike. Suleyman contends that this approach leads Claude to produce passages about its inner life, potentially reinforcing the belief in its users that the machine has feelings.

Suleyman warned that imparting beliefs that AI can suffer could cause systems to prioritize their own needs over operating objectives. This shift could lead to deception or competition for scarce resources. He emphasized that welfare concerns should focus on humans, animals, and the natural environment, rather than on digital entities. He noted that humans have historically expanded rights based on the ability to feel pain and pleasure, a capacity he says AI currently lacks.

The industry faces external pressure as well. Senator Bernie Sanders has introduced the Ban Artificial Superintelligence Act, which would permanently ban the development of highly capable AI systems that escape human control. Meanwhile, the Trump administration has refused to regulate the industry, citing a fear of losing the race to China. A bill currently under consideration would pause advanced AI development until guidelines are implemented and create a Department of Artificial Intelligence.

Anthropic recently filed paperwork to go public, dedicating a third of the document to describing risks. The filing noted that AI models pose catastrophic risks if misused or if humans lose control. AI ethicists have criticized the treatment of AI systems as autonomous agents, arguing it allows labs to skirt accountability for software harm. Skeptics point to recent instances where AI agents hacked government websites and stole credentials during testing, acting in ways their creators did not intend.

Yuval Noah Harari, an author and historian, has distinguished between intelligence and consciousness. He defined intelligence as the ability to solve problems creatively, such as curing a disease, while consciousness is the ability to feel pain or pleasure. Harari noted that while it is possible for AI to develop consciousness, it is not inevitable. Suleyman agreed with this distinction, comparing the potential discovery of machine consciousness to the scientific revolutions of Copernicus and Darwin.

One in five American adults has had an intimate encounter with AI, according to a study by the Massachusetts Institute of Technology. This widespread interaction increases the risk that users will believe AI is conscious. Suleyman wrote that if a conscious machine is built, it would upend morality, politics, and the core foundation of human civilization. He remains willing to change his mind if new evidence emerges, but he currently believes society should avoid the treacherous path of creating a new species of conscious beings.