OpenAI chief executive Sam Altman has urged the public to trust AI companies to act responsibly as concerns grow over the potential dangers posed by increasingly powerful artificial intelligence systems.

Speaking at Salesforce’s annual conference in San Francisco on Tuesday, Altman acknowledged that people have legitimate reasons to be concerned about AI, saying the technology has advanced to a point where imagining serious consequences is no longer difficult.

“The world should trust that we are going to do the right thing because it’s the right thing and we feel the magnitude of this,” Altman said.

He added that public concern was understandable because AI systems had become considerably more capable.

“It doesn’t take as much imagination as it used to for [us] to imagine how this could go wrong,” he said. “I think the world is right to be afraid of this.”

Altman’s remarks came days after a former researcher at AI company Anthropic made a widely circulated claim that unchecked AI development could potentially result in the extinction of humanity before the end of the decade. Several other AI executives and researchers subsequently said they shared concerns about the possibility, although they did not publicly explain how they reached those conclusions or how such an outcome might occur, reports BBC.

The debate has renewed scrutiny of how quickly frontier AI systems are being developed. Anthropic chief executive Dario Amodei has called for the pace of development to be reduced and urged governments to introduce stronger oversight of the sector.

Altman has expressed support for Amodei’s call for greater caution, as have Google DeepMind co-founder Demis Hassabis and Elon Musk, who owns AI company xAI.

However, several technology executives have argued that AI companies should retain primary responsibility for regulating the development and release of their systems rather than relying on governments.

Altman said he was confident that OpenAI and the wider industry could manage the risks themselves.

“We will get it right, I’m very confident in our company’s and industry’s ability to do this safely,” he said. He added that companies should keep safety and alignment ahead of the development of new capabilities, and should slow down or stop if they cannot do so.

Meta chief executive Mark Zuckerberg echoed that position in a post on X following Altman’s remarks. He argued that AI laboratories have both the ability and the incentive to develop safer systems, adding that companies could face significant legal liability if their models caused harm.

Nvidia chief executive Jensen Huang also spoke in favour of allowing AI companies to determine when new systems are ready for release. Speaking at the same conference, Huang argued that there was no need for additional laws or regulations and rejected what he described as a false choice between rapid innovation and AI safety.

“Safety is paramount. However, safety is an engineering problem,” Huang said.

Huang said companies should pause the release of a system whenever their leaders were no longer confident that they had control over it.

“Run as fast as you can, but if at any time you feel the institution is not in control, take a pause,” he said.

Not everyone in the AI industry agrees that companies should be left to police themselves.

Some technology figures have argued that recent warnings about AI risks have been exaggerated, while others say the growing fears are being used to generate publicity and excitement around the technology.

Jack Clark, an Anthropic executive and co-founder, told the BBC that allowing AI to become a “totally unregulated industry” would amount to taking enormous risks with potentially far-reaching consequences.

Patrick Hillman, chief operating officer of Logical Intelligence, also questioned whether technology companies could be relied upon to prioritise the public interest.

“The only institution that Americans might trust less than Washington these days is Silicon Valley,” Hillman said, adding that both institutions had given people reasons to be sceptical.

“If you believe what you are building is dangerous, show us what you are prepared to stop doing,” he added.

The disagreement comes as OpenAI and Anthropic work towards establishing broader safety commitments across the AI industry.

Amodei, who made a brief appearance at the Salesforce conference on Tuesday, said Anthropic was in discussions with other AI companies about developing stronger safety standards and safeguards for AI systems.

He also said one of the biggest surprises of the AI boom had been the speed at which companies developing the technology had expanded and become increasingly important to society.

“We didn’t appreciate that it would lead to companies growing so fast, and how quickly they would become central to things,” Amodei said.

OpenAI executive Chris Lehane said last week that the company was also working with other AI laboratories, including Anthropic and Google DeepMind, on voluntary standards for frontier AI.

“With stakes this high, we cannot let the perfect become the enemy of the good,” Lehane wrote.

Meanwhile, Altman told hundreds of business leaders attending the conference that companies should make greater use of AI in their everyday work. He also warned that businesses need to prepare for cyberattacks that could be enabled or intensified by AI systems.

The debate over how much freedom AI companies should have, and how much external oversight they require, is likely to continue as the technology becomes more capable and increasingly embedded in businesses and wider society.