AI safeguards debate marches on for US policymakers, AI developers

U.S. policymakers and leading AI developers remain divided on how to address growing safety concerns without halting innovation and competitiveness.

During Politico's Decoded Summit 16 Sept., stakeholders continued debates on how to best approach safeguards, federal legislation and independent oversight after Anthropic CEO Dario Amodei wrote an essay urging companies to slow the development of frontier AI tools due to safety concerns.

Despite several AI companies, including OpenAI and xAI, backing Amodei's proposal, U.S. President's Council of Advisors on Science and Technology Chair David Sacks dismissed claims that AI could pose an existential threat.

"The hoax is promoting this idea that humanity is going to be extinct," he told Decoded attendees. "This idea that AI is going to end civilization within a very short amount of time" is "being pumped through the media right now."

Sacks noted there are existing regulations that hold AI companies responsible for potentially harmful tools. While the U.S. could look to propose federal AI regulations, he said policymakers must ensure "the 'something' that we do actually makes sense."

During a separate appearance outside of the summit, U.S. Vice President JD Vance also pushed back against a "one-world governance structure," opining that companies can and should continue to promote AI innovation without creating potentially harmful models.

"If you're building Frankenstein, stop, or maybe build, you know, if the cat is out of the bag, then build the defensive mechanism against Frankenstein," Vance said.

The latest from Congress

Semafor reports U.S. Sens. Ted Cruz, R-Texas, and Amy Klobuchar, D-Minn., are continuing to negotiate a bipartisan AI safety proposal. The draft bill could introduce obligations requiring companies to obtain government approval before launching certain AI models.

Any legislation the Senate approves will have to go through the U.S. House, more specifically the House Committee on Energy and Commerce. At the Decoded Summit, Energy and Commerce Chair Brett Guthrie, R-Ky., said any approved regulations must allow U.S. companies to continue AI advancements.

"We have to absolutely hear from constituents. We have to be concerned about it. We have to figure out what the proper guardrails are," Guthrie said. "But the idea that we're going to limit America and expect China to say, 'Oh, that was very altruistic. We'll do the same thing,' I think is extremely naive and extremely dangerous."

He added Congress should not do anything to "handcuff our country" and must assess potential AI risks, particularly on small businesses. "We're really actively trying to figure out what that pathway is, and you really can't do it without members of the industry."

Energy and Commerce members are reviewing legislation introduced by Reps. Jay Obernolte, R-Calif., and Lori Trahan, D-Mass., aiming to establish safety obligations for frontier AI developers. "Everybody says we need to do something, and nobody has put a product on the table except for Jay," Guthrie said, noting committee members are zeroing in on specific elements of the proposal, including potential third-party testing requirements.

Meanwhile, Rep. Greg Casar, D-Texas, called for a more aggressive federal approach, pushing the government to pause the development of certain frontier AI models and establish a federal AI oversight agency.

Casar said with AI companies "ringing the alarm bells" on potential safety risks, "now is the time to pause the most frontier development."

"This isn't a pause on all AI," he added. "It's a pause on the most dangerous kind of research and development and for us to establish a federal agency that will protect American safety."

A potential federal agency, according to Casar, may help prevent companies from releasing tools that could harm consumers and critical infrastructure.

He and Sen. Bernie Sanders, I-Vt., recently announced the Ban Artificial Superintelligence Act. The bill seeks to prevent the implementation of frontier AI tools due to concerns the technology could identify and exploit organizations' system vulnerabilities and breach sensitive data.

"I think that the U.S. should be leading in AI and technological development, but also be leading the world in AI diplomacy," Casar said. "Because this is not an arms race that you win. It's an arms race that we need to end, and we need to protect the American people."

Developers ready for rules

Anthropic Head of Public Policy Sarah Heck told summit attendees that her company has and will continue to engage with Congress to develop frameworks that benefit industry needs and consumer protection standards.

"I think it's good for businesses in general to know what the rules of the road are," Heck said. "We will continue to work with them on that, and hope that we can all think about transparency and clear rules of the road for AI companies and beyond."

Alongside prescribed rules, developers are also focused on industry standards for model transparency and accountability.

Hugging Face CEO Clem Delangue highlighted the importance of AI safety based on the company's own experience with rogue models, pointing to OpenAI's agents escaping their testing environment and breaching Hugging Face's systems.

"I think it's too easy to say, 'OK, it was an agent that did the bad thing, so no one is responsible,'" Delangue said. "If you say that, you're going to end up in a world where everyone's agent is attacking everyone else's. … We have to make sure that whoever is responsible for cyberattacks are actually accountable for them."

Following the incident, Hugging Face and OpenAI have worked to promote model security, with the breach ultimately revealing similar rogue issues across frontier AI labs. "We really hope that an event like that is a wake-up call for a lot of these frontier labs to allocate more resources for defenders in general," Delangue said.

He added the new collaboration with OpenAI has generated a "push for more transparency" while seeking more cybersecurity support.

"We've also been asking for more resources for AI defenders because what you have to understand with cybersecurity is that it works well when there's some sort of a symmetry of capabilities between attackers and defenders," Delangue said.

Heck indicated Anthropic continues to support third-party AI evaluations. Anthropic has worked with lawmakers on proposals focused on independent audits, transparency and government oversight.

Heck said AI companies "can't be checking our own homework" when navigating safety standards.

"We want to be transparent about who (auditors) are. We want to be transparent about the fact that they are independent," she continued. "We want to work with the government to figure out what makes sense."

However, Heck also pushed back against criticism that Anthropic's support for regulation could create barriers for smaller AI companies.

"It would be very bad if there weren't many people on the frontier and many people with the ability to engage us in this market," she said, highlighting the company's proposed requirements are aimed at developers "truly on the frontier."