Why California’s AI safety bill is driving a wedge through Silicon Valley

Meanwhile, OpenAI rival Anthropic appeared to emerge as a supporter of the bill last week, as its CEO, Dario Amodei, wrote in a letter to California Gov. Gavin Newsom that the “benefits likely outweigh the costs.”

The bill has proven to be so divisive for a number of reasons.

Why Silicon Valley is split on SB 1047

First, there is the question of its impact on innovation.

According to the anti-SB 1047 camp, the bill’s stringent measures threaten to slow the development of future models, something that could prove dangerous as the US faces a rising AI power in the form of China.

That, at least, was the argument put forward by OpenAI’s chief strategy officer, Jason Kwon, in the company’s letter to Sen. Weiner. Not only would the bill “slow the pace of innovation,” Kwon said, but it could trigger an exodus of “California’s world-class engineers and entrepreneurs.”

The regulation’s impact on AI models that aren’t among the most powerful and possible consequences for startups is also raising questions.

Legal ‘patchwork’

A16z general partner Anjney Midha, for instance, wrote in The Financial Times last month that the bill’s categorization of powerful models as anything with a training cost of more than $100 million set a “relatively low bar” given that “AI development costs run to billions.”

There is also a view that the federal government should take the lead on regulations of this magnitude. OpenAI’s Kwon, for instance, suggested that governance of AI at a state level could create a “patchwork” of laws.

Sen. Weiner, of course, dismisses this idea. Responding to OpenAI’s letter, he said that “ideally Congress would handle this,” but its lack of action to date makes him skeptical that it will take action anytime soon.

Of course, AI companies will know there is an urgency to ensure the tech is rolled out safety. Earlier this month, OpenAI revealed that it had to take down “a cluster of ChatGPT accounts that were generating content for a covert Iranian influence operation” called Storm-2035.

Its purpose, OpenAI said, was to generate content and commentary on topics such as the US presidential election.

That is just one illustration of AI’s nefarious potential. Whether SB 1047 is the right way to address that threat will be a subject of fierce debate in the coming months.