Ion Stoica is a professor at the University of California, Berkeley and cofounder of Databricks, an AI company built on Apache Spark. If SB 1047 passes, he predicts that within a year open source models from overseas, likely China, will overtake those made in the United States. Three of the top six top open source models available today come from China, according to the Chatbot Arena evaluation method Stoica helped devise.
Open source defenders also voiced opposition to SB 1047 at a town hall hosted with Wiener at GitHub, an open source repository owned by Microsoft, and a generative AI symposium held in May.
Newsom, who has not taken a position on the legislation, told the audience it’s important to respond to AI inventors like Geoffrey Hinton who insist on the need for regulation , but also said he wants California to remain an AI leader and advised lawmakers against overreach. “If we over regulate, if we over indulge and chase a shiny object, we can put ourselves in a perilous position,” the governor said. “At the same time we have an obligation to lead.”
Aiming to protect tech workers and society
Sunny Gandhi, vice president of government affairs at Encode Justice, a nonprofit focused on bringing young people into the fight against AI harms and a cosponsor of the bill, said it has sparked a backlash because tech firms are not used to being held responsible for the effects of their products
“It’s very different and terrifying for them that they are now being held to the same standards that pretty much all other products are in America,” Ghandi said.” There are liability provisions in there, and liability is alien to tech. That’s what they’re worried about.”
Wiener has disputed some criticisms of his bill, including a claim, in a letter circulated by startup incubator Y Combinator and signed by more than 130 startup founders, that the legislation could end up sending software developers “to jail simply for failing to anticipate misuse of their software.” That assertion arose from the fact that the bill requires builders of sufficiently large language models to submit their test results to the state and makes them guilty of perjury if they lie about the design or testing of an AI model.
“It’s very different and terrifying for them that they are now being held to the same standards that pretty much all other products are in America.”Sunny Gandhi, vice president of government affairs at Encode Justice
Wiener said his office started listening to members of the tech community last fall before the bill was introduced and made a number of amendments to ensure the law only applies to major AI labs. Now is the time to act, he told startup founders, “because I don’t have any confidence the federal government is going to act” to regulate AI.
Within the past year, major AI labs signed on to testing and safety commitments with the White House and at international gatherings in the United Kingdom, Germany, and South Korea, but those agreements are voluntary. President Biden has called on Congress to regulate artificial intelligence but it has yet to do so.
Wiener also said the bill is important because the Republican Party vowed, in the platform it adopted last month, to repeal Biden’s executive order, arguing that the order stifles innovation.
In legislative hearings, Wiener has said it’s important to require compliance because “we don’t know who will run these companies in a year or five years and what kind of profit pressures those companies will face at that time.”
Learn more about legislators mentioned in this story.
AI company Anthropic, which is based in San Francisco, came out in support of the bill if a number of amendments are made, including doing away with a government entity called the Frontier Models Division. That division, which would review certifications from developers, establish an accreditation process for those who audit AI, and issue guidance on how to limit harms from advanced AI. Wiener told the Y Combinator audience he’d be open to doing away with the division.
Kokotajlo, the whistleblower, calls SB 1047 both a step in the right direction and not enough to prevent the potential harms of AI. He and the other signatories of the June letter have called on companies that are developing AI to create their own processes whereby current and former employees could anonymously report concerns to independent organizations with the expertise to verify whether concern is called for or not.
“Sometimes the people who are worried will turn out to be wrong, and sometimes, I think the people who are worried will turn out to be right,” he said.
In remarks at Y Combinator last month, Wiener thanked members of open source and AI communities for sharing critiques of the bill that led to amendments, but he also urged people to remember what happened when California passed privacy law in 2018 following years of inaction by the federal government.
“A lot of folks in the tech world were opposed to that bill and told us that everyone was going to leave California if we passed it. We passed it. That did not happen, and we set a standard that I think was a really powerful one.”
