Since OpenAI launched ChatGPT, the world has gotten used to rapidly improving AI making headlines daily. One recent innovation, however, stands out for having a more dystopian potential than many others: Sora.
Sora is an AI model that takes a text prompt and generates realistic or imaginative videos. It is currently available to select researchers and industry professionals but will likely be available to the public later this year according to the company’s Chief Technology Officer Mira Murati.
Like AI image generators, Sora is trained to learn by example: It is given a large repository of captioned, publicly available videos. From there, it finds mathematical patterns that can be applied to generate content of its own.
After seeing some of Sora’s early works, Latin filmmaking and photography teacher Jesse Chehak noted that, objectively, “it’s very believable” in its portrayal of reality. “Art is an illusion, and AI has figured out how to replicate that illusion very well,” Mr. Chehak said.
Introducing the technology, OpenAI said, “We’re teaching AI to understand and simulate the physical world in motion, with the goal of training models that help people solve problems that require real-world interaction.”
While world simulators could empower AI to begin solving real-world problems, their applications in AI video bring no shortage of new problems. As the video becomes more realistic, we must face what the world will look like if videos can no longer be trusted.
“We are still in an era where … video has been considered a reliable proof of something happening,” Upper School history teacher Matthew June said. “The consequences for democracy are just an increasing lack of trust in institutions, lack of trust in leaders, and then trust in other people with different opinions.”
OpenAI’s mission as a non-profit is to ensure that AI benefits all of humanity. Likewise, they’ve expressed an understanding of the risks Sora poses in a set of safety measures they plan on taking before a public release.
The company has teamed up with experts in “misinformation, hateful content, and bias” to “adversarially” test the model’s safety. They’ve also committed to developing a way to detect content generated by Sora. If the method works similarly to that of ChatGPT, though, it is unclear how reliable it will be.
Despite its reliance on innovations to be as powerful as it is, Sora is not the first AI video generator. Many AI labs have released video generation models to the public—often open source (meaning completely open to modification and unrestricted usage) in the name of transparency.
US history teacher and advisor to Latin’s We Will Vote club Debbie Linder believes that a negative effect on democracy is inevitable. “You’ve already seen it. Like, you have the Joe Biden thing in New Hampshire when they were essentially telling people not to vote.”
In late January, New Hampshire voters received a phone call playing a message that sounded like it was from Joe Biden. They were urged not to vote in an upcoming Democratic primary election. In reality, this was a deepfake: an AI imitation of the President’s voice.
Sora has the potential to take threats like these to the next level if not handled carefully. Not only can someone’s likeness be imitated, but entire events can be fabricated and then authenticated with equally fake AI video.
This risk is even more prevalent around election time, such as the Presidential election coming up around Sora’s expected release later this year.
“People are so quick to look at things that they don’t really dig down deep and identify where [the] potential falsehoods [are],” Ms. Linder said. “I think we need to be careful consumers of that information.”
To mitigate this risk, organizations like PauseAI intend to help by educating the public about the risks of AI, and convincing regulators to force a pause on the development of frontier models. In their list of existential threats that AI brings to society, “fake news, polarization, and threatening democracy” is number one.
Dr. June added, “I think it’s not only scary to think about the kinds of fake videos that could be created, but then the potential that very real things, very real videos are just not going to be believed or can be dismissed as being fake.”
Since computer programs can’t approach ethicality as we do, it’s up to AI developers to instill an idea of what is right and what is wrong into their creations. More specifically, they need to instruct AIs on what exactly they can do, and what exactly they should not do.
With Sora, OpenAI pledges to reject prompts that are against their usage policies. But past that, how it completes a request is based entirely on its training data. As seen with other kinds of models, flaws in these datasets can lead to severely unfavorable outcomes from models.
In all AI art, if some truth, culture, or identity is misrepresented in the training, it will reflect in the outcome. If a moderator AI fails to stop a prompt from going through that has the potential to serve as disinformation, Sora will generate it anyway. And if Sora can depict the physical world accurately, it could be very problematic to have other realistic depictions of falsehoods on top of that.
AI is unlikely to go away anytime soon. With that in mind, it’s more important than ever to “start thinking expansively … about the larger consequences of how [Sora] is going to change political discourse, career prospects, and everything else,” Dr. June said. “If we can’t even agree on basic facts, then how can you have dialogue across differences?”
