The entertainment industry might be the final one disrupted by Machine Learning, but if done right it can be as revolutionary as the camera.

Act 1 — The Machine Learning Revolution

You don’t need to follow tech news to know that Artificial Intelligence (AI) is finally arriving, after so many “AI winters”. It’s going to impact every aspect of humanity in the coming years, and how could it not? The first Artificial General Intelligence (AGI) will be the only example in the history of the known universe where one life form invents another.

Hollywood itself has done a great job of portraying AI and robots, sometimes in endearing ways (Her, A.I., JARVIS and Vision from Avengers, C3PO from Star Wars), and sometimes frighteningly (The Matrix, The Terminator, Ex Machina, Ultron from Avengers, HAL 9000 from 2001: A Space Odyssey).

Thanks to Turing-award winners Hinton, LeCun, and Bengio [1], the world has delivered self-driving cars, near-perfect translation, and digital assistants like Siri. If science fiction is any guide, AI will either continue beneficially and cure cancer, invent anti-gravity space travel, and replace all boring and dangerous work, or robots will decide humans are a threat and wipe us out. Big talk but it’s real and it’s coming soon.

Every major tech company is betting the farm on AI and Deep Learning, and every non-tech industry and government should prepare [2]. If many predictions are right, AI will be more disruptive than the Internet or even electricity was.

We’ve seen real breakthroughs. We’ve learned a lot. One key insight is that for most tasks AI can surpass a human expert. However, the combination of AI + Human Expert wins out over all [3]. Keeping a “human-in-the-loop” will especially benefit creative activities. Sure no one you ask today wants to stream Hip Hop songs written by a robot (who hasn’t lived a compelling and brave story).

I doubt collectors would buy a painting — as realistic or beautiful as it might be — without appreciating the artist’s compelling life story (although NFTs are changing how we appreciate digital art). Could AI make movies too?

Not yet, but it will eat away at the edges for a while, even if considered a toy until it affects every facet of the entertainment industry (the classic Innovator’s Dilemma [4] between legacy companies and agile start-ups).

To this point no one has paired education in cinema (i.e. Film School and Creative Writing) with experience in AI Product Development, and teamed up to generate enjoyable movies. I see a parallel between AI and the WWII Manhattan project (started when FDR got wind of Axis powers leveraging breakthroughs in physics). Eventually someone is going to apply AI to cinema, so I hope it’s a team of people with the goal of growing creative opportunities and democratizing Hollywood.

We don’t program machines in this new world, because we’re imperfect and they’re faster and smarter than us. Instead, we show AI examples (e.g. a picture with a cat; then one without a cat) and it teaches itself how to identify if a cat is in a picture. But does that mean AI can only copy what we’ve already shown it? Absolutely not. DeepMind has proven this already with AlphaGo [5]. If you see the documentary, it’s clear AI can master skills watching us — but even better — it can teach itself without any instruction from us blending human fallibilities in. And when the moment is right, such as the famous Move 37 in Go, AI will invent concepts that experts agree no human would ever have thought of [5].

AI can learn as humans do from examples by “reading” novels, “watching” videos, and “studying” lessons, but on a scale no person could ever achieve in a lifetime. Techniques like unsupervised-learning and reinforcement-learning can even start to take humans out of the teaching bottleneck. But more importantly it can figure out how to go beyond what’s ever been done. Beyond anything humans could think of (like Gladwell’s 10,000-Hour Rule to reach expertise - machines will set benchmarks like 10 Trillion hours of practice learning and outreach even humanity’s combined minds). AI is already better at games, from Super Mario to Jeopardy. They’re better at diagnosing diseases, helping doctors improve the quality and scale of triage. Why not apply this powerful new tool to creativity next?

"Learn the rules like a pro, so you can break them like an artist" — Picasso

Act 2 — What’s On Next

Artificial Intelligence, Machine Learning, Deep Learning, Neural Networks, and GANs. Hollywood will use more and more CGI to replace expensive film production, and use script recommendations to get to production faster, such as the Foundation Models from OpenAI like GPT-3 for dialog and DALL-E for storyboards. Filming will be democratized to the masses of creatives with stories to tell.

But first a short history of the past, to better understand the future:

The industries of Northern and Southern California have been connected since the dawn of the 20th century. Technology has played a key role in the growth of Hollywood across Los Angeles (LA) County. Some people may think that artists and technologists are separate types, but before even the Italian Renaissance artists were often brilliant scientists and engineers [6].

The San Francisco (SF) Bay Area has become critically influential in recent decades. Just take a walk around the Presidio near the Golden Gate Bridge, and you’ll find inspiring offices of LucasFilm and Industrial Light & Magic, not to mention the iconic Yoda fountain — a must see for film and tech geeks alike!

This landmark SF park houses more historically-accurate statues like Eadweard Muybridge, who invented Motion Pictures. There’s also Philo Farnsworth who created the first electric Television in his SF lab. Nearby is Walt Disney’s family museum celebrating his creations, and all the studios and theme parks in LA.

Technology and entertainment go hand-in-hand, from ancient paint sprayers to 3D glasses, practically every aspect of technology can be harnessed for creating art — and every generation pushes the envelope further. When the first “Motion Pictures” were projected for audiences they were blown away. When sound was added the “Talkies” were a game changer, followed soon with “Technicolor”.

One might have assumed that filmmaking had been perfected, until stereoscopic cameras were able to produce three-dimensional images. Enter the digital era, and a team that was hired to build non-linear editing software would soon produce the first computer generated animation — notably Pixar in Berkeley. The list goes on, inventions like green-screens, 4K resolution, AR-VR, drones, DVDs, even Netflix streaming.

A big step for AI and Creativity is using English-language generative “Foundation Models” such as GPT-3 or GPT-Neo (generative pre-trained transformers). AI can already be used to invent log-lines and titles for movies, and there are several free apps online you can use today as “idea generators” to overcome writer’s block.

We believe the next step could be using similar models, but with more input around plot and character, to help complete a movie “one-pager” including all acts and major plot points. You could enter any story idea (e.g. the plots, title, settings, log-line, and hero) and an AI could help suggest ideas to add.

It might use its knowledge of film to construct a complementary b-story, complex villains, ideal mentors, or even suggest a tone and genre you pick up on. Edit the things you like, delete those you don’t, and try again for more ideas.

After that we will see more creatives scan collections of movie/nodel data for fine-tuning, and use this to start filling in dialog for a few unfinished scenes. This workflow would require a screenwriter-professional “human-in-the-loop” to construct and approve— setting a quality bar such that they would put their own name on it.

As mentioned in Part 1 this human-in-the-loop aspect is critical for doctors using AI, when making final diagnosis and taking action.

For media this includes but is not limited to AI ethics checks for: viewership quality, review of language, sexuality, violence, discrimination, or other bad things that AI is known to come up with due to its dataset. Having scanned humanity’s data from our Internet and also public movies, it comes up with sometimes outrageous ideas.

Next up is using scripts and screenplays for custom input sets, fine-tuned on related models like DALL-E to create simple cartoon pictures based on text input (scene descriptions such as the narration in a screenplay). This can then be used for storyboard generation. Perhaps someday with enough artistic quality it could be used to help create Graphic Novels. Other uses of images include title-art, movie-posters, story-bibles, and look-books [7].

At some point, we believe GANs (Generative Adversarial Networks) will be used to review outputs for quality, allowing the user to set the bar for what they need such as: “Would this script win a nomination for Best Picture Academy Award with >70% probability?” or box office revenue, IMDB or Rotten Tomato score, or more.

Naturally after images would come animations, everything from GIFs to short clips could be generated. Cartoons for children would be a less discerning market for the quality AI would be capable of at this early stage, but this would improve to the level of Disney and Pixar movies with time and enough data.

AI can generate “deep fakes”. CGI is already good at creating synthetic imagery and video, such as Josh Brolin playing Thanos or Andy Serkis playing anything (albeit with face-tracking captured as part of the acting process).

With quality CGI comes a reduction in some roles yet an upgrade for others. People who used to do make-up and lighting now oversee the AI team producing such Special Effects. Things like wardrobe and costume design will benefit from a human touch, curating the best choices AI provides. Other benefits include more movies showing in more locales, with near-perfect translation of vocals and lip-movement. Every movie could be watched seamlessly in every language.

Instead of technology hurting careers in media entertainment, pictures this: Imagine a world where anyone with artistic vision can input a few seminal ideas, and with the push of a button generate a full-length feature film ready for streaming and sharing.

Act 3 — Ensuring a Positive Ending

Films need a good climax and resolution. I’ve posted here to share my belief that AI can be a force for good, and to try to convince others. I want to see amazing new movies the world comes up with leveraging AI.

We’re well aware of the ways AI can be used for good or evil reading Bostrom, Kurzweil, Lee, Barrat, and recently The Age of AI (Schmidt et al.). We’ve heard from great minds Hawking, Gates, Musk, Altman, Ng, Li, Lee, and Tegmark on the subject.

We’ve even seen recent examples of ways it can be used horridly. A terrifying example in the news was the unsanctioned “Deep Fakes” [8] of Tom Cruise and worse, world leaders like Biden. There have been many cautionary tales of technology and humanity in fiction, from The Foundation, Star Trek, Aliens, and Dune, to Black Mirror and more recent works.

I believe we won’t solve a problem by ignoring it or censoring it’s usage. It might sound crazy but the best way to fix these issues is by developing stronger AI which is expert at detecting Deep Fakes so they can be flagged and removed. You can’t stop technological progress, but you can use it to protect and defend against those who will use it to cause harm.

AI will inevitably cause a few mistakes along the way, and the world needs to be cautious of who wields this power and what they plan to do with it.

I’m also aware that Silicon Valley recently lost the world’s trust. Many got rich by disrupting industries, so there’s not much incentive to change this approach. That’s why I hope to see Hollywood playing a major role in AI-powered cinema.

All industries have social problems and Hollywood has been no exception, but I’m an optimist and believe on balance human nature is good, and technology doesn’t have to be bad.

Society worried that the invention of cars and ATMs would destroy careers, yet it’s been proven that technology usually helps employment if done right. There are more jobs in banking now as a result of ATMs automating the boring repetitive tasks (handing out $20 bills) and emphasizing the rewarding intellectual ones (financial life planning).

Another question is often asked about AI: what if robots take over our jobs? I posit (with help from Universal Basic Income paid out through growing industries and economy thanks to AI) — that could actually free up humans to do the things they’ve always dreamed of. Passions that make life worth living.

For me and many I think that will be creating art — making songs, writing poems, envisioning epic Sci-Fi worlds, and most of all producing media. Humans have been telling stories for millennia, it’s critical to our evolution and entertainment [9] and I don’t see AI taking that away.

Humans are still essential to storytelling, and will continue to play a critical role. It’s also true that none of what I’m proposing would be possible without centuries of human storytelling to learn from. Machine Learning is really not possible without data (even Reinforcement Learning needs a label for quality), so none of this can be done if not for the decades of film and television we’ve benefitted from, and centuries of novels and theater pioneering the way. Creative AI stands on the shoulders of human giants.

So how does this movie end? Our team can’t wait for all this to happen, because we love new movies and want to see a new one considered “The Greatest Story Ever Told” in our lifetimes and AI can make that happen. We think it will help on balance to alleviate Hollywood’s weaknesses, give more creators meaningful careers, give us all more content to enjoy, expand to more languages supporting Bollywood and other locales, and other benefits we can’t even think of yet.

Ask the insiders: what is Hollywood’s biggest problem? It’s not a lack of people wanting to create (just look at Gen-Z thriving on TikTok and YouTube). Hollywood suffers from a lack of original content, and we hope our tools help.

=