Like many young boys and girls that grew up during Disney’s Golden Age of Animation, one boy named Ed dreamed the biggest dream possible: to work with Disney and create the enthralling cartoons he watched religiously.
Eventually he reached the age where dreams start to fade and reality sets in, but he didn’t let that stop him. While he wasn’t a very good drawer, his skillset was in physics and computer science.
So he spent days in front of his computer, painstakingly creating a model of his left hand and writing a three-dimensional animation program that would move the hand. Ed named the project A Computer Animated Hand. It would became one of the world’s first steps in using computers for 3D animation.
The creator’s name was Ed Catmull – the eventual founder of Pixar Studios.
Ed played an instrumental role in showing us a computer’s ability in creating great animation. And today we’re wondering how we can use AI programs to make painstaking, laborious animation a thing of the past.
What follows are many AI projects working on automating animation. They’re not flashy. They’re not complete. But one of these might be the next breakthrough, as Ed Catmull once brought us.
AI Makes A Cartoon
Drawing characters seems to be the most micro unit of creating a cartoon. While dreaming up completely new characters would be a tough job for an AI. An AI might be able to take existing characters within a cartoon and design new ones, say, for the background.
For instance, Greg Surma used a GAN (Generative Adversarial Network) to create new Simpsons characters:
The result is many images that look like deformed Simpsons. But you can see how this might improve to create minor, background characters.
A slightly different approach is CartoonGAN, which takes photographs and turns them into cartoons (or cartoon-like features).
Now that we have our characters, how might we animate them?
Last year, researchers fed an algorithm more than 25,000 3-second clips from The Flintstones and trained it to generate new clips based on text descriptions. The animations aren’t fooling any lover of the show, but it’s an interesting sneak peek of the future:
Google is working on a project called “inbetweening”. When given a first and final frame of a video, their AI can fill in the gaps.
Adobe has made progress in automating the lip syncing of characters with voice overs. Their Character Animator also has the ability of matching user movements and expressions with an animated character (kind of like Apple’s Animojis).
The 2D animation projects above all would be great additions to a tool like Harmony, which is used by nearly every cartoon imaginable. It would shorten the time to production and give novices the confidence of generating long-form content.
But what about major motion picture productions?
AI Makes a Movie
The real incentives for creating an AI-animator are in the movie department – replacing some of the current roles for CGI.
A minute of CGI for Game of Thrones required 1600 human hours which led to a cost of $80k.Myelin Foundry, Medium
A Pixar movie is in a production timeline of no less than 4 years. It’s a painstaking, detail-oriented process. And as of late, many animated movies are going toward a realistic style. Look at the most recent Lion King or Alita: Battle Angel. They look like real life.
Realistic animation requires intense attention to detail. For example, our eyes know exactly what honey looks like dripping from a honey dipper. We know how hair moves in the wind. Getting these details right is difficult. And details like textures and movements are exactly what AI is after.
Here’s a research project into simulating the interplay of pasta and sauce:
This one simulates water and debris flowing:
Here’s a gooey simulation:
Some other fascinating visual simulations are wringing a towel, simulating grains of sand, splashing liquid, realistic hair, and gooey honey.
These are the innovations that will push CGI and animated movies forward. Research into this field requires a rich understanding of physics and computer science. Just as Ed Catmull pushed animation forward with technology, so too will the next innovators.
Overall, the integration of AI into animation will be piecemeal. A feature here and there, little by little, until one day we reach a platform that gives anyone with an idea the ability to create an animated video. Until that time, animation will remain a rich interplay of art, storytelling, and human connection.