Art of the Future: Pairing AI with Animation and Design
Xander Smith has a background working as a Concept Designer and 3D Artist for film and television, and right now is the Lead Digital Artist and founding member of Aliza Technologies- a company that pairs deep-learning/AI with animation and design.
Tell us about yourself.
I’m originally from Colorado but have been living in Los Angeles, CA, for about a decade. I moved here to take classes at the Gnomon School of Visual Effects and the Concept Design Academy so that I could work as an artist in film.
Since then, I’ve been fortunate enough to work on projects like American Horror Story, The Greatest Showman, Aquaman, The Mandalorian, and Godzilla vs King Kong, to name a few of my favorites.
Tell us about Aliza Technologies. What is it? How does it work?
Aliza is on the cutting edge of deep-learning technology that allows us to bring new techniques to character animation. As we’re pioneering our techniques, we’re building a process that allows artists to work with deep-learning networks to create state-of-the-art animations that will be on-par with the traditional 3D animation pipeline. We want to give animators and artists the power to use these networks to create beautiful animations.
Right now we’re working on a variety of projects that are pushing the boundaries of 3D and deep-learning animation. You can meet a couple of our public characters so you can see how we’re using the process, check out on Instagram @itsbinxie and @alizarexx. Aliza Rexx (whom our company is named after) recently just did an interview for Forbes Magazine which you can find here to see our animation in action.
We have a lot more incredible projects on the horizon, it will be exciting to share them as we go and as our deep-learning techniques continue to improve.
How does this type of art differ from the gaming or movie industry? Or is it similar?
It’s extremely similar, and definitely exciting for both the film and games industries. Using deep-learning processes for design and animation is going to elevate what’s possible in the world of film and games.
The barrier for entry continues to get lower and lower, as artists continue to push the limits of what’s visually possible, but also as tech becomes more powerful and cheaper. We’re living in a time when small teams of passionate designers can produce world-class content, that only just a few years ago was possible by huge studios in only a few cities in the world.
I have no doubt that we’ll be seeing wide-spread use of deep-learning techniques this decade, in every piece of production, from design to final animation, and I think it’s going to just continue pushing what’s possible in entertainment.
How do you think AI is changing the art world?
I’ve been working with the developers of Artbreeder.com, which is a generative network that allows artists to ‘sketch’ as fast as their imaginations, and am already seeing how this type of technology is going to elevate artists.
I’ll give you an example, when I’m designing a character for a film, I’ll go through the script with the director and production designer, and figure out what that character needs to look like. I can imagine hundreds of ways a character could look, each different idea bringing something fresh to the surface. But, I only have so many hours that I can be painting that character. But usually I don’t get to explore every idea I would have wanted to try out. I just don’t have the time on a production schedule and quite often, I have to play it safe in my design, and not take risks.
But since I’ve been using Artbreeder AI, I can speed-up areas of my design process and give myself more time to try out ideas, and take those design risks that I wouldn't normally be able to take under a production deadline. I can try out different iterations of a character's age, ethnicity, color, style, expressions, shapes, and more, all on the fly in real time. Now, even on tight deadlines, I have much more time to explore different designs, and take those risks in my design.
Do you see opportunities opening up for artists in the future because of AI?
Absolutely. Like any advancement in technology, artists are the first to tilt their necks, look at it, and say ‘hmm, maybe if I use it like that, I can create this.’ AI will open new doors for creativity and storytelling, and artists are going to work with the tech, and continue pushing the envelope for what’s visually possible.
It’s as if a whole new medium is being born right before our eyes.
For artists just starting out, is this an area they should explore?
I definitely think so, it’s important to keep an eye on emerging trends in tech, especially if you’re more on the commercial side of the industry. There are no dead-ends to creativity.
But the caveat, I’ve found, is always thinking about the ROI of exploring each new skill. We’re at a time now where you might start seeing less of a return on traditional drawing, and maybe a bit more on new technologies (for better or worse- I still recommend learning how to draw though!).
One of the coolest things you can do as an artist is position yourself at the forefront of emerging tech, and allow yourself to make mistakes, to make not-so-great art, to stumble, and to let yourself get excited at the potential. It might be more of a gamble, but the payoff is potentially huge, like any new, untested medium.
I think that artistic AI is just getting started, and it will be really exciting to see what artists can do when teamed up with coders, developers, and deep-learning networks.
Can you describe how you pair deep learning networks with animation? Do you like the process?
Yes, it’s an extremely interesting process. We have a focus on facial animation, so I’ll talk a bit about that.
Basically, we’re teaching our deep-learning networks to learn features of a character’s face, so that it can generate reconstructions of that face on top of an actor’s performance. With enough training time, the network can reconstruct the character face from any angle, as guided by the angle of the actor’s face.
Next we teach it the expressions of that face, and the transitions in-between expressions, each from every angle, and then finally what the face shapes are doing under different lighting scenarios. That’s a lot of compute, and as you can imagine, learning each one is no easy feat, especially when all of them are converging to make a whole animation.
But when the network has all this information learned and can start generating novel results, that’s when the artistry comes in, and the actor is able to drive the character animation, knowing that their face will create beautiful animations when paired with the deep-learning network..
Now, an actor can play any character, regardless of size, age, ethnicity, and have the deep-learning network use the actors performance to generate the character animation on top. Finalize that with some compositing and final artistry, and you have a very liberating artistic experience to create world-class animations without traditional CGI or motion capture.
Tell us a little more about what inspires you to do this work?
Standing on the shoulders of giants. I think it's a really incredible thing that we live in a world of instant communication where we get to witness so many accomplishments of others. It makes me grateful that even though I only have this one life and don’t have the time to accomplish everything, I can still celebrate and witness other’s doing it.
Unfortunately I don’t have another lifetime where I can dedicate myself to Jiu-Jitsu, or coding, or oil-painting, but I can still see what a lifetime dedicated to that craft looks like, by celebrating the accomplishments of someone who has. And I hope to have the same contribution, and to work with many more great minds, to create meaningful and inspiring work.
I’m inspired by the prospect of so many people, all striving on their own path to mastery, coming together and creating this incredible thing we call human culture. I hope my work can contribute.
What’s your ultimate goal?
I have too many to count! It's hard to pick an ultimate one, but conceptually I’d like to help make a contribution to human culture, and I think the best way for someone like myself to do that is through visual storytelling and design. My hope is that I’ll create something brilliant, something that gives someone an amazing experience, at least once in my life.
Any advice for ambitious artists out there?
I’m nervous to write this down, because it would be easier and just as important to write something along the lines of ‘follow your passion’ (excellent advice, by the way), but I’m actually going to give some advice that I need for myself:
Fail. Fail as often as possible, because that means you’re trying. And if you’re doing it right, a failure is only an indication that you tried something, pushed it to its limit, and found that there is more to learn and a better way to do it. Use failure as a catalyst to learn where you went wrong.
Personally I don’t like failing, but I also recognize that failure is very key to ambition, because it's the fastest way to learn. It’s built into the process; you will of course fail way more than you succeed, unless you’re just exceptionally lucky (and luck has its limits).
The only reason I have some success in my life is because I failed 10 times as much as I’ve succeeded, and it was through that process of failing that I found the way to success. I absolutely hate that feeling of failure in the moment, but I recognize that it was that pain that pushed me to try something different, to keep learning, to find other ways to achieve my goals. So for any ambitious artists, and for myself, please, keep failing; there’s no faster, or better, way to succeed.
And if you can, (advice for myself!), just try to enjoy the process.
See more from Xander here: