A Tribute to Performance Capture

Movies have been around for over a century, and special effects have certainly come a long way in that time. The last two decades or so, in particular, have given way to numerous advancements in entertainment spectacles. One of the most impressive of those is motion capture, more appropriately known as performance capture. Basically, this involves an actor donning a skintight suit and being covered in reference dots. Numerous cameras are then pointed at the performer while he or she acts out a scene. The movements and/or facial expressions are then recorded and can be applied to a computer-generated creature, creating a living, breathing character. This may sound like a hindrance at first. After all, why don’t they just create the character on the computer and let the actor give a vocal performance later? Well, in actuality, this technique often yields a much more tangible and realistic result. Think about it. Instead of the visual effects artists having to animate the character frame-by-frame and hope that it looks real, performance capture provides the movements for them and allows the actor to inhabit the role more completely. However, it hasn’t gone without some evolution over the years.

One of the first major motion pictures to utilize this tool was The Lord of the Rings: The Two Towers. Director Peter Jackson and his team were faced with a challenge: bringing the wretched creature Gollum to the big screen in a way where audiences would see him as a character rather than as an effect. This problem paved the way for the most prolific pioneer in the performance capture industry: Andy Serkis. When he was initially hired for the role, the plan was for him to only provide Gollum’s voice. However, in order to channel the eccentric nature of the character, he would get his whole body into it, spitting, drooling, and contorting himself into all kinds of different positions. This, combined with his desire to play off of his fellow actors, convinced the filmmakers to put him in the scene, which then allowed Elijah Wood (Frodo) and Sean Astin (Sam) to give better performances due to actually having someone there to interact with. Serkis would then redo the scenes in a studio so that the computer could pick up his movements and expressions to apply to the CG model. The result was simply amazing. To this day, Gollum is viewed not only as a landmark in visual effects, but as one of the most memorable characters in the series.

This technology became so instrumental in the success of the Lord of the Rings trilogy that Jackson used it again in his 2005 remake of King Kong. Once again, he turned to Serkis to convey the animalistic actions of the titular gorilla. Just like before, many viewers found themselves sympathizing with a creature that could have easily been seen as just a mindless movie monster. If audiences could empathize with a CG gorilla, why not apply the same technology to other forms of art, like video games?

Over the years, developers have sought to get their players invested in the narrative in addition to the gameplay. With many games becoming more cinematic in quality, performance capture has grown into a useful tool for studios. Although this technique was implemented in fighting games like Virtua Fighter 2, more recent titles have begun telling their stories with mo-cap actors in a soundstage. The detective title L.A. Noire uses a tool called MotionScan to record an actor’s exact facial movements from every angle. This is then implemented into the gameplay. During interrogations, players use these expressions to figure out if a character is lying. As creative as that is, some of the best examples motion capture in gaming come from Naughty Dog, the developer responsible for the Uncharted series and The Last of Us. These games do a great job of keeping the player involved while delivering high-quality storytelling. Cast members like Nolan North and Ashley Johnson, instead of simply providing the voice, are able to inject their own subtle nuances and mannerisms into the characters. They inhabit the roles in a way that every working actor wants to. Even Andy Serkis has dabbled into the world of video games, performing in the titles Heavenly Sword and Enslaved: Odyssey to the West.

As such, it’s really no wonder why Robert Zemeckis, best known for directing the Back to the Future trilogy, went on to make several films realized primarily through performance capture. Such works include The Polar Express in 2004, Beowulf in 2007, and A Christmas Carol in 2009. While the quality of some of these films has been called into question, there is no denying that there aren’t many other movies like them. Unfortunately, this brings up a potential downside of performance capture: the uncanny valley. This is basically when a machine or a computer-generated recreation of a human makes audiences feel uncomfortable. It’s that eerie area between real and fake. A digital character may look and act the part, managing to be almost lifelike, but something about it just gives away the illusion. Instead of being awe-inspiring, the finished product is just off-putting. In the case of The Polar Express, many critics pointed at the eyes as being the cause of this, with the humans coming across as soulless zombies.

Some performance capture films manage to avoid this. Steven Spielberg’s The Adventures of Tintin was very detailed in its visuals and movements, but the filmmakers also opted for a stylized approach, designing the characters to look somewhat cartoonish to pay tribute to their comic book roots. Additionally, the film was produced by Peter Jackson and animated by Weta Digital, the company responsible for the effects in The Lord of the Rings and King Kong. As such, they had a firm grasp of the technology.

In fact, said technology had come so far by this point that filmmakers could now shoot the mo-cap performers on the same sets and locations as the other actors without having to redo the scenes in a soundstage. This was beautifully demonstrated in the three most recent Planet of the Apes movies, Rise of the Planet of the Apes and Dawn of the Planet of the Apes. The apes in the previous entries had all been realized through makeup, prosthetics, and masks. While there were certainly advantages to this, switching to motion capture allowed these animals to move and express themselves in ways that audiences had never seen before. As the apes steadily grew more intelligent over the course of the narrative, we could connect with them more due to their behavior and mannerisms slowly becoming more human. Making the experience all the more complete was the involvement of Andy Serkis, turning in yet another stellar mo-cap performance. Much like how Caesar led his fellow apes to freedom, Serkis led a slew of other motion capture actors in portraying those apes. More and more viewers started to see this technique as a new form of acting and a meaningful contribution to the entertainment industry. In fact, many people actually voiced their desire for the Academy of Motion Picture Arts and Sciences to recognize Serkis for his work and grant him an Oscar.

Performance capture was now more popular and polished than ever, so much so that it could be used far more extensively in Peter Jackson’s grand return to Middle-earth. In the trilogy of prequel films based on The Hobbit, not only did Serkis reprise his role as Gollum, but there were also several other characters realized using this tool. Manu Bennett was menacingly intense and physically imposing as Azog the Pale Orc; Barry Humphries lent a sly wit and condescending regality to the Great Goblin; and Benedict Cumberbatch’s serpentine movements and beastly voice perfectly suited Smaug the Dragon. Again, these could have been just digital creatures for the heroes to fight. However, utilizing performance capture helped the filmmakers bring them to life as characters, giving them real weight and presence.

That is ultimately why I’ve been an avid supporter of this technology for years. Like many other aspects of the entertainment industry, there is a genuine art to using such a tool. To me, motion capture has always been another form of traditional makeup and costumes, offering actors a new outlet in which to test their talents of expression. It’s an intriguing and revolutionary technique that will hopefully be used to bring many more characters to life in the future.