motion capture and real-time production

James Cameron's "AVATAR"

After years of working for smaller commercial houses at the start of my career, "AVATAR" was my very first feature film. Getting a chance to work for James Cameron's motion capture studio was a dream come true. Luckily my foot in the door was his company Lightstorm was looking for Motion Builder artists, which I've dabbled in before, so I decided to apply. Immediately they had me start and I was able to jump right in! Throughout my time at the studio I got a chance to work on several aspects of the production. My biggest contributions were on the "Thanator Chase" sequence which I feature in this breakdown.

Performance Capture Layout

Very early on I had a chance to work with the Performance Capture Layout team. It was awesome to be able to look out at the stage every day and see the motion capture process with the actors happening right in front of you through the upper bay windows. It was our job to manage the layout of the motion capture digital set. We were almost like live stage hands in a way. We were in charge of the digital set in Motion Builder all working in real-time. If Cameron wanted a tree moved because it was effecting his shot, we would move or delete it. If Cameron wanted us to adjust anything, we would be able to make the change in real-time and it didn't need to hold up the shoot. Obviously me saying that sounds like it could be a tense situation, and believe me it was! As soon as you were asked to adjust anything you had to be quick. Below I share the raw footage of the motion capture session that I was involved in producing.

To explain a bit more about the Performance Capture process, listen to this interview from the behind the scenes documentary. In it you'll see how much effort was put into the actor's motion capture performance and what actually made it into the movie. Pretty much everything we established at Lightstorm in the performance capture and previs of the set had to be retained when it went to Weta. Cameron was very adamant of that. It should be the director and actor's intent and not an animator's interpretation of the scene.

Data Processing

After the performance capture of the actors is complete, then the data needs to be processed. I was also on the team that would take the raw motion capture data and clean it up for Cameron to review. Sometimes when the motion capture data wasn't perfect, that is when we needed to reanimate some of the elements on the rigs. Predominantly cleanup would happen on elements like the hands. In our scenes we were also in charge of taking the recorded face reference from the performance capture shoot and project it onto the digital avatar. Sometimes if the lip-sync was off we would need to correct it. Also during this process we would make modifications to the set layout if it were requested by our CG Supervisor. The intent of the data processing would be to get it ready for the next phase of the process which I'll cover in the next section. Below I share what the processed data looks like out of Motion Builder.

Camera Tracking

The next part of the process is when Cameron would go out on the motion capture stage, without the actors present, to reshoot all the cameras. This was after we cleaned up the motion capture data, projected the actors faces onto the avatars, and made the layout adjustments he requested. This gave him a much cleaner scene to work with to focus on final camera polish. During this process we might be asked to make more adjustments to the sets, which the turn around time would need to be quick.
‍
After Cameron completed reshooting the cameras, I also assisted in running our studio theater controls to review the camera footage with our Director of Photography. This control setup involved running the live Motion Builder session as our DP watched on the big screen. In this session he would instruct me to modify the camera keys and adjust the motion curves. We would also review in Stereo to adjust the separation of the 3D cameras to either push or converge the 3D effect. Afterwards when we completed our cleanup session, I would run the controls to play it for Cameron. Speaking of nail biting situations! lol

Below I share another behind the scenes interview that explains this process in more detail.

The last thing I'll share is showing you what Weta's amazing final renders ended up looking like and compare it to the motion capture work we made in the beginning. As you can see, Cameron made sure the original performances from the actors made it to the screen, and the results look incredible! I am so proud of the work we made on this ground breaking film. It truly changed my life since then. I am so thankful to have been a part of it!