[wordup] James Cameron on "Avatar" and 3D cinematography

[Adam Shand]

I got to be involved in this while I was at Weta and some of the
technology that is being developed is incredible. It's going to be very
interesting to see how it plays out, from both a technology and an
adoption point of view.

Adam.

From: http://www.businessweek.com/magazine/content/07_14/b4028005.htm

James Cameron on the Cutting Edge
APRIL 2, 2007
COVER STORY

The director of Terminator and Titanic explains how movies will be
transformed by motion-tracking and 3D technology

Three-time Academy Award-winning director James Cameron is a pioneer in
the field of motion capture. In the mid-'90s he used the nascent
technology to create the massive crowd scenes and stunts in his
blockbuster Titanic. These days he's still at the cutting edge of the
technology, but he prefers to call motion capture "performance capture"
because, as he points out, "actors don't do motion, they do emotion."

Cameron is in the midst of his latest film project, Avatar, which is his
most technologically innovative film to date. The futuristic movie about
an ex-Marine will be released in 2009 simultaneously with a massive,
multiplayer, video game based on the film.

BusinessWeek couldn't catch up to Cameron for a sit-down interview,
since he's busy creating Avatar, but reporter Aili McConnon was able to
engage the director, via e-mail, in a discussion of how motion-capture
technology has spurred innovation in cinema and made filmmaking more
cost-effective. The following are excerpts from their virtual conversation:

What has motion capture meant to the film industry and to your work?

Performance capture (Perfcap) in recent years has enabled such stunning
[computer generated] characters as Gollum (in Lord of the Rings parts 2
and 3), "King Kong," and Davy Jones (in Pirates of the Caribbean) to be
brought to life. The technology is critical to the realization of my
dream project, Avatar.

In fact, Avatar wasn't possible when it was first written 11 years ago,
and only through pushing the technology to new levels over the past year
and a half have we reached the point where the film is finally possible
to make.

What innovations have you developed for Avatar?

We have greatly enhanced the size of the performance-capture stage,
which we call The Volume, to six times the size previously used. And we
have incorporated a real-time virtual camera, which allows me to direct
[computer-generated] scenes as I would live-action scenes. I can see my
actors performing as their characters, in real-time, and I can move my
camera to adjust to their performances.

In addition, we have pioneered facial performance capture, in
conjunction with our visual effects partner, Weta Digital. This
technique eliminates hours in the makeup chair, and various other
discomforts, for the actors. Previously, actors needed to have hundreds
of tiny spherical markers glued to their faces, and they couldn't touch
their own faces throughout the shooting day as a result. With the new
system, a lightweight head-rig can be donned minutes before shooting.

We have had great success, and other filmmakers such as Steven Spielberg
and Peter Jackson have worked on our virtual stage doing tests for their
upcoming films, and given high praise to the system.

Does the rig cover the whole head, including your face? Does it capture
fine facial movements?

The rig is a small skull cap, made from a cast of the actor's head, so
that it fits comfortably while being tight enough to avoid shifting. It
acts as a base for a strut which resembles a concert microphone
(visualize Madonna in concert), except instead of a mike in front of the
face, it has a tiny camera. The key to it is the software, which
interprets the movement of the actor's face, pupils, and eyelid
responses as the image flows in from the video feed of the head-rig camera.

In what directions do you see the technology going in the short term?

Improvements to the software and higher computation speeds and storage
densities will enable us to have more realistic environments and more
refined facial emotions and hand movements. Hand movement, for example,
is still at a crude state.

On Avatar, we're working on-stage at a reality level equal to an '80s
video game. At the end of the day, after a year and a half of post
production, the images seen by audiences will be 100% photo-real, i.e.
indistinguishable from photography. But for our day-to-day shooting, the
image can be improved a lot.

Another area which needs improvement is the lighting. We need to improve
its ability to handle cinematic lighting, the casting of shadows and so
on. All of this can be improved as Moore's Law raises the speed of
processing and as upgrades to the software become available.

In addition, we're developing ways for [computer-generated] characters
to interact with actors who are being photographed on real, live-action
set. We will have real-time stereo (three-dimensional stereoscopic, or
3D) composites of characters, which will be viewed by me in the eyepiece
of the camera while I'm shooting a live-action scene. This will be
revolutionary. We're not quite there yet, but we hope to have that by
August, in time for our live-action shoot in Wellington, New Zealand.

Long term, what do you expect?

I expect that more filmmakers will embrace the technique and apply it to
different types of scenarios. For the creation of fantasy and
science-fiction characters, Perfcap will largely replace makeup and
prosthetics.

Actors need not feel threatened by this change in technology. It doesn't
replace acting, in fact it's designed to empower the acting and
directing process, as opposed to the traditional [computer-generated]
animation process, which uses only the actor's voice, and in which a
committee of animators perform the character, operate the camera, and do
the lighting.

I believe it will make fantasy filmmaking much more user-friendly for
filmmakers, actors, and studios, and ultimately bring down costs. It's
just now possible to create photo-real human [computer-generated]
characters, but it isn't cost effective.

Many other fields, from medicine to automotive design, now use similar
motion-capture systems (though on a smaller scale). Do you ever run
across or dream up non-entertainment applications yourself?

I'm bore-sighted on the cinematic process. While one can generally
imagine all the industrial and science applications, I'm not interested
in developing them. However I can visualize a number of uses for the
technique in advanced forms of entertainment, at theme parks and so on.

What role will 3D play in the future of film?

Here's what can happen, although it's too early to say if it will: 3D
can become ubiquitous as digital cinema replaces film. As digital cinema
rolls out, stereo follows—and in some cases leads the charge, as we have
seen recently with the digital 3D releases of Chicken Little and Monster
House forcing the installation of hundreds of new digital projectors.

There will eventually be major titles available from all studios at some
screens in almost all multiplex cinemas worldwide. I would say the
horizon for this is five years. 3D can become a fully accepted way in
which audiences view movies. It will become another consumer choice,
like premium or regular gas. The premium experience of 3D will be the
preferred viewing experience for action, animated, fantasy, and
science-fiction films.

3D's broad acceptance at theaters will generate enough content that
consumer-electronics manufacturers will make home players and monitors
available. The technology exists now, but is not readily available as
off-the-shelf products. 3D display will become a must for video and
computer games.

In 20 years, stereo media may become the preferred method for displaying
all information, including news and other broadcast media. The density
of information one can place on a small screen becomes much higher if
it's stacked in three dimensions.

Is there something beyond 3D in film? Could we ever see in cinema the
same kind of physical participation we're starting to see in video-game
consoles like Nintendo's Wii?

Imagine a movie in which the viewer is swept along by a narrative,
following the action from place to place, but without the intervention
of a camera. You can choose which character to watch in a scene, as if
you're an invisible witness standing there while a real event plays out.
This is still years away, at a level of realism people would consider
cinematic, but certainly not decades away.

I can imagine the dense fantasy worlds I like to create for movies
having an equal or greater life in a world of interactive play, authored
by others, in a partnership. Of course, add massive multiplayer
capability to this, and people will never leave their homes.