Making It Real: The Future of Stereoscopic 3D Film Technology
In this initial feature for the launch of SIGGRAPH Quarterly’s online magazine, Sony Pictures Imageworks’ Rob Engle and Rob Bredow discuss the subject of stereoscopic 3D film production and presentation, and offer their ideas as to where this increasingly important technology may be heading in the future.
Article author: Eden Ashley Umble All images courtesy of Sony Pictures Imagesworks unless otherwise stated |
||
![]() Combined left & right eye final shot - IMAX |
||
![]() Left eye camera render |
![]() Combined left & right eye camera render |
![]() Right eye camera render |
|
Films using a variety of 3D technologies - from 3D
animation to stereoscopic Real-D and IMAX 3D presentation - have grown
in popularity with both audiences and filmmakers in recent years. For
audiences, the 3D experience can provide thrills that are visceral as
well as visual, while filmmakers are using this technology to tell
their stories in a way that is more immediate, more detailed, more real
than ever before, allowing them to push the boundaries of filmmaking to
the limits of their imaginations. |
![]() Red & cyan filtered glasses |
![]() Crew: Sony Pictures Imageworks' IMAX 3D artists and staff |
||
Engle was the Digital Effects Supervisor on the groundbreaking adaptation of Robert Zemeckis' theatrical 2D film "The Polar Express" for IMAX 3D, which was the first feature-length all-CG project to be created in stereoscopic 3D (and at 96 minutes, the longest IMAX 3D film ever made). A separate team of 60 artists and support staff worked for 6 months to create the IMAX 3D version, all while the 2D film was still being finished. The IMAX 3D version of "The Polar Express" opened on November 10, 2005, the same day as the theatrical feature, winning critical acclaim and setting box office records for an IMAX attraction, grossing $35 million domestically on just 60 IMAX screens. Bredow was a Digital Effects Supervisor on the theatrical release of the film and was instrumental in the early phases of testing the viability of the IMAX 3D project. |
||
![]() Rob Engle |
Engle is presently supervising the IMAX 3D version
of "Monster House", which will be released using the Real-D
stereoscopic projection system. The second film to employ the
Imagemotion(TM) performance capture innovation developed by Sony
Pictures Imageworks, "Monster House", directed by Gil Kenan, will be
released July 21, 2006. |
![]() Rob Bredow |
The demand for stereoscopic 3D films is growing. What 3D projects are currently in production at Sony Pictures Imageworks? Rob Engle: We have "Monster House" in REAL-D [set for release July 21st, 2006] and "Open Season", set for release Sept. 29, 2006] as an IMAX 3D film. Rob Bredow: Basically, I don't think there's a project in house where 3D isn't discussed. There are thoughts about just about every one of our shows because of the renewed interest in 3D across the board. I think this is certainly in part because of the success of the IMAX 3D "The Polar Express" . It's becoming something that's on everybody's mind for every one of our projects, particularly our CG features; it looks like every one of our CG features could possibly have a 3D version. I think it's interesting to look at the different marketing angles that have yet to be explored, in terms of having a 3D version of your movie in combination with your 2D version, and how you release those. Day and date is how we've been delivering so far and that's cool because you get great word of mouth on the 3D version since it's a phenomenal experience, and then the nice thing is, if the IMAX theater is sold out, they can still go see it in the 2D theater. In the case of "The Polar Express", by all accounts the 3D version added significantly to revenues. How are Stereoscopic 3D films perceived by our eyes? Engle: There are really 2 phenomena that the eyes use to perceive depth. One is where they focus, meaning specifically at what depth our individual eyes are adjusted to see, just like a camera lens focuses. The other aspect is called convergence, which is basically the phenomena where your eyes cross or uncross in order to bring two similar features in an image together. You are constantly adjusting your convergence and your focus to tell you the relative depth of objects. That's how we see things in the real world. In stereoscopic films, where they're projected flat on a screen, your brain is being asked to separate those two phenomena. It's being asked to focus on a fixed point, which is usually 20 to 30 feet away, and then converge independently. |
||
![]() Imageworks' crew |
||
How does stereoscopic film projection differ from conventional film presentation? Engle: Normally, we would render a single camera
point of view and project that and that's how you see a CG film in a
theater. To create a stereoscopic movie we render two viewpoints and
project both of those viewpoints simultaneously. What's important is
that the projection system is capable of delivering an independent
image to the left eye from the right eye. There are a wide variety of
technologies out there for achieving this, and almost all of them use
glasses in one form or another. In active systems the viewer wears a
set of glasses that have electronically triggered filters over each
eye. The filters switch between opaque and transparent in synch with a
projector that is alternating between the left eye and the right eye.
In passive systems the viewer wears glasses with fixed filters, which
have some unique property per-eye that selects the image to pass. For
example, with the anaglyph system a red filter allows only one color
light to one eye while a cyan filter allows the rest of the light to
the other eye. The more sophisticated IMAX and Real-D systems use
color-neutral polarizing filters to select the left and right eye
images. There are also some systems out there that are called
autostereoscopic displays where you don't have to wear the glasses at
all. |
How do the experiences of viewing IMAX and Real-D differ? Engle: I think the biggest difference is that when you're in an IMAX theater, you're usually immersed in the screen without even looking at any content. Once you sit down, it takes a good turn of the head to look from one end of the screen to the other. The result of that is that you generally feel like you're in the image on an IMAX screen, more so than you are in a conventional theatre screen. The Real-D system currently is targeted at multiplex type theaters with 40 to 50 feet wide screens where you can see the edges. What that means in terms of the experience for the audience and for the way in which you create the content is that an IMAX theater can be much more immersive, and of course, that's why they call it "the IMAX 3D experience". Contrast that with a multiplex theater, where it's literally as if you were looking through a window and experiencing a deep world. We're capable of pushing things out of the screen but that effect really depends on how things are composed. Fundamentally, IMAX will feel like you're more in the world, and multiplex Real-D will feel like you're watching the world. Both IMAX and Real-D offer compelling 3D experiences for their audiences. As co-creators, Imageworks is always trying to find the best way to match the director's vision to the best use of stereoscopic presentation. Sometimes that will mean IMAX and sometimes it will mean Real-D. |
![]() Cross section view of a typical IMAX theater |
|
How was the IMAX 3D conversion process accomplished on "The Polar Express"? Engle: For the 2D theatrical release of the movie,
[the main Imageworks filmmaking team] would produce the content in the
first place, and they did that though a process which was basically a
combination of Imagemotion, performance capture and hand animation to
produce the final shot. The 3D team would pick it up from there. We
would start by looking at their final animation files and we would
produce a stereo camera that matched their camera as much as possible.
We were trying to preserve their movie without changing the composition
of their shots if we didn't have to. We wanted to be as faithful to the
original as possible. |
![]() "Polar Express" - Sony Pictures Imageworks (combo image) |
Can you talk about working with a filmmaker as creative as Robert Zemeckis? Bredow: It's interesting to get to work with somebody who's obviously experimental and innovative in filmmaking. Basically, when you get to work with someone like Robert Zemeckis, his focus is to be able to tell stories with whatever means are available to him. When he sees an actor like Tom Hanks, and wants him to play a six year-old kid, you start to realize the extremes that he's interested in going, to be able to tell his stories, which makes it a lot of fun. That starts with things like acting and characters and who's playing his main characters, and goes all the way to technical innovations in terms of how to make the audience experience his movie firsthand in 3D. That was my experience on "The Polar Express". Are there any striking differences between the pipelines created at Imageworks to process IMAX and Real-D? Engle: The primary difference between the Real-D and IMAX pipelines are in the ways the cameras are created. With a Real-D presentation you need to be more aware of how the edge of the screen can interfere with the 3D effect. You need to adjust the overall depth of the scene into the screen plane. Can you talk about the look and production of "Monster House" in Real-D? |
![]() "Monster House" from Columbia Pictures will be presented in Real-D |
Engle: I think "Monster House" is a unique creature in respect to its look. The look of the film is very much like a Claymation miniature, and there's a lot of use of global illumination, bounce lighting and very intricate shadow detail that wasn't used as much on "The Polar Express". It's a very different look. In the Real-D world, in a multiplex, it's more of a window environment, and we're trying to direct that much more carefully on "Monster House" than we did on "Polar". We'e using a different renderer and a different lighting package on this show, so behind the scenes there's a lot going on to make sure we can do this show, but the basic concepts are the same as "The Polar Express". Should all films be in stereoscopic 3D, or do some films possess characteristics that specifically call for a 3D viewing experience? Engle: I think that CG features have a special quality which lend themselves to stereoscopic presentation I look forward to seeing a few live action blockbusters in stereo but do we really want to see every film in 3D? Maybe I'm too old, but I saw a clip of a classic movie musical not too long ago which had been converted to 3D and I thought to myself, yeah that's neat, but there's something to be said for leaving those historical gems alone. Maybe it comes back to the whole question of colorizing a film, for example, do you do it or not? If you did a version of "Casablanca" that was in color and 3D, it would just be a different movie; I mean, why bother? I'm generally against it. I would much rather leave it alone and let people enjoy it for what it was. I suppose there's the other argument that says if you make it 3D or in color, then it reaches a new audience that wasn't there before, but I like to think that people are cleverer than that. Bredow: I think "The Polar Express" was an example of a movie that was particularly well suited for 3D. It was "stereo friendly" for a lot of reasons. Robert Zemeckis loves to move the camera, he loves to use really wide camera angles, long shots, and all those things are great for a 3D movie. I think that's one of the reasons it was such a good showcase piece for this kind of film. Not all films are going to be as well suited as "The Polar Express". Interestingly, I think CG features generally have a better shot than your average live action movie, just because of the way that they tend to be cut; they tend to not move at the same sort of pace of some of the more fast-cutting live action movies. On [the IMAX 3D version of] "The Polar Express", what was so successful for me personally when I went to the movie theater and watched it, was that it was so immersive. By the time the movie had come out I had seen the 2D version a lot of times, but seeing the 3D version was honestly like seeing another movie. There's something so immersive about that, especially when you've got the opportunity to do stereo and a huge screen. When you can fill the audience's peripheral vision, it really does do something different in terms of putting them inside the movie, which is fun. How do you see 3D technology being applied to everyday communication and other consumer applications in the future? Engle: I certainly think the technology is moving forward to the point where we will have autostereoscopic displays in the consumer's hands. Right now, they're very expensive, but it will happen. The most interesting question in my mind is whether or not we can make use of the third dimension to make computer user interfaces more accessible. Imagine if the desktop on your computer was actually dimensional. Would it be much more cluttered, or would it be better organized? I don't know, but I certainly think if people haven't experimented with it, they should be. As far as how it's going to change communication, I wouldn't be surprised if someday maybe everybody will have a stereo camera phone and you can have a stereo telephone conferencing. I don't know, but I certainly think that 3D will get better and cheaper, and as a result, it'll be everywhere. Bredow: I think what everybody thinks of when they think of 3D, of course, is the glasses and the hassle that they can be. Anytime you have to put glasses on to carry on a conference call or something like that, I think that that's something that people are not going to choose to do very often. There were a lot of different things in SIGGRAPH this year, various monitors that did some sort of 3D experience, like a single monitor without glasses, and actually all of them had the disadvantage of they weren't very sharp or very detailed, but it was interesting to see various prototypes. Most of them were showing short little test animations, or things that had been acquired in 3D, or still images. When you can perfect the idea of not having to put on glasses, I think you're going to see more broad application. In terms of integrating it into other areas of people's lives, it's a good question. And the simple matter is, it costs at least twice as much as your standard projection setup, just because you're going to have two projectors instead of one, in most setups. Can you imagine, for example, people buying Real D glasses in the same way they purchase their reading glasses now? |
Engle: It's funny that you mention that, because
I'm always joking with our digital production manager that I want to
get polarized contact lenses, so I don't have to take my glasses off. On the future of 3D as a growing visual effects process, can you see Real-D technology being used in a classroom application soon? Engle: Absolutely. I think not necessarily specifically Real-D, as much as the more it becomes commonplace, I can certainly see it being used in the classroom. What I find interesting is the use of the technology to produce compelling content, for example, imagine National Geographic specials shot in stereo where you feel like you're actually in the lion's den, as opposed to just seeing it. I personally think that would be amazing. Of course, if you've seen any 3D IMAX films, you know that you feel like you're there. |
![]() |
Bredow: It's not impossible, but just
generating the content for stereo rather than 2D is easily twice the
work so when you're talking about a Power Point presentation or
something like that, there has to be a really specific reason that
you'd need to present it in stereo just to get your point across. It
may not be worth the time. Just having, for instance, fonts floating
over a background probably wouldn't be worth the effort, whereas if
you're trying to describe something that's inherently 3D, then that
could have some payoff, for sure. Real-D specifically is a very
high-end theater based system. The projector's the size of a normal
film projector and the cost of course is up there too. But the basic
concepts behind the technology which are similar between many of the
different 3D options, with polarized lenses, 2 projectors with
different sort of polarization and a special screen -- there's nothing
keeping anybody from setting that up with a few thousand dollars' worth
of hardware in a classroom setting and very cheap polarized glasses.
It's technically feasible. How practical is it in terms of generating
the content, that's probably the biggest question. Do you think stereoscopic 3D technology will soon be a viable addition to current curriculum for digital art and animation students? Bredow:With the trend of a lot more movies going into 3D, there are good opportunities from the education side. It would be great to have more education about the way stereo works, the way our eyes perceive 3D, and the way you can trick the eye with various techniques, [such as] whether you aim their cameras toward a focal point, how you handle depth of field, etc. There's lots of different ways of thinking about these kinds of things, and there's some good research out there, but I don't think there are lots of people covering that currently in schools because the recent popularity of the medium is a relatively new thing. 3D in general is an interesting topic for people to explore. Another |
|
interesting opportunity from the educational
perspective, and for people just getting into the industry, is that
often the 2D movie is done by a team of people and then the 3D team
comes along and picks up the assets and makes the stereo version.
The first time that happened to us, it was simply by necessity of the
schedule. We had our hands full making the 2D version of "The Polar
Express" and late in the schedule, they decided to make the 3D version,
so we started a whole other team to do it. It turned out that that was
actually a pretty efficient way to work. One of the good opportunities
for people from an educational standpoint is there's a higher number of
positions that require less experience across the entire board, who can
work on some of these shows. [A stereoscopic 3D film] can be one of the
first shows that people might get to work on who don't have the
traditional 5 years' experience in feature film. So, one nice thing
about these shows is they do create some entry level positions. Engle: Absolutely. In general, with the introduction of any new technology there is the need to train people on the best way to use it. Creating films for stereoscopic presentation is not a new field but, with the wider availability of 3D venues, there will be a stronger demand for good 3D content. |
![]() |
About the author: Eden Ashley Umble is a writer based in Northern California, where she lives with her husband and two kids. She worked in film production and publicity for fifteen years on films such as "Edward Scissorhands", "The Long Kiss Goodnight" and "Fat Man and Little Boy." |