Red Alert: VFX Supervisor Richard Stammers talks about The Martian
All images copyright 20th Century Fox
All images copyright 20th Century Fox
VFX Supervisor Richard Stammers discusses in-depth the effects created for Matt Damon's latest blockbuster, The Martian...
Going into outer space with British filmmaker Ridley Scott (Blade Runner) is nothing new for Richard Stammers who supervised the visual effects for Prometheus (2012); he does the same with the cinematic adaptation of The Martian without the presence of alien creatures. Based on the novel by Andy Weir, the science fiction adventure revolves around stranded American Astronaut Mark Watney (Matt Damon) who has to be inventive and maintain a sense of humor in order to survive on the Red Planet while NASA orchestrates a rescue mission back on Earth.
Ridleygrams, Previs & The Knuckle
"Ridley storyboarded quite a lot of The Martian off of the original script which gave us a starting point for some of the visual things that he wanted to see, one of which was the storm,” explains Richard Stammers, who had to deal with a tight production schedule which saw the whole movie completed within a year. "We did a lot of previs trying to work out the geography of where the various things, like the HAB site or M.A.V., were going to be located. But more importantly was the zero gravity work."
"Commander Lewis [Jessica Chastain] travels from the bridge to the REC Room," continues Richard, "and part of their journey is to go through this spinning intersection situated in the core of the ship called The Knuckle that has these various tubes going off to the four pods. Ridley did some storyboards and we worked it up as a previs. From that we had to inform the Art Department as to what bits of sets needed to be built, as well as what pieces needed to be removed to get access to the wires the actors would be suspended on, or to allow a camera to be put into the wall. It was also informative for what the stunt team needed to do. The previs was broken down into techvis, so we import the movement of the CG person into the stunt rig wire wench computer. Based on the previs, we had a motion-control camera move in the form of a Technodolly, that we could program so it would never cross the path of where the actors would go on their wires.”
"There were little innovations along the way, even with the way we were shooting RED DRAGON cameras for the first time in a stereo rig for a large production,” states Richard Stammers. "We had one of the largest stages in the world [situated in Budapest] with a full-360 green screen, and a ground surface that was built to match our Mars surface which we had chosen as a location in the Jordanian desert of Wadi Rum. We went there at the end of our schedule to do a lot of our wider shots. We also did multiple environment shoots of location at different times of day, as high definition 360-degree panoramas with HDRI at 20-minute intervals, and picked the time of day that would be most suited to matching our studio space. There was a single light source [on the stage] that matched up to a time of day that we had decided to base our environment build on, which was about 8:30 am when the sun was nice and low."
"We could stand anywhere on our green screen stage," continues Richard, "hold up an iPad, and use a virtual reality software that we built custom for the show, which would allow us to visualize what would be beyond the green screen, like the surrounding mountains and desert. Any point where Ridley would say, ‘I'm going to put the camera here and point in that direction.' I could stand next to him and say, ‘This is what the background is going to look like.' We could make compositional adjustments based on what the final composite was going to be without actually having it there to see for real.”
Integrating Stage with Locations
"Originally we were going to do complete scenes on the green screen stage, and shoot different ones and vistas once we got to our location in Jordan at the end of our schedule,” notes Richard Stammers. "We ended up having to shoot more scenes in Jordan or parts of scenes in Budapest on our green screen stage, and parts in Jordan. We had scenes intercutting shot-by-shot from green screen to a full live-action plate in Jordan. The pressure to make it match was high. We knew what we needed to match to when shooting our green screen, so we got a lot of the way there with great lighting."
"The limitation of having a single light source on such a big green screen stage meant it was bright at the source and quite dark the further away you got from it. A lot of the work that we had to do was balancing the brightness and the corners of the stage; where it got dark we had to do some selective matting and rotoscoping to bring up the exposure levels to blend neatly into our digital environment, that was a perfect replication of the location in Jordan.”
"Ridley had looked at a lot of various desert locations around the world as possible choices for Mars, including Australia, South Africa, Namibia, but settled on Wadi Rum,” states Richard Stammers. "The great thing about Wadi Rum is that it does have a lot of similarities with the ground texture and terrain.” The landscape needed to be augmented. "We analyzed a lot of the Mars imagery that had been created by NASA and the European Space Agency over a number of years. One thing that was apparent is that none of them were the same. Mars has dust in the atmosphere that tends to make the sky look yellow, but depending on how you adjust the white balance it could appear to be grey."
"Knowing that we were going to be shooting in Jordan," continues Richard, "it was decided to color balance these Mars images into a color temperature similar to Earth's. It informed us as to what the sky color ought to be, which was grayish with a hint of a greenish yellow, and the ground color in Jordan is similar to what Mars looks like; that gave us a good starting point to push the color further."
"We always knew when shooting on location or on-set that we had a CDL pipeline, which was being applied to shots so we could see what they looked like, and generally that was making everything warmer than it was in reality. There was red and yellow being added to give a stereotypical feel of the Red Planet that people would expect to see. Once we got the sky into a place where it was gray with a bit of warmth, Ridley and Dariusz Wolksi [our DP] with this further grade would push the sky into this golden butterscotch or coppery color.”
"We wanted to avoid rotoscoping mountains and people in order to do an extensive blue sky replacement,” explains Stammers. "We came up with a new algorithm which we called our Earth to Mars Filter. It worked differently from keying or rotoscoping or spill suppression tools. It gave us a nice, smooth, grain-free no-edge artifact way of removing the blue and colorizing it to the Mars sky color that we needed. This was a process that was developed at MPC using algorithms to write expressions to find smooth ways to remove the blue. That was an incredible important part of our process of color matching our green screen to the locations as well."
"We went through a methodical process of making sure that if anything was shot on a green screen stage we could remove the green from that. We found a shot from our location in Jordan and removed the blue from that. Then we found a way of grading the two to match together, so once the compositing work started our goals weren't so far apart from each other; they were very similar."
Some creative license was taken with the Martian environment, as Ridley Scott wanted to add clouds to the sky. "After doing a bit of research I discovered that there is a lot of cloud activity on Mars in the form of high altitude CO2 ice clouds,” reveals Richard Stammers. "They are similar to Cirrus clouds, but Ridley wanted us to give them a movement that was different and otherworldly. We created these fast-moving strata and streaks of high altitude clouds that were a combination of fluid simulations that we did in Maya, mixed with the practical element of pouring salt, which gave us different textures and organic movement."
"The fluid simulations gave us a larger scale of textures and shapes that could be blended together. These were all positioned in the sky in creative ways and in interesting patterns. Sometimes they would be like CO2 ice clouds and other times we colorized them to be browns, reds and yellow tones as if they were streaks of dust that had been caught up blowing at high altitudes. It gave us this dramatic look to the sky which Ridley was always keen on.”
"The wider shots of the storm approaching involved a full CG simulation of all the particles to create that pyroclastic cloud effect,” explains Richard Stammers. "The team at MPC used a combination of Houdini and Flowline to achieve the physical simulation that they were running to get the animation to look right. Then it was all rendered in RenderMan to get the final look. Many particles were required to get the level of detail needed in all of the cauliflower shapes in the front edge moving towards you. They are surrounded by huge mountains, and the distant volcano that the storm is coming around is meant to be enormous. We want to pitch that the storm is thousands of feet high, so the more detail we can put into there the better it looks, but with that comes a large overhead of simulation and render times.”
The Storm Sequence. which features Mark Watney getting hit and sent flying by a wayward satellite dish. was filmed indoors allowing for more control. "Ridley wanted to get it as intense and violent as we could,” states Richard Stammers. "We got 60-percent of the way there with what we were able to shoot practically. We were shooting on a massive green screen stage for our Mars exterior shots for daytime scenes, put black curtains up, lit it dimly and allowed the lights in the spacesuits to illuminate what is around them as much as possible. We did a separate element shoot to layer up between the actors and camera. Sometimes we had to put in deeper layers behind them.
We were trying to maintain the stereo quality so you get a real sense of the depth of the storm but also create an energetic feel to it. The actors were put on wires so that they always had something to struggle against. There were compositing challenges in getting the flying satellite dish in the right depth behind particles that were in plate in the foreground. There were a few full CG shots for the wider ones where we couldn't get far enough away from the actors in our stage space.”
Mars from Space
The cloud innovation assisted with the view of Mars from outer space. "When you start looking to some of the Mars references there are some quite big dust storms that happen from time to time, which create interesting patterns across the surface that tends to be soft,” notes Richard Stammers. "These thin streaks of clouds, when you see them from space, gave us interesting textures and dimensions to the surface of the planet. We weren't just dealing with a dusty textured sphere. We had this extra depth to it, which allowed us to put cloud layers that cast shadows and gave Mars a beautiful look from space.”
Relief maps of Martian surface from NASA were valuable reference material. "From that, Framestore was able to build a fairly detailed model of Mars that was geographically accurate,” explains Stammers. "It had craters and all of the trenches in the right places. There is one huge trench across the surface of Mars called the Valles of Marineris which is featured in the backdrop of the Rescue Sequence. It was important to get the most beautiful part of Mars. We went through a lot of careful planning through the whole sequence to make sure that geographically we were always telling the story of the planet rotating, and the Hermes and M.A.V. orbit around Mars at a high speed. Framestore had to take this texture map, which gave them a relief of Mars, and enhance it so we were getting good enough details to see in a lot of our close-up shots or tight shots, where we were seeing a specific area of the surface. That was further augmented with digital matte paintings and layering CG cloud layers over the top, and casting shadows onto the surface which was needed to get a nice realistic quality of Mars.”
Martian Vehicles & Habitat
"The actual Rovers that you see driving around the surface of Mars and around the Hab are practically built by the vehicles team of the Art Department,” remarks Richard Stammers. "We did have to do digital versions of those for some of the high helicopter shots where it wasn't always possible to get the Rover where we needed at the right time. We either shot plates of the Rover that we cut out and put into other shots, or we did a full CG Rover for some shots. We had the Rover driving on our green screen stage in Budapest and we shipped it out to Jordan as well.”
Stammers adds, "There was a partial set build for the M.A.V. On our green screen stage we had the tripod base of it, which is the bit that gets left behind when the crew leave Mark Watney on Mars, and the leftover fuel in the hydrogen tanks which allow him to make his water. They were there as practical elements that Matt Damon could interact with and use as a set piece. It also gave us something we could drop a ladder down from to see the crew climb up into, and we also had separate sections with tubes through the middle of the M.A.V. as a different set, and the capsule at the top of the ship was another set piece. The bit in the middle that we see in the outside is all CG. The Hab was a fantastic set. A lot of the equipment was practical so we had very little work to do in there. We had the occasional monitor replacement and views out of windows but there weren't that many of them. Outside of that, for the depressing moment when all of the potato plants get ripped out of the ground, we did some augmentation there.”
"Mars has 40-percent of the gravity of Earth, and Ridley wanted to try to find a way to feel that,” remarks Richard Stammers. "He had Matt walk slightly faster than normal, and overcranked the camera. It gave lightness to the way Matt walked. We found that the optimum speed for that was either 32 fps or 36 fps; however, we were shooting with stereo rigs that couldn't sync at that speed so everything was shot at 48 fps, resulting in having to do a lot of stereo retimes. Stereo retimes are typically difficult to re-interpolate. When left and right eye do the same thing they end up with various artifacts in places that need to be cleaned up. We retimed the original plates and sent them back to editorial for approval."
"In the meantime, we had to start with the match move and triage process, which is doing the stereo plate alignment and color matching between the two eyes. We worked on the original plates for those and would retime our match move camera to match our new plate of the slightly faster speed. We would then progress with the compositing that way. But all of that put a massive overhead on the starting point of all of the work we were doing. In some cases we had been working for weeks to get one plate ready to start the visual effects work. We ended up with 160 retimes of stereo shots of Mars.”
The Hermes spaceship was influenced by NASA vehicle designs for a long-term mission to Mars, as well as the modular structure of the International Space Station. "As soon as we started shooting, Framestore was already working on building the ship,” states Richard Stammers. "They had the experience [with movies like Gravity] in the complexity with the rendering and the way light bounces off of every surface. All of those things take a huge amount of time with such a heavy model.” The interior of the Hermes was divided by the sections that did or did not have gravity. "In our Gravity Wheel there are these REC Room, living quarters and gym which were redressed from the same set. Even though you are standing on the floor and have simulated gravity within it, the view outside needed to be spinning."
"Another challenge was wherever you saw the light and sun, that they were also spinning. We had to have a light that was constantly moving on a crane that was operated by a grip, who gave us a rotating movement that allowed the shadows across the set to be constantly moving. We made sure that our CG background tied in with the position of where that sun was, and that would inform the CG extension of the Hermes that you see out the window. Beyond that you also have a star field that is also spinning. The other parts of the ship when they are in zero gravity, such as the corridors, were partial set builds. Quite a lot of the interior shots required digital set extensions.”
"We had a real challenge with this rotating environment once you're in the capsule with Mark Watney; he is spinning and trying to get the movement of light as practical as possible. Dariusz Wolski brought a number of different ways of creating this spinning light, one of which was a circular strip of spinning LED TV screens which we factored into the previs for the Rescue Sequence. One of the tools used to make sure that we were looking at the right direction at the right time, was the simulcam that gives a real-time preview with a previs background, while we are composing and shooting our shot."
"We had the stereo RED camera on a Technodolly that is naturally encoded, because it works with the motion-control camera. We took the camera feed and ran it through MotionBuilder, which does a real-time track of the camera position that renders out a real-time version of the previs. It then goes to the video system department on-set, and they do a live composite of the shot with the background there.”
Explosion in Space
"We had one explosive element that blew-up the airlock at the front of the ship,” remarks Richard Stammers. "The fireball that comes out is so quick, so there was nothing that we did any different to that in terms of how that works. The decompression that follows, where you see all of the air being sucked out; that was a visual cheat to imply how we see this happen. We worked on the idea that the atmosphere of the ship condenses and gets sucked out and that's why you see the gas escaping, and additional CG particles were put into it to help sell the idea and have the windows frost up as that happens.”
"When Jessica comes out of the ship on her jetpack, she is suspended on wires being physically moved across our green screen stage at the appropriate travel distance,” reveals Richard Stammers. "Not everything worked perfectly, but it gave us a good point where we had a cut that worked. There were originally four or five shots going in to previs, and it made sense to do it as a full digital shot because the actors are not quite close enough to us. There was a specific motion going on, and it would probably benefit from us having some freedom in the animation to do more than what could be achieved on a wire rig. There were some cases with Matt Damon where we kept his face inside the helmet and replaced his body with a CG one. When Matt first flies out of the capsule in his Iron Man pose, loses control and does a lot of tumbling. With an animation process we were able to do more convoluted flips that could ever be achieved on a wire rig.”
"Ridley wanted to build the tension of the rescue to the point they miss, and the last chance for Mark Watney is to grab hold of the tether,” states Richard Stammers. "There was nothing that was shot practically on-set that felt weightless enough, so we always knew that was going to be a digital element. We did shoot with a real tether, and wherever they were holding it in their hands, that bit was kept and we replaced the section that might have been sagging. It required quite complex simulation work from the Framestore team to get a nice movement that conveyed that zero gravity feeling. When Mark Watney intercepts the tether and pulls Commander Lewis off her position, they start this endless spin where there isn't anything to decelerate them, so the only way to stop is to pull each other closer together until they connect. Through all of this motion the tether builds up this complexity of a bird's nest around them, which is a nice shot dressing to the last shots when they come together.”
"We had to remove the visors because of reflection issues with the cameras, lights, crew or green screen,” reveals Richard Stammers. "It became a big amount of work for the show, both for MPC and Framestore. We knew going in that we would be doing some and bid the show accordingly for adding CG visors. Some shots were underestimated and where that became an issue was when we needed the actors reflected in the visor. Quite often they were doing tasks that involved needing to see their own hands, and what the hands were doing reflected. Where we could, plates were shot of hand elements [with a wide angle lens]. In most cases, MPC had to do a full digital double animation of what Mark Watney was doing just to get the reflection in his visor. The visor provides a 180-degree view of the environment which also needed to be reflected.”
"All of the sequences had their challenges, but the biggest one overall was the schedule,” notes Richard Stammers. Having to do 1,100 shots in a 24-week post-production schedule, in native stereo, was incredibly challenging. "We went in knowing that it would be a short schedule, but we originally had 28 weeks, then we were six to eight weeks in and the filmmakers and studio decided to bring the release date earlier. That meant a lot more overtime and bringing on a few more vendors to help out with some of the overflow work that we didn't want to burden our three main vendors [MPC, Framestore and The Senate] with."
"It became a scheduling, mental and stamina challenge to be able to keep everyone going at the pace, and excited about everything they were doing to get The Martian delivered on time.” Stammers adds, "It was a fantastic project, and witnessing the positivity that people are seeing in it now is inspiring and makes it feel all worthwhile.”