Section:

From VFX to virtual reality with Nathalie Mathé

Dive into the future of virtual reality with VFX artist-turned VR compositor Nathalie Mathé, and discover the possibilities of this nascent technology...

Dive into the future of virtual reality with VFX artist-turned VR compositor Nathalie Mathé, and discover the possibilities of this nascent technology...

Meet with Fast & Furious 6 and Dark Knight Rises digital matte painter, Nathalie Mathé, who successfully left the world of feature film visual effects to use her skills and expertise in the brave new world of emerging virtual reality...

3dtotal: Who are you and what do you do?

Nathalie Mathé: I am a virtual reality (VR) compositor and supervisor since 2013, using my previous expertise in films visual effects and CG animation to create fully immersive 3D 360-degrees videos seen in the Oculus Rift or Gear VR (head-mounted displays). My role consists of stitching multiple cameras together into a seamless spherical image, correcting parallax issues, improving color and 3D stereo, stabilizing camera moves to avoid VR motion sickness, as well as adding effects or CG elements into the spherical image if needed. On the technical side, I design VR compositing workflows and templates adapted to each project. I also collaborate with developers to create new tools or plug-ins for working with full stereo spherical images, since most of the commercial post-production tools are not adapted to this kind of image yet.

3dt: What are some of the projects you have worked on/clients or studios you've worked with? What are you most famous for?

NM: I currently work at Jaunt, a VR start-up, as director of creative technology. I've contributed to several VR experiences such as the music video Paul McCartney - Live and Let Die, which was our first online release in November 2014, as well as The Mission, an action-packed cinematic VR short produced in collaboration with New Deal Studios. Other VR videos I've worked on include North Face Climbers, an extreme sport experience, and Kaiju Fury, a short fiction homage to classic Kaiju films which was presented at Sundance Festival 2015 New Frontier.

With Condition One, another VR start-up, I supervised compositing on Zero Point, directed by award-winning filmmaker Danfung Dennis. It was the first movie made for the Oculus Rift in 3D 360-video. This documentary follows the pioneers of virtual reality, the researchers and developers creating an entirely new digital dimension. It was released online in October 2014, and also selected at Sundance Festival New Frontier in 2015.

Before that, I worked for 12 years as a digital matte painter, environment artist and textures painter on VFX feature films and CG animated movies in Europe and Canada. I truly enjoy creating environments; it might entail destroying buildings, extending existing environments, removing roads or adding mountains, or whatever needs to be there to fulfill the direction of the story. I also enjoy the first research and design phase, when you try to define the concept for the shot.

I had the chance to work on Persepolis, an animated film by Marjane Satrapi which won the Cannes Jury's prize in 2007. At Double Negative in London, I created matte paintings, textures and 3D projections for Captain Philips, Skyfall, The Dark Knight Rises' explosion sequence, Fast and Furious 6's tank chase sequence and Les Miserables, among other films. I later worked as concept artist and lead matte painter on Teenage Mutant Ninja Turtles at Image Engine, in Canada.

Prior to being a visual artist, I had another career as a computer scientist in the 90s. I worked for 8 years at NASA Ames as lead of the advanced interaction media group. My team developed an adaptive online documentation system for Space Shuttle operations, and one of the first Internet bookmarks sharing tool integrating a machine learning engine.

3dt: Was there a decisive moment that made you decide to leave VFX? What was your motivation to get into VR?

NM: For years I have been interested in creating more interactive experiences. With films you get to work with incredibly talented people and on very challenging projects both creatively and technically. But in my opinion, big blockbusters movies have kind of become all of the same lately, and you spend weeks or months sometimes working on a scene that only lasts a few seconds on the screen. So VFX became less interesting and challenging to me.

The other reason was the necessity to constantly move around the world to get work. I've lived in Paris, San Francisco, London, and Vancouver, and didn't always enjoy it there. When I made the choice to come back to California, it was because this is a very creative place where the future gets invented, and an amazingly beautiful landscape. I knew clearly that the VFX industry in the Bay Area was having a real hard time, but nonetheless I decided to take my chances and quit my job. And after a month of being here I met with a virtual reality start-up, Condition One, and that's how it all started.

3dt: What was your first VR experience?

NM: At the first SFVR meetup in San Francisco in July 2013. I tried ZSpace stereoscopic display with a 3D stylus, and Condition One's first 360-video in the Oculus Rift. Quite impressive. I was hooked.
I also remember trying the Data Glove in Jaron Lanier lab years ago when I was working at NASA.

3dt: What is it that you are personally most excited to try as an experience in VR?

NM: The VR experiences I find the most compelling personally are the ones where you have an emotional immersion, not just a 'waou' kind of experience, and you feel connected to the characters. I would love in the future to be able to have an experience where I could participate in the life of someone else, in another country, a different culture, being able to experience how they think, how they feel. And the second thing that is really special about VR is the fact that your body feels that it is really there in this alternate reality. I've tried some snowboarding apps and my legs were starting to move on their own. There is really an opportunity to be able to interact with your entire body, and not just with a game console.

3dt: Did you feel any sim-sickness or do you ever feel any sim-sickness?

NM: Yes I am fairly sensitive, especially when real or simulated body movement is involved. I usually feel uncomfortable in my stomach, never really sick to the point of having to stop. Except with some games I tried where the movements were really crude. Stable and constant translation is less an issue.

3dt: What's the most indispensable tool you use in creating VR experiences?

NM: Since I work on video content, I've been mostly using NUKE. It's an awesome tool for compositing 2D as well as 3D elements, and for stereo too. I also use proprietary tools developed at Jaunt to integrate images from multiple cameras.

"After a few minutes I could feel tingles when the balls were touching my palms in VR, even if they weren't truly there… really strange. It's called synesthesia"

3dt: What's the strangest experience you have had with VR?

NM: In a game where I was playing with Leap Motion bouncing balls in my hands; after a few minutes I could feel tingles when the balls were touching my palms in VR, even if they weren't truly there… really strange. It's called synesthesia.

3dt: What's your current favorite VR experience that you have not developed yourself?

NM: TiltBrush, for painting directly in 3D in the Oculus Rift with hand gestures. I just saw 3D paintings and demos made by artists at a Gray Area exhibit this month in San Francisco, and I tried it for myself. It's an amazing tool and I can't wait to paint all kind of worlds in VR.

3dt: Whose work (both past and present) do you really admire and why?

NM: I was very inspired the first time I saw Ralph McQuarrie's matte paintings at a San Francisco exhibit 20 years ago. They were huge paintings on glass, bigger than me, and they were so realistic looking with planets and alien worlds – all painted by hand by this incredible artist. I had the chance to meet him in person, and a brand new career path opened up for me.

I find live and interactive digital art very inspiring too, whether it's projection mapping, light painting, augmented dance performance, creative coding. I particularly like the works created by Design I/O, Obscura Digital, and AntiVJ. Being able to play, interact with body and hand gestures, and being part of a piece of interactive art is really fun.

In terms of immersive media like VR, I truly enjoy the emotional experiences created by director Chris Milk or journalist Nonny de la Peña, as well as the piece Cloud created by artists James George and Jonathan Minard. It's a computational documentary exploring art and code, mixing artists interviews, direct experience of their works, and interactive navigation.

3dt: Besides games what do you think will be the most useful application for VR?

NM: Definitely education and training using immersive videos or games. VR is the medium to put you into someone else shoes, show you the world from a different point of view. Immersive journalism and medical therapy seem very useful as well.

3dt: Is there a way to prevent escapism through virtual reality? What do you see as the major obstacles to VR?

NM: Right now the pieces I have experienced are very short, and I see mostly friends sharing them and speaking about them afterwards. There is another thing that in my eyes might be dangerous about VR: it is an extraordinary powerful emotional media, even more than film, because it's not just your emotions, it's also your body that is involved. So, given everything going on in the world right now, this is an extraordinary powerful tool for manipulating people. The only way to counteract that is to encourage more people to create VR content that is going to create more connections, and more understanding. Because you can use any media for any purpose that you want. That's my concern.

3dt: How does prolonged use of virtual reality affect our brains? Like hours of it. What do we do about it, because the research isn't here yet?

NM: I read some study that was saying that with VR we are exciting about 50% of the neurons that you usually use when you interact with the real world. But it seems to me that we already spend most of our time looking at flat screens, which is the most unnatural thing in the world. Our eyes have to focus on this flat screen at 30 centimetres, when naturally we need to change the convergence distance of our eyes all the time. So actually getting onto VR is much more natural, and should be less taxing on our senses. The limitation right now is more the technology, the quality of the pixels, the quality of the 3D rendering that you see in some games. So once the technology evolves, I would expect it feels more natural than looking at a flat screen. I don't know that it won't have any effect, but it can't be worse than what we are already doing to our brains.

3dt: How are things different in this industry? Would you go back to VFX?

NM: Right now VR technology is still in its infancy, and is dominated by hardware and software engineers. But things are evolving very fast. What started as a small independent project 2 years ago has now attracted major media and technology players who want to be part of it. On the other end, content creators are still struggling to create original experiences with tools that are not fully adapted. There is also no clear funding or distribution channel to create revenue for artists so far. So it is very different from VFX which is an established field with best practices, workflows, markets, and so on. However I enjoy these new challenges, and hope there will still be a place for independent and creative VR projects.

3dt: What advice would you give to people on how to get into VR development?

NM: Learn Unity, go to lots of VR meetups in your area, it's the best way to make connections, experiment and be flexible.

3dt: Do we want more women in VR? Not just as consumers but as creators? And how do we go about accomplishing that goal?

NM: That's a very interesting question. Yes, we want more women in VR, because there are so few of us, not even 10%. That's how it started in computer science 20 years ago, and then it went up, and now it's down again. That how it was when I started to work in VFX, then it went up to like 25-30% depending on the studios. And if you look at the media industry, because this is a new medium, all reports show that 90% of the media, newspapers, TV, online, are created by white males. And even when you look at the statistics of so called female topics being covered by the news, they are covered by men mostly, and they interview more male experts on these topics than female. So this sort of bias has been there for years and years.

This is a new medium and there is a great opportunity there. But if we just keep repeating the same biased approach that has been used until now, it's just going to be the same, nothing is going to change. And how we bring more women into VR is a very good question. There are many factors right now, negative stereotypes, in education, in the workforce, and in our culture, that discourage women. Even when they are in the computer or media industry, they drop out more than men. So you have to make conscious efforts and policies both at the educational level and in the work culture to bring women in and to keep them.

On the positive side, recent studies have shown that companies and start-ups with better gender balance are more successful, so that could help change some mind-sets.

3dt: What can we do to design VR experiences for women too?

NM: The first thing to do is to test your VR experiences at all stages of development with 50% women, and see how they react. Another thing is to look for what women are really interested in beyond obvious stereotypes. Let me give you an example of some apps that were developed for women that women really use. Recently there was a UN Women hackathon around the world that was done in India, New York, Oakland, and other countries. And most of the apps developed by these women were related to personal safety. Some of the girls in Oakland developed a mobile app for reporting in the streets where it was safe to be, or where they noticed something dangerous happening to let their friends know about it. The girls in India developed a safe to use and friendly sex education website, with counselling for girls, resources that they usually don't have access to. So these are practical things that interest women.

The opinions expressed here are personal and do not represent Jaunt in any way.

Related links

There's loads more to see on Nathalie's site
Check out Nathalie's linkedin
Nathalie on IMDb
See more of Condition One's work
Take a look at Jaunt's latest work

Fetching comments...

Post a comment