Treehouse Ninjas: studio interview
Mauro Frau and the Treehouse Ninjas are back to discuss working on Cyberpunk 2077, The Heretic and a vision of the future of the games and movies industries…
Could you tell us a little bit about yourselves; what is Treehouse Ninjas, what do you do, and what have you been busy with lately?
Mauro: Treehouse Ninjas is an independent studio creating high-end computer graphics, principally specializing in all-round environment art and lighting.
Being a team of generalists with mixed backgrounds from both the movie and the game industry, we decided to embrace this hybrid nature of ours since the beginning. Our working style is characterized by a strong narrative-driven approach, where every element is just in the right place to serve the story moment and dramatic composition, as a legacy of our movie industry experience. We’ve also been digging the advancements of real-time engines for years now and in general the high potential of game technology and games as an evolving industry and medium.
We’re coming out of two quite intense years spent full steam on being part of the titanic task of building and lighting Night City for Cyberpunk 2077, in support of our good friends at CD PROJEKT RED. It has been quite a ride and the team here has gone through all the phases of creating more than a hundred locations, some of them pretty huge and complex, from the early stages of post-greyboxing to final lighting. The results are top-notch and we’re all proud of the work done, but you’ll see it for yourselves as it is out now!
Mauro, you recently did an epic two-hour talk about the convergence of movie and computer-game production. Can you give us a brief summary?
Mauro: Yes, during the first half of 2018, together with Unity, we developed a short movie titled The Heretic, on which we were in charge of the environments. It’s one of the first high-end attempts at a fully real-time photorealistic movie (featuring a human character,) entirely developed and run on a game engine.
That experience consolidated a bunch of thoughts that had been spinning in my head for a while: before co-founding Treehouse Ninjas I was well into my career in VFX and I have memories being at ILM working on Star Wars and keeping up with the news on all the latest advancements in VR, AR, games and non-linear storytelling. I started wondering “How long would have audiences around the world been happy to only watch stories by staring at a rectangular flat screen?”
That’s when I started putting some more serious analytical considerations into this idea of the movie and game industries converging and mutually inspiring their (needed) renaissance. This is not a radically new idea, it has been prophesied in the past; however back then was a theory, now it’s an incoming reality.
I thought of organizing my views on what’s going on in the world with new generations of movies, new generations of games and new generations of people
We live in a world of buzzwords and trending hashtags, it can be tricky to find a solid perspective among so many flashy posts bragging about this and that.
So this is a three part presentation where Part 1 takes you through a behind the scenes of the environment work on The Heretic short movie, showing how we designed and created the world it takes place in.
The Convergence of Games & Movies, Part 1
Part 2 puts games and movies on a parallel timeline, finding the common denominators of their history as industries, to then focus on the current times we live in and the upcoming generations of content creators. The core section of part two is about the grammar of games and movies, how the storytelling structure differs in these media and in which ways it overlaps.
The Convergence of Games & Movies, Part 2
Part 3 is a bit more advanced, and either some movie or game industry experience is beneficial to follow, but not essential. It’s about the similarities of how games and movies are made, but mainly about giving some guidelines to the artists willing to jump from one industry to the other, which skills are usable cross-industry and which ones need to be adjusted in order to successfully make that jump. It also touches on the dreaded topic of AI making all of us unemployed.
The Convergence of Games & Movies, Part 3
The overall goal is to give a very grounded ‘weather cast’ of the times that are coming, and hopefully create a more organic view about what the immediate future holds.
Szabolcs, what were you guys working on before Cyberpunk and what were the main challenges? What kind of workflows or approach do you have at Treehouse Ninjas?
Szabolcs: Our team has a very cool characteristic: narrative mindset. What is the purpose of the location? Who uses the location, what is the given prop used for? What happened to that prop before? These seem to be unimportant questions, but the answers to these questions will tell a story of an object or location that will add a lot to the experience. Even a simple coffee maker can tell a story about the location. So I can say our approach is to focus on back story.
At Treehouse Ninjas every team member is generally responsible for an entire area, from the modeling, to the texturing, scene assembly and lighting. We are not specialists, like modelers, or texture artists, every single member of the team is capable of modeling, texturing, shading, lighting, etc. We are Ninjas!
Our previous project, before we started working on Cyberpunk 2077, was Wolfenstein: Youngblood; we were in charge of an entire section of the game, a complex of secret underground laboratories called the “Lab-X.” We built up the areas using the idTech6 engine. It’s a big thing, since idTech engines are not commercially available. Modeling was done in Maya and Modo, textures were created using Substance applications. We also had our own custom tools maintaining seamless integration between Maya and Modo, speeding things up, and also resolving everyday updates during the night, so when the artists arrived at the studio the Ninjas arrived at the dojo, they didn’t have to spend time on updating the latest data from the database and builds.
We had custom scene builders to automatically reassemble scenes in and out of the engine; from the engine to Maya, from Maya to Modo, from Modo back to the engine, etc. and other smaller pipeline tools. One of these smaller tools for example was the THN Initializer, where I could set up which area I was about to work on, a set dressing element or an architectural model, then this tool would set up all the folders and all the naming conventions and working environment for me. When generating contents as a team, you sometimes risk creating things that are redundant to what somebody else has already created, or simply put them in the wrong place. As basic as it may sound, it happens even to seasoned professionals, so the right tools can save a lot of headaches.
Location building starts with a blockout, also called “greybox model.” This represents the location's physical dimensions. At this point we start to work tightly with the game's level designers to support the player movements and the combat metrics, like moving on stairs, vaulting through windows, sliding through doorways.
The location of the Lab-X contained multiple areas and these areas were split among the Ninjas. Everybody was working on their own area from the architectural pass to the last tiny bit of asset. With this method we were not just creating scattered contributions to the location, but everybody owned a sector of it, and started to add narrative elements. When the player entered an area, almost every corner had some story to tell (when you play the game, just check out the computer screens, or the photos on the walls here and there!)
The fun starts after the block out stage. Modeling and texturing is based on real world references. It’s the best approach, real world references help maintain proper scale, and also the player feels that everything is more believable if it’s based on reality. For example, on the Youngblood project the art direction suggested a sci-fi theme as it was imagined in the 1980s. Many of us in the studio were kids (or born) in that period.
We started collecting reference images from the 80s (computers like Atari2600, Commodore; screenshots of sci-fi movies like Star Wars, Star Trek, we had retro tech reference books, and we had a special playlist on Spotify including soundtracks of the popular games of 80s and 90s (yes, there were video games back in those times!) and Synthwave music.
For the Lab-X we defined early the primary materials used throughout the location. As the lab is operated by the Nazis, we decided to use black metal as the primary material. Almost every wall in this location is dark metal. The color and the hardness of the material represent the evilness of the place. We also used grey to give a corporate, inhuman feeling to the place. Finally a pinch of muddy cyan adds a creepy hospital feeling. The red color (monitor screens, painted stripes on the walls, floors, signage) represents the danger the player is approaching.
When the materials and mood were defined, we started to create our library of building modules. These modules had to fit to each other, so once a wall or a floor is assembled, the result is both continuous and free from unwanted repetitions.
After the architecture pass of an area was finished, we started to populate it with decorative assets. There were hero objects, which had their own concept art, but in many cases we invented items based on the function of the area. This is how we invented devices like the “Head Preservation Unit”, “Cryogenic Container”, “Arm Controlling Device” for the medical labs. We relied a lot on our reference collection of devices from the 80s.
During modeling, we had to consider that the devices in the 80s were still quite bulky, their edges and corners were quite softly beveled (but not as much as in the 60s) and this had to be present in the models as well. Also, texture-wise we recreated materials from the same time period. For medical devices we used a lot of polished metals, steels, for electronic devices we used painted metal or plastic surfaces. For the screens we even introduced the glitches typical to the cathode ray screens of the 80s. A couple of the computer screens have error messages we invented. Of course translated to German and beside the environments, we also supported with some weapon work.
What was really important for the team was our daily screening. We met every morning in our meeting room, sitting on the poufs and watching each other’s progress on a huge screen. We saw each other's progress and could give each other feedback and propose new ideas: inspiration was born during those meetings.
As usual, one of the challenges was to adapt to the changes. Changes happen time to time. It might turn out that the game-play suffers from the layout of the location, or there must be a new room set up here or there. Or a gigantic robotic arm looks cool on the concept drawing but doesn’t really work in 3D. In these cases we need to be proactive, and offer solutions to cut the time that is needed to readapt the contents.
Levi, can you tell us more about Treehouse Ninjas’ work on The Heretic? How did it come together and how did it differ from your previous work?
Levi: When we started work on Unity’s Heretic project, I already had some experience in game development, gained during the work on Wolfenstein: The New Colossus. That project was my first real game development experience, until that point I was working on the movie side, mainly focused on cinematic trailers for AAA games. The Heretic allowed me to put into practice the storytelling and composition skills I developed in the past few years.
In the world of The Heretic we had three main sectors, all of which were created in real scale, involving three different types of environment: a realistic basement corridor, a semi-realistic rock passage and a huge cave - beyond imagination… really huge: 2 km long and about 500 meters tall.
First we had to decide on the tools and technical methods we were going to use to get the most efficient approach. We decided to use a standard asset workflow for the basement section, and photogrammetry for the passage and the cave. I improved my photogrammetry skills a lot during this project and we perfected our internal processing workflow.
To be honest, I really enjoyed the whole process from the very beginning. We went to a natural cave in Hungary, Tatabánya and took thousands of pictures; basically we scanned the whole cave. It was my task to process the pictures and make game-ready assets from them. During this stage, I used Reality Capture to create raw geometry and textures, then made the geometry cleanup in ZBrush, made the textures PBR-ready using Unity's de-Lighting tool, then in Substance Designer created the final texture maps. We also had useful support from the Unity artists because they provided many tips and tricks acquired during the making of the Book of The Dead forest demo.
Some of the scanned rock assets created for The Heretic.
At the end of the process, we had a very nice catalogue of different rock patches, and we were ready to assemble the cave and the passage using those patches. So instead of having one giant single frozen mesh of the cave, we built up the location from several re-conformed scanned rock modules, which gave us flexibility for later changes in the layout.
As this project was an unusual experiment to mix up cinematic storytelling with game technology we had to find answers to a few questions that creatives were facing with the spread of real-time in movies and back in 2018; when we worked on The Heretic, it was quite tricky to find those answers. For instance, in a movie, the director has many tools to control the attention of the viewer, to define the point of interest, but in the end everything is going to be locked, so there is the luxury to optimize the visuals to very specific camera angles.
The Heretic, Movie Reveal at GDC 2019.
In the case of The Heretic we had to retain some flexibility to allow for changes to be executed in real-time on the virtual set; for example, when the robotic bird flies around in the basement - in a relatively small, dense, heavily dressed area, surrounded by floating point lights. Because of the dynamic lighting from these small point lights we never knew until the very end which elements of the environment were going to be seen clearly and which others were covered by shadows, so we had to create every single asset to hero quality to keep the flexibility of later directorial decisions, like a new view from a different camera angle.
In terms of composition and story-telling the main principles were: using portals and gates as story landmarks, in sync with the protagonist stepping deeper in the adventure, the composition of the images relies heavily on a strong visual-directional force, giving strength and consistency to the storytelling. This gate-to-gate motive leads the viewer into the story, indicating threshold moments, single points after which there is no turning back for the protagonist. The way we used doors, portals, and gates can be observed as the story goes on. They give a journey-like feeling, as though you are getting deeper and deeper into the world as you pass these thresholds.
The biggest difference in this project -compared to the previous ones, for me, was mixing technologies. I would say here we had the opportunity to be part of a future where games and movies blend together. That was a really good example for the convergence between gaming and movies. The artistic approach was same as if we were making a movie, but to achieve that purpose in the engine, especially the asset creation and organization, we were using mainly real-time game production methods, which is so exciting to me.
Bojan, you’ve been involved in all these projects, what was your experience? How is a day at Treehouse Ninjas? How is the studio like? How is the working environment?
Bojan: Working with the Ninjas was my first experience with realistic game art. Before that I worked mainly with very low poly models or stylized artwork. When I first started here I was utterly humbled by the projects we were on and as the time passed I could almost feel my head throbbing with the sheer amount of knowledge I was receiving on a daily basis.
The fact that the team is made up of generalists with vast knowledge of every stage of asset creation, instead of narrowly specialized people, gives us the edge and flexibility we need to cope with everyday tasks that we take on. With that kind of a team it’s very easy to exchange ideas, gather new insights and (if need be) change the approach to the problem quickly and efficiently.
We usually start the day with a morning routine where we all get acquainted with the current status of the project we are working on, as well as the individual progress of the tasks we are each assigned to. These meetings are a great way for us all to be in sync with the status of, both, the specific single tasks and the overall projects, but also allow us to engage in friendly and constructive peer to peer scrutiny, criticism and consultation. It doesn't really matter if we are all on the same project, or we are assigned to multiple ones, we all tend to engage in this together.
On Cyberpunk 2077 each of us had several locations we worked on, and we took full ownership of them. In our projects, we normally start from the blockout, as Szabolcs explained, which we are presented with when we begin work. This gives us a huge amount of artistic freedom to create the environment because, apart from boundaries that the blockout provides, we are free to do everything in our imagination in order to make the location look great and tell its own story, while staying within the style and lore of the game. We then usually start building the architecture (exterior and interior,) followed by set dressing and polishing, generally moving from the largest to the smallest component.
When this is done, we move on to the lighting, which also works in stages. This is possibly the most fun part for me, as it requires both the artistic approach, but also quite a lot of technical art and precision work in order for everything to work perfectly. However, coping with the complex overlap of technical understanding and artistic expertise required to make our locations look perfect and at the same time empower both the storytelling and the gameplay, can sometimes be a very hard challenge.
We started off in a smaller studio, but as our team grew, we moved into bigger offices. Nevertheless, we are still all together in one huge space with no walls, so there is this constant feeling of camaraderie, friendship and steel on steel! The stuff of legends! But seriously, the Ninjas are, hands down, one of the healthiest communities I have ever had the pleasure of working with. There is a true sense of team spirit at work here, as everyone is immediately ready to help, share knowledge, give advice and be understanding, not only of work, but also of your day-to-day life.
We have a strong “No Bull****” policy in the studio. We don’t use fancy phrases and tiptoe diplomacy among us. We are professionals and we are humans. We do our job as best as we can, we strive to improve, we speak our minds, we own up to our mistakes, and we humbly receive our praises when they are due.
Apart from the work related stuff, we have some fun traditions we do here. After every six months on the job we “level up” as a Ninja, and we are ceremonially presented with a belt of color, ranging from white (for a beginner) all the way up to the black one. Also, we do drawing sessions with live models, which is a great way to simultaneously relax, blow off some steam, and practice your skills.
What industry developments and innovations do you see happening in the next five to ten years? What is exciting about them?
Bojan: I believe in the future we will be seeing a shift in trend towards studios like Treehouse Ninjas. In the recent years, we have witnessed many giants of the industry losing reputation or falling out of grace with the customers (not to mention the public outrage and employee dissatisfaction,) which reflects on the quality of the content being produced, and instigates people to leave those studios in search for something more satisfying.
Having a relatively small team of people, capable of performing an insanely wide spectrum of tasks at top quality level, is something that will be giving studios like us the edge. From the business aspect, this industry is constantly changing. AI is getting better and better at replacing people in some of the more menial tasks and procedural modelling is already in full swing. Having people who are capable of quickly adapting to the currents of the job is pretty much crucial. From the aspect of an employee, it is significantly more satisfying to work in a studio where you have the feeling of accomplishment and contribution, rather than being a small cog in a big machine.
I mean, would you rather work in a huge corporation, making fifty seven texture variations for the same object every day, or with the Ninjas on producing a full environment from scratch all the way up to lighting, where the only question is “How good can I make this?”
Szabolcs: Being a huge fan of procedural modeling and texturing I believe that one of the future developments is going to be the more common utilization of proceduralism in games. Content creation is more and more demanding. Games need more, better quality content as game hardware develops (desktop computers and consoles alike,) and usually it’s very hard to create content using an “orthodox” method. Think of the buildings in any game. At this moment artists manually create the buildings using the blocks they are given. With a procedural approach the artist is given the building modules and can set up certain rules (like the footprint of the building, type of windows, facade, decoration elements, number of floors, etc) and the process builds up a variety of these houses ready to use. We see this approach nowadays appearing in commercial game engines, like Unreal and Unity, utilizing Houdini Engine.
For me it’s one of the most exciting things in the future. As a former programmer, I always found interesting how the real world can be imitated using procedural workflows. This approach requires not just technical skills and understanding, but also a pair of eyes to the minute details that makes our world. Also, we need to accept the fact that machine learning is becoming more and more popular. As an artist and service provider it’s the best to start watching where machine learning goes, and how we could apply it in our daily work. I’m pretty sure that proceduralism combined with machine learning could be a huge asset in our hands.
Levi: Since I was working in the movie industry, I expect a higher level of real-time and augmented reality technologies integration into the movie shooting process, because these days the filming action is still mostly taken in front of the green screen, and that makes the visual storytelling even harder for the director. That's when real-time and augmented reality technologies could come in handy and might help a lot; for better visualization of a scene and give the opportunity to try different ideas or more iterations very quickly, resulting in faster production and better quality.
Treehouse Ninjas is recruiting, any guidelines to the aspiring ninjas? What does it take to be a Ninja?
Levi: I would say mostly: be open-minded and have a sense of professional humility. I read somewhere that humility is a routinely misunderstood virtue. It does not mean self-deprecation or having a mild temperament. It does mean knowing that you don't know it all and that it's not all about you… and acting accordingly. It means you are willing to get in there and do anything yourself that you would ask any of your colleagues to do. I think to be a good Ninja you need this virtue, because we are team workers, we rely on each other, and help each other, because it’s all about our passion: creating excellent, groundbreaking environment artwork.
One important thing for those who may want to join us: we have a band, named Not Shuriken Play, and we’re looking for a singer and a sax player for the band, so when you send the portfolio, please let us know if you qualify for that position too...
Bojan: I would point out the ability to embrace new methods, technologies and software. We call ourselves “software agnostics,” meaning we don’t really feel bound to one specific set of tools or skills in order to get the work done. A lot of work we did in the past involved a huge variety of tools we needed to learn how to use and acquiring new skills in order to either get the job done, or fit into the pipeline. Not being exclusive to one tool and set of skills significantly facilitates this process.
Then, and I know this sounds cliché, there is willingness to listen and learn, which is actually crucial. We have amazing, experienced artists here and just interacting with them is a real treasure trove. I have been working with these people for some years now and I have yet to see the day when I have not learned something new.
Also, not being a snowflake helps. We all depend and rely on each other heavily. Often it happens we jump into each other's work to help out or just inherit tasks. It is important not to focus only on how you will finish the task but also what will happen if the next person needs to take a swing at it.
Szabolcs: My advice is to develop a good eye for details. The ideal candidate has the sense of proportion, composition and detail. You need to develop the skill to deconstruct the world into elements and phenomena that you can recreate in 3D.
When you make your portfolio, do not forget that it is only as strong as its weakest element. Be aware of your strengths and weaknesses! Be confident using your application of choice. We don’t hire you because of the software you are using, we hire you because of how you use the software you are using. And one last thing; like the others pointed out before me: you are about to work in a team. Our team requires team members to complete the mission.
What’s next for Treehouse Ninjas? What long term goals and developments are you aiming at?
Mauro: It’s been three and a half years since we started Treehouse Ninjas, we’ve done a good job so far and we have established healthy relationships with top-of-the-line clients on AAA projects and franchises. We’re now busy with the new round of projects going into 2021, which are obviously classified, but in general we’re now aiming at consolidating further by expanding our Ninja crew a bit more. Given the number of artists and talented minds out there it sounds strange, but it isn’t easy to find the right people that can keep up with the expectations of our studio. It takes focused work so we’ll put more effort in that next.
Then of course it’s no secret we’re often gathering thoughts and ideas for our own proprietary IP. That’s also not an easy task, but we’re optimistic we’ll get there. We’re not in a rush though; we’re enjoying the journey so far.
For more info about The Heretic, please visit this website