![\"\"](\"https:\/\/demo.techsystintel.com\/press_kitchen_dev\/wp-content\/uploads\/2022\/04\/taika-waititi-rhys-darby_2-624x416-1.jpg\")
<\/p>\n<\/p>\n
Virtual production has grown exponentially during the last two years, becoming a safe way to keep productions moving and on schedule while also keeping people apart. The travel costs saved have also been significant. No getting on planes, no being away from home for weeks at a time.<\/p>\n
<\/p>\n
That\u2019s just one of the new technology\u2019s many upsides. While the volume is reinventing film and television workflows, the crews brought back together on set are much leaner than they were before. More creatives and executives can simply dial in via myriad stable apps and platforms. It\u2019s a streamlined set, enabled by remote workflows, that lets every virtual production flex its intrinsic scalability: they can go anywhere, go small or go grand, depending on budgets.<\/p>\n
<\/p>\n
So while virtual production does take a team, it also allows a majority of the team to work and collaborate from anywhere. And even with COVID restrictions loosening, the idea of not sending an entire team to shoot in the Atlantic Ocean while others remain remote, is a win-win for most.\u00a0 It\u2019s why the majority of studios will continue to\u00a0invest heavily\u00a0in technology and assets to support those shifts in workflow in the near term. The initial investment may seem steep, particularly for smaller shops, but it has become clear that the transition to virtual production will drive costs down dramatically over time.<\/p>\n
<\/p>\n
For this piece, we spoke to two well-established VFX studios \u2014 Stargate Digital and Zoic Studios \u2014 that have both embraced virtual production.<\/p>\n
<\/p>\n
Stargate Studios: At Sea in the Volume<\/strong> <\/p>\n The limited series, written and created by David Jenkins and just greenlit for a second season, also stars\u00a0Flight of the Conchords\u00a0alum Rhys Darby as an eccentric pirate with a heart of gold. Waititi, who executive produces and directed an episode, gives his Blackbeard a gentler, unexpected backstory.<\/p>\n <\/p>\n For\u00a0Stargate Studios\u2019 founder Sam Nicholson, ASC, bringing the ocean to the soundstage in real time was just another day in the studio. Stargate has built its VFX business on being an early adopter of new workflows and technology. \u201cWe have been pursuing virtual production for many years in our VFX work,\u201d he says, developing what Stargate calls the \u201cVirtual Backlot,\u201d an expansive library of 2D and 3D assets of locations from around the world. \u201cThis allows us to create photoreal virtual environments relatively easily, wherever we are.\u201d<\/p>\n <\/p>\n He traces the shift to virtual shoots to the moment after 9\/11 happened, when actors couldn\u2019t get on planes. \u201cAll of a sudden,\u00a0ER\u00a0was shooting in Chicago and\u00a0CSI\u00a0was shooting in Las Vegas. More shows started shooting in New York. It was the first great location challenge the industry faced.\u201d<\/p>\n <\/p>\n Stargate began investigating how it could scale its greenscreen work by capturing real locations and turning them into greenscreen assets at Warner Bros. \u201cNBC gravitated to the concept, then ABC did. Sometimes they even said it looked better than what they were shooting on location.\u201d<\/p>\n <\/p>\n Flash forward to the COVID-19 pandemic, and the perfect storm produced another grand shift. \u201cAt the end of the day, I don\u2019t think people really like shooting on greenscreen,\u201d says Nicholson. \u201cIt\u2019s way less realistic than shooting on location. Plus, greenscreen workflows are quite complex and slow, with a considerable cost per shot. But when shooting on an LED volume, the whole process suddenly becomes easier. We bring all of the traditional visual effects and 3D CGI into this new medium and achieve much more in real time.\u201d<\/p>\n <\/p>\n Nicholson has always been interested in real-time visual effects. \u201cIn the very early days of the LED process, we wrote our own programming for off-axis projection,\u201d he says. \u201cThis was about a year before Epic Games released the Unreal Engine. We\u2019ve now fully integrated Unreal into our ThruView process to control virtual environments as well as kinetic lighting and camera-tracking on-set.\u201d<\/p>\n <\/p>\n As VFX shot tallies grew exponentially over the years, he noticed a pattern. \u201cWhen you look at traditional visual effects shows, with perhaps a thousand shots, and you are doing multiple shows, we were delivering almost 16,000 greenscreen shots per year throughout our global network,\u201d he says. \u201cThat\u2019s when I realized that about 50% to 75% of these shots are pretty simple, usually just two-layer composites for dialogue coverage. The other 10% to 20% are really difficult shots, and about 30% are somewhere in between. If you can finally track in-camera with a new tool like virtual production, which is what ThruView is all about, and bring in better lighting, you can now have an on-set, integrated system that allows you to look through the screen rather than at it.\u201d<\/p>\n <\/p>\n Mark Costa, HBO Max\u2019s new VP of production, was all-in with Stargate and the\u00a0Our Flag Means Death\u00a0filmmaking team to create the topsy-turvy ship on the high seas that is central to the show\u2019s storyline. \u201cWe\u2019d worked with Mark on a greenscreen show before, and it worked great,\u201d Nicholson recalls. \u201cMark came back to us and said, \u2018I might have this thing that\u2019s going to be really challenging. We haven\u2019t greenlit it yet, but could you do dry for wet? Could you shoot on a stage with an ocean for 14 weeks for a series and make it work?\u2019 So we went about solving that problem.\u201d<\/p>\n <\/p>\n Preproduction is an important part of any virtual production, and Nicholson\u2019s team shot a series of tests in Stargate\u2019s studios combining photographic water, 3D ships and LED screens. \u201cGoing in to production, we knew we didn\u2019t have time to create 3D ships or 3D water,\u201d he says. \u201cSo It had to be the real world.\u201d The results blew them away. Footage of ocean water in Los Angeles looked so compellingly realistic in their tests that they headed for a proper shoot in Puerto Rico.<\/p>\n <\/p>\n \u201cBecause it\u2019s a comedy, and because it\u2019s Taika, we wanted it to be full-resolution on the 160-foot screen, edge to edge,\u201d he says. \u201cTaika creates a wonderful vibe on set, and things get funnier and funnier as the takes progress. The actors are working off each other, and we didn\u2019t want it to feel like a traditional greenscreen visual effects show, which can sometimes be like shooting with a straitjacket.\u201d<\/p>\n <\/p>\n Using 360-degree rigs of Blackmagic\u2019s 12K Ursa Mini Pro cameras stabilized on a boat, the team captured camera-original RAW footage of the ocean that was later subsampled at 20K horizontal across the 160-foot LED wall. \u201cThat\u2019s pretty extreme data capture,\u201d says Nicholson. \u201cWe brought all that footage back here and started testing it on the wall.\u201d<\/p>\n <\/p>\n As the set on the soundstage was built out, Stargate added more greenscreen beyond the LED wall \u201cso there is literally no place on set you couldn\u2019t shoot,\u201d he says. \u201cWe tied the lighting in with our ThruView system so the lights automatically adjust to the plate, which lets you move very, very fast.\u201d Everything is DMX-controllable. \u201cYou\u2019ve got five cameras or nine cameras all completely rock-steady on a boat that can go up and down. But the horizon has to stay in the same place.\u201d<\/p>\n <\/p>\n The final-composite ocean footage is run through Unreal Engine to give the filmmakers precise camera-tracking control and allow for off-axis projection on the curved LED wall. For Our Flag Means Death, the wall has to rock right along with the gimbal-controlled boat. \u201cEverything is adjustable on the fly\u201d and fed through multiple on-set DaVinci Resolves, simultaneously linked to drive the 20K wall. \u201cThat\u2019s really part of the ThruView code that lets you have full resolution on the wall with all the bells and whistles of any color-timing session,\u201d Nicholson says. \u201cIf you want to put a grade on the wall, or you want to defocus it, you can literally be compositing live. It\u2019s definitely not a fixed asset. This puts the control back into the hands of the DP and DIT; you\u2019re doing what you would normally do in a great color-timing session, but you\u2019re doing it really fast on a 160-foot wall with an entire crew.\u201d<\/p>\n <\/p>\n That\u2019s the goal, he adds. \u201cWe want to make this level of real-time technology invisible\u201d and bring traditional VFX onto a live-action stage that easily flows into any editorial and post pipeline. \u201cThe more invisible the technology is, the more creative virtual production will become. I\u2019m really happy to say we never had a single minute of downtime throughout the 14-week shoot on\u00a0Our Flag Means Death. The filmmakers on-set can now be completely in the moment and do what they do best.\u201d<\/p>\n <\/p>\n Zoic Studios: A Front-Row Seat at the Revolution<\/strong> <\/p>\n Although Zoic\u2019s core team was initially underwhelmed with the quality of Unreal video, that all changed when raytracing was added to the game engine\u2019s feature set. \u201cI knew it was the future, and I knew I needed to be fully in it,\u201d he says. All became clear for Brami and his colleagues when Disney released The Mandalorian, the secret project Platteaux had been working on at ILM, which was shot entirely in an LED volume. \u201cThen they were like, \u2018Oh wow, I get it. This is the future.\u2019\u201d<\/p>\n <\/p>\n Launching its\u00a0Real Time Group\u00a0in 2021, Zoic joined the industrywide race to go all-in with real-time virtual production. \u201cWe knew we needed to embrace it, to recruit people, so that\u2019s when we started discussing opening a real-time department to see where we could feed Unreal. We didn\u2019t want to use it just for production or previz, either, but literally for everything.\u201d<\/p>\n <\/p>\n A mega grant from Epic helped move the needle, giving a \u201cton of artists the time not just to learn how to use Unreal but to see how far we can push it, where it breaks, and learn from that.\u201d Through trial and error and several borrowed LED screens that Zoic co-founder and executive creative director Chris Jones brokered from a tech partner, Brami and a handful of others tried to recreate what they saw on\u00a0The Mandalorian. \u201cI was lucky because I knew I could ask J\u00e9r\u00f4me questions as we went, and that helped us avoid a lot of unnecessary mistakes.\u201d Their big discovery? It wasn\u2019t that much different from existing VFX pipelines. \u201cIt was mind-blowing because we realized that even with not that much knowledge and only eight or nine months of using Unreal, we could still pull off a shoot as complex.\u201d<\/p>\n <\/p>\n Before the pandemic hit, Brami had already been working off an HP Teradici box, \u201cessentially remotely, but still working together in the same office,\u201d he says. It was an easy transition to go fully remote from home, thanks to Zoic CTO Saker Klippsten. \u201cKudos to Saker. In less than a week after lockdown, we were all working perfectly from home with zero lag.\u201d What was initially challenging was finding a way to communicate effectively in Microsoft Teams. \u201cYou couldn\u2019t just interact with someone normally or walk to their desk and discuss a shot,\u201d says Brami. So we started introducing dailies. We didn\u2019t want to overwhelm them, so we also created a general chat room where people could just hang out.\u201d<\/p>\n <\/p>\n Going remote, says Brami, has leveled the field for many artists. Location is no longer an issue. \u201cIt\u2019s brought way more people together, so to me, it opened us up to a world of talent,\u201d he says. \u201cWhen we collaborate with other companies or agencies, they feel like they belong to the same company. There are no more walls.\u201d<\/p>\n <\/p>\n When working with the Carolina Panthers on a recent Unreal-driven, augmented reality in-stadium presentation, Brami says it was as if the entire team was working in the same room. \u201cThere\u2019s no more posting; we just share our screens,\u201d he says. \u201cIt\u2019s just so much more efficient.\u201d Seen by millions on social media, the larger-than-life panther was brought to life by Zoic and Epic with help from Quince Imaging, Stype, Field Day Sound and Pixotope.<\/p>\n <\/p>\n Although Zoic still has an indefinite WFH policy in place, Brami can tap into the servers from home or head into Culver City for a session. \u201cI still go on-set too,\u201d he says. \u201cAnd I like being in the Flame bay, which we still offer for the client as well. So if you want to do finishing in the bay, that\u2019s an option. We\u2019re all vaccinated.\u201d<\/p>\n <\/p>\n For him, virtual production is just another workflow revolution that makes that collaboration possible. \u201cIt\u2019s like when we went from film to digital. That brought so much creative energy back into the process, and this is doing exactly the same thing. I\u2019m coming up on my 25th year working in this industry,\u201d Brami says. \u201cIt\u2019s never been about the project. For me, what\u2019s important is the people I work with. I love collaboration. It\u2019s why I\u2019m still crazy-passionate about this industry.\u201d<\/p>\n
\nOur Flag Means Death, Taika Waititi\u2019s latest series for HBO Max, takes place mostly at sea on a pirate ship. For the most part, it was shot in a virtual production volume on the Warner Bros. lot in Hollywood.<\/p>\n
\nJulien Brami, a VFX supervisor, creative director and Flame artist at\u00a0Zoic Studios, has always been interested in real-time filmmaking. \u201cI\u2019m attracted to everything and anything real-time,\u201d he says. After joining Zoic\u2019s Culver City office 7 years ago, he began experimenting with new workflows in Flame. \u201cFlame was one of the best places to start because you had the ability to do real-time comp timelines combined with 3D. But when I started seeing software like Unity and Epic\u2019s Unreal, I knew I should look into it.\u201d At the time, his best friend, J\u00e9r\u00f4me Platteaux, a VFX supervisor at ILM, added fuel to the fire. \u201cHe told me, \u2018You know, we\u2019ve got a secret project, so I can\u2019t tell you more, but I can tell you that you should really start learning Unreal right now.\u2019\u201d Brami even created a game he could play on his phone so he could better understand how the engine scaled and what he needed to do to make it work.<\/p>\n