Virtual production is the cutting-edge of modern film and video creation, and a strong source of job growth within the entertainment industry. Mindful of these facts, DeSales University of Center Valley, PA, is building its own virtual production studio (VP Studio) in its Labuda Center for the Performing Arts.
[Why dvLED Is Gaining Market Share]
The new studio, which is currently operating at an off-campus location, is built around an AOTO Electronics high-performance LED screen, which covers two sides of a corner wall. The video wall serves as a high-resolution greenscreen replacement for in-studio video productions, supporting interaction between human actors and computer-generated characters, props, and environments.
“Virtual production in this format relates to live action performers acting in front of an LED screen,” explained Karen Ruggles, associate professor of computer science at DeSales University and the visionary who led the systems design and integration team for this project. “On the LED screen is a photorealistic world constructed by computer science/digital art students. The final product is captured by a physical camera navigated by our TV/Film students.”
DeSales University and its Performing Arts Department built the VP Studio in an effort to "stay ahead of the curve when it comes to technology in the TV/film industry,” said Stephen Furry, the virtual production supervisor in charge of the new facility. He is also a 2001 DeSales University graduate and video production pro whose credits include Star Trek: Picard and The Mandalorian. “Virtual production is becoming commonplace in motion picture and television production.”
Moving (and Non-Moving) Parts
The heart of DeSales’ VP Studio is its LED video wall. It is composed of 96 AOTO Electronics RM series LED display panels (2.31 mm pixel pitch) assembled into two 13x10-foot sections that join at a 90-degree corner within the studio. The wall’s full-motion visuals are assembled on a computer known as the "Primary Operator" within the VP Studio’s control room, which is loaded with Epic Games' Unreal Engine 5 (UE5) software. “The Primary Operator also sends the high-resolution worlds processed by its GPUs to the LED wall,” Furry said.
A Mo-Sys StarTracker camera tracking system communicates with UE5, telling the software where the on-set physical camera is in relation to the world displayed on the video wall. “To record the performances, we use the Sony FS7 4K camera with prime lenses," Furry added. "The camera is mounted on a mini-jib arm with a remote head for easier adjustment of the camera positioning and flexible camera moves.”
To light performers and sets built in front of the LED video wall, the VP Studio uses LED lighting fixtures from Aputure. “Their Nova 600c and 300c series fixtures are RGB LED units that provide a wide range of color with built-in gels and lighting effects, and can be controlled through DMX to emulate lighting consistent with the virtual lighting within the worlds,” Furry explained.
The VP Studio’s camera tracking system is connected to the Primary Operator via UE5 plug-ins such as Live Link. The actors in front of the Sony camera can be shot as is, or they can don motion capture suits so their actions can be used to animate digital characters stored within the system. "We can create video games with characters,” explained Anne Lewis, division head of performing arts at DeSales University. “We can also capture dance on film, while being anywhere in the world.”
Whatever project is being produced, the Primary Operator renders the full motion backdrops in real time and sends them to the AOTA video wall. “From there, it’s like a normal film shot on location, except that the location is a completely digital world that my students can customize without traveling outside of the studio,” Ruggles said. “Furthermore, I can load up any kind of weather or time of day that I want and keep it there for hours—and then reload it up again tomorrow if we need to reshoot.”
“We are really only scratching the surface of what our students will imagine to fill the space.”
Karen Ruggles, DeSales University
Currently, DeSales University is storing all of its virtual worlds on the Primary Operator for easy access by students. “This saves time in post production by marrying the background to the performances,” said Furry. “The students’ footage can then be transferred directly to their external drives in one of the TV/Film Department's editing suites using Adobe Premiere or Avid Media Composer.”
Strange New Worlds
DeSales University recently broke ground on the addition to the Labuda Center that will be the permanent home for the VP Studio. Meanwhile, the university has been renting a bay in a local car painting facility so students can use the system. “Our vice president of campus environment, Marc Albanese, was instrumental in helping us to get the beta site space as well as getting the addition built," Lewis added.
So far, the university’s virtual production capabilities have been kept in-house, although the studio may also support commercial work down the road. “The classes that can utilize the VP Studio are only limited by our own imaginations,” said Lewis. “We are starting out with our computer science classes first, as they build the ‘worlds.' Then, cinematography classes will begin in the spring. It needs to follow this sequence, as the TV/Film students need the ‘worlds’ to be completed in order to shoot their films.”
Students are already producing scenes that are "all over the map, from galactic gambling and historic courtyards inspired by Shakespeare filled with choreographed expressive dance, to a man returning to his village to find it destroyed, and a post-apocalyptic scene with fire demons,” Ruggles said. “And this was just in one semester! We are really only scratching the surface of what our students will imagine to fill the space. I am excited to see what their creativity will unleash once they feel oriented to the technology through a few courses.”
Investment in the Future
When it comes to the university's eventual return on investment for the new studio, Lewis said, "Our ROI lies in having more students in TV/film and computer science wanting to study this new technology and preparing them to use it in the work world, as well as making some very fun and awesome films.”
Ruggles sees the ROI being spread across multiple departments. “Virtual production, in this format, involves computer science, TV/film, acting, and dancing students,” she explained. “There is a lot of opportunity for students to enter the space as a knowledgeable scholar, but only as one step in the process, so that they have to engage with other knowledgeable students in a different discipline and communicate for a common goal. In this way, they learn together and about each other’s discipline, which is really the whole purpose of a liberal arts education.”
[Beyond the Tech: Is Leadership Taught or Learned?]
Translating the VP Studio from concept to reality hasn’t been easy for DeSales University. “But sometimes, God just has a hand in aligning all the stars,” Lewis concluded. “He has certainly helped all of us realize this dream.”
