Back to selection
LED Walls: Working with the LED Volume for Cinematographers
Surrounding actors with LEDs to light them with images of virtual sets Origin of Image-Based Lighting
The LED Volume is derived from the concept of Image-Based Lighting (IBL) invented by Paul Debevec in 1998, which refers to mapping an HDR image onto a photosphere enveloped around a subject to serve as primary light source in CGI.
The Light Stage 3 (2002) was the first physical set to use an array of RGB LED lights surrounding an actor to create an omnidirectional source. Although its 156 LED bulbs did not have enough density for close-ups or reflective elements, Light Stage 3 laid the foundation for how an LED stage could become an omnidirectional display device, presenting a panoramic image of the environment to the actor. This stage was used in Spider-Man 3 (2007) and The Social Network (2010) to capture references to help with 3D composites and face replacement. In 2004, Digital Domain used a Virtual Production LED panel to run tests for The Curious Case of Benjamin Button. From here, “The Light Box” was developed for Gravity (2011), while Rogue One (2016) saw the first wrap-around wall.
Set-Up
LED volume walls can be flat, curved or horseshoe-shaped, and can include ceiling panels and wild walls. When setting up a stage, it is imperative to consider the requirements and goals of the production, which dictate power requirements, what panels might be removed or added and the off-camera area dedicated to tech stations, hardware and gear storage. No one solution fits all. Shooting with an LED wall requires extensive preparation from every department, a render engine (Unreal, Unity, Stagecraft, Disguise, Pixera) and a sophisticated workflow.
Prep
Virtual location scouts can be done remotely with hardware such as virtual reality headsets. Most virtually-scouted sets have baked-in lighting, offering performance benefits, while dynamic lighting calculates light in real time, allowing for changes.
Working with the Volume requires a bigger team. Communication is key to achieve continuity in lighting between foreground and background. All the lights in the virtual world need to be organized and labeled correctly as they often simulate real units with a myriad of modifiers like fresnel lenses, barn doors, etc.
In prep, a virtual gaffer pre-lights the stage, working closely with the VAD (Virtual Art Department) team. This step can be done remotely, and the lighting scenario can be saved and re-used in later productions. Unlike any other form of production, it is necessary to have the camera and lenses you are using, along with a calibrated monitor to pre-light an LED stage, because of the metamerism and other imperceptible artifacts occurring between our eyes and the camera. You will essentially work with two gaffers—one in charge of virtual lighting, one in charge of practical lighting on set.
While an LED Volume can nearly wrap around a scene at 360 degrees, the gap between the wall and floor remains a formidable issue. From the camera’s perspective, the disconnect may break the illusion of continuous space if not properly mitigated. While the blend between the floor of the set and the LED wall might look fine to our eyes, it could look completely inaccurate in camera. Production design plays a crucial role in hiding this seam, integrating foreground and midground elements that interact naturally with the actors. The right blend can be achieved by elevating the set floor and tweaking stage lighting, as well as adjusting part of the virtual environment using CCR (Color Correction Regions) until the gap disappears to camera.
Image Processing
One of the most important factors to determine workload and set up parameters regarding processing power is the type of dimensional image processing the production will require. 2D consists of captured plates or rendered graphics characterized by permanent parallax; therefore, the camera should not move. 2.5D distributes 2D layers with alpha channels in a 3D space and creates an illusion of parallax. It is very effective at creating a parallax while being more efficient for rendering. 3D preserves parallax and can have dynamic or permanent lighting. It is best used when the practical set extends into the Volume. 3D image processing is very demanding and introduces potential latency issues resolved through foveated rendering or limited camera tracking. Synchronization between all systems is paramount for accurate image capture; otherwise, the illusion falls apart. This requires compromises to be made when pushing the technology beyond its current limitations, which can cause frames to drop.
The Volume control, known as the Brain Bar, is responsible for managing all the technology for delivering the pixels to the LED panels. The closest collaborator of the cinematographer from pre-production on is the VP Supervisor, who communicates to a vast number of engineers, operators and technicians, including the Virtual Art Department (VAD). Once the camera is tracked and lens information transferred to the graphic generative software, a full-resolution image is created in the area seen within the frame of the camera, referred to as the frustum. By isolating the area visible to the camera for full resolution, and rendering the rest of the scene at a lower resolution, the overall processing load is lightened, reducing potential latency and other artifacting.
The “control” node is the main user control for operation of the system. It drives all render nodes and synchronizes them by delivering real-time data, such as camera position. LED tiles are usually square and include hundreds of LED diodes. Each tile has a receiving card, where the display signal transmission is received from the LED processor through an ethernet cable. The processor maps values to the appropriate brightness levels and is responsible for the proper assignment of the digital content displayed.
Color Management
The goal for color management on the LED wall is to manage consistency and accuracy throughout the process to make the ICVFX (In-Camera Visual Effects) look seamless with the foreground elements (actors, props, etc).
Rod Bogart, principal color scientist in the Virtual Production Tools team at Epic Games, notes that the easiest way to keep the consistency of color through the pipeline is to ensure that the color space from one thing to another matches. The recommended approach is to have both in a wider color gamut so the result can represent more of what goes on in the real world. Unmatched color spaces will lead to visual artifacts such as banding, loss of saturation, contrast and color shifts.
Real-time graphics engines calculate the lights and shading by using linear values. In Unreal, it is preferred to use units that relate to the real world rather than using arbitrary numbers so that when the image is sent to the wall, the real-world reading will have a known relationship to the units in the virtual scene values. For example, a software value of 5.3 equals 530 candelas per square meter on the wall. LEDs are not linear, so the dynamic range of their output is mapped to the software linear curve. If, in the virtual world, an object is twice as bright as the background, this should match the LED wall. Game engines simulate cameras in their render. A final step called “Tone Mapper” or “Camera Emulation Pass” should be skipped to avoid double LUT-ing the image by baking in bloom, vignetting, white balance and auto exposure.
LED Specifications
While cheaper and lighter tiles are often used as ceiling panels, the average weight of an LED wall is around 50 lbs/m², which calls for a proper rigging plan (often provided by the manufacturer).
Frieder Hochheim and Ramiro Montes de Oca of Kino Flo explained that an LED panel’s refresh rate determines the maximum frame rate a camera can capture. To shoot at 60fps, for example, a standard panel requires a refresh rate of at least 7,640 Hz. For optimal efficiency at standard frame rates, the LED refresh rate should be no lower than 3,840 Hz.
Another potential issue is scan rate, which refers to how many LED diodes refresh simultaneously. A high scan rate (e.g. 1/28) can create visible scan lines or even chunks of the LED wall refreshing in sync, causing distracting visual artifacts. The lower scan rate (i.e. 1/4, 1/1) is better if we have a heavily-rendered 3D background.
The angular coverage (viewing angle) of the diodes should be at least 170 degrees. Beyond a certain limit, diffraction causes green or magenta shifts, restricting camera positioning. Another challenge is pixel pitch. Pixel pitch refers to the distance from a location on a pixel to the same location on a neighboring pixel. This determines the maximum proximity of the camera to the wall. Placing the camera too close to the panel can cause moiré. While a 2mm pitch allows filming as close as 13 feet to the wall, a 10mm pitch requires about 65 feet. Most Volumes typically utilize a pixel pitch between 1.5-2.8mm. The narrowest pitch currently available is 0.6mm (e.g. Planar DirectLight Ultra 0.6), but such panels are not optimized for virtual production due to their lower refresh rates.
Lighting
LED screens were originally created as display devices, not as light sources. Their energy efficiency comes from turning on the electricity consumed into the narrowest wavelength possible, generating a high peak count of light but not necessarily quality. LEDs are not mapped linearly and are not full spectrum. Since LED pixels consist of red, green and blue (RGB) diodes, true white doesn’t naturally exist. Even if they can create any apparent color, skin tones can easily look off if lit exclusively by the LED Volume. In effort to make up for their lack of full-spectrum coverage, manufacturers have added up to nine channel sources to introduce white, yellow, amber, cyan and turquoise. White is used as often as possible, while RGB channels are rarely simultaneously on. An alternative is Brompton Technology’s TrueLight® processor, which expands the color range of RGB LEDs to better emulate true white by using X-ray emitters to expand the range of colors on RGB.
Paul Debevec ran multiple HDR tests, taking images of actors and color charts in real locations and trying to reproduce them inside the Volume under each of the LED colors. As a result, he developed a 3 Matrix color correction technique. First, a post-production matrix is applied to the camera 3×3 matrix, which changes the way the photosites are processed. While this improves color rendition, it will interfere with the background image in the frustum. The way to fix this is to apply the inverse of the first matrix to the area inside of the frustum so that it lands back where it should. LEDs are not black when they turn off as most of them have a small reflector behind the diode to increase brightness, which also causes reflections and loss of contrast. A black level subtraction can be applied to achieve the right contrast.
Most tiles are capable of supplying 5K-6K nits. Cinematographers often find themselves running these LEDs at 20% or less, which generates a lot of heat and leads to more frequent calibration of the panels. Some panels might have a cooling fan, but it should be off when shooting because the Volume itself creates an echo chamber. Sound blankets or DIY “Echo Shades” are often used to break up the sound bouncing in the chamber. Since the brightness is controlled by pulse with modulation (PWM), when running LEDs that low the diodes are more often off than on. This, along with a slow refresh rate, can cause flicker. While LEDs can get very bright, they cannot match the sun. If you are using a real world IBL image, an effective technique developed by Debevec is to run it through a dilation algorithm. This will first identify all points of the image that would clip if sent to the LED panel and create a shape big enough so that the average pixel value of that region is just below clipping level. You won’t get a hard shadow, but the brightness on your subject will be close to the real world. If you need hard shadows, you still have to use cinema lights.
The way LED panels are assembled impacts troubleshooting on set. Some cabinet designs allow individual tiles to be replaced or removed easily, allowing access for lighting, while others require dismantling multiple components to fix a single malfunctioning diode. Bright windows and midday sun are scenarios to be avoided in the Volume because of the panels’ fall-off. Objects closer to the Volume will seem brighter than the ones further away, and the bright panels can potentially fight against the hard shadows in the foreground. Productions that have shot outdoor looks effectively (Oblivion, Mandalorian, House of the Dragon, 1899) were successful because the foreground is motivated to be in shadow or diffused.
An advantage of using an LED wall is that you can dial the contrast in the background, simulating different depths. They are very good at providing a soft ambient light ideal for dawn and dusk scenarios—which, in reality, offer tight shooting windows—as well as for shooting reflective objects like cars. Reflective surfaces might reveal areas of the stage not covered by the wall and, if curved enough, could magnify the panels to the point of revealing the single diodes. David Procter, BSC has spoken about lifting the area outside of the frustum brighter than the frustum to enhance the ambient light in a scene. “Lighting with a Volume allows flying in digital cards such as patches of colors, negative fills or even moving graphics displayed strategically on the screen to control lighting.” Procter added that atmosphere and particles can help blend the foreground and the background, as well as hiding moiré, but could cause loss of tracking data and contrast.
Cameras and Lenses
Ideally, any production-specific camera and display test should be done using the same camera, lenses and LED panel systems which will be used during production. When it comes to lenses, primes are favored because they usually allow for less depth of field. However, while zoom lenses can be calibrated once, primes might need additional calibration time if swapping lenses. Anamorphics offer a limited depth of field, and therefore can help keep the LED background out of focus, but require more time to be calibrated. Lenses are individually mapped using a series of charts on the stage. Optical distances, distortion coefficients and FIZ data are used to update the environment displayed on the LED Volume. With this step, the lens characteristics can be controlled and focus will change according to the actual focus position on the lens barrel. Procter suggests allowing at least 30 minutes per prime lens for VP lens mapping—an important consideration when planning prep time, which varies greatly from traditional film production.
Moiré is one of the most common issues when working with the LED stages, occurring when two patterns very similar to each other interfere. The camera sensor can interfere with a similar grid pattern from the individual LED diode. To avoid moiré, it is helpful to keep the depth of field to a minimum by using longer lenses, larger apertures and focus distances closer to the camera and farther from the LED wall. Larger sensors lower the potential for producing unwanted moiré as the depth of field diminishes.
Global vs. Rolling Shutter
Global shutter cameras are preferred because all pixels are read out simultaneously, while a rolling shutter sensor reads the signal of the pixel one line at a time. On a Volume, rolling shutters can cause artifacts such as skewed lines when panning quickly, partial exposure when using flashing light and bright horizontal lines when tilting, as both camera and LEDs are scanning vertically. Necessary compensations inside the LED electronics are needed to prevent this artifact.
Synchronization
The display system, tracking system and game engine should all be synchronized with the camera. The frame rate on the original plate content, media server or playback device and processor feeding the LED panels should match the shooting camera’s frame rate from the final pixel on the day of production. Genlock uses a pulse to synchronize the many devices within the entire system to ensure that all units generate or capture the start of frames at exactly the same time.
Sub-Frames
Frame remapping is a technique for multiple cameras to view different content displayed concurrently on the same LED screen. By operating the LED wall at multiples of the project frame rate, one can interleave frames of different content and display them at the same time. Each camera is sent a unique phase delay from a common genlock signal, and each shutter is synchronized to capture only one of the interleaved video signals displayed on the LED wall.
The virtual background can now display multiple images at nanosecond intervals which are then synced with the camera in order to capture multiple different backgrounds simultaneously. GhostFrame is a software tool set within Megapixel’s HELIOS LED processing platform that allows for ICVFX workflow. To the naked eye, the image is a single, persistent scene displayed on the wall, but to the camera, up to four perspectives can be recorded in a single take including tracking marks, Chroma key mattes, multiple lighting conditions and displaying cues for actors’ sightline. This process of capturing both the intended lighting of a scene, paired with the lighting of a second scene, as well as a chromakey frame, allows the director to make changes to the background in post while retaining a general feeling of the surrounding environment. The clear drawback to sub-framing is the extreme demand on the computer process, which must render multiple scenes synced with compatible lighting and potentially multiple cameras as well as the compromise to shutter angle and frame per second recording required to include the capture of multiples of each frame in the same interval of time. While global shutter cameras work well with this technique, faster exposure time or smaller shutter angle should be used with rolling shutters to avoid capturing multiple frames on the LED.
Camera Tracking
Real-time camera tracking is what sets virtual production apart from its predecessor, rear projection, which relied on static 2D backgrounds. By integrating real-time rendering, virtual production offers filmmakers the ability to dynamically adjust perspective and parallax between the background and subject as the camera moves.
Outside-in Optical Tracking Systems use a network of stationary infrared cameras positioned around a designated Volume. These cameras detect markers placed within the space and determine their 3D positions by analyzing the overlapping views from multiple cameras. Common solutions include OptiTrack, Vicon and BlackTrax.
Inside-Out, Feature-Based Optical Tracking is another form of optical tracking that involves mounting a computer vision system and time-of-flight depth-sensing array onto the camera. The system generates a 3D point cloud of the environment and uses machine learning to track the camera’s position in the space surrounding it. Similar to Xbox’s Kinect technology, this method eliminates the need for physical markers or tracking lighthouses. A leading example of this technology is Ncam, known for its efficiency and flexibility.
Marker-Based Optical Tracking relies on an infrared sensor attached to the camera, which detects specific marker patterns placed within the environment. Systems such as Stype, MoSys and Lightcraft Previzion use this method. While effective in controlled studio environments, this approach can be more challenging to implement outdoors or in locations where placing markers is impractical.
Encoder-Based Tracking Systems use magnetic or optical encoders to track the movements of camera support equipment such as tripod heads, dolly wheels and crane fulcrums. These encoders translate mechanical motion into positional data, determining the camera’s location in 3D space. Examples of integrated encoder-based solutions include StypeKit, Egripment and Technocrane. Although these systems require more initial setup and calibration, they offer high precision for virtual production workflows.
Beyond cameras, tracking technology is also applied to tracking lights, objects and actors. For instance, stage lighting equipment can be equipped with tracking sensors to sync their real-world positions with virtual lighting systems in Unreal Engine, allowing seamless interaction between physical and digital elements.
Actor tracking also plays a crucial role in virtual production. A notable example is The Mandalorian, where certain shots were too complex to be captured entirely in-camera using LED walls. To address this, filmmakers tracked actors’ positions and placed green screen halos directly behind them, tightly synced to their movements and the camera’s position simultaneously with OptiTrak’s motion capture system, an outside-in optical tracking system, as they shot in a stage already equipped with this system. This technique preserved interactive lighting while ensuring clean green-screen footage for post-production compositing.
The Future of Virtual Production
LED Volume stages can present considerable drawbacks. They are expensive to install and maintain, draw significant amounts of power, can lack flexibility in their arrangement, experience color shifts over time, suffer increased latency across wider arrays, result in aliasing artifacts and can be too dim to achieve desired contrast ratios. It is important to identify upfront whether or not a Volume is the right tool for your production’s needs.
Some tools at your disposal to help with common issues are utilizing your VP Supervisor as a go-to for the Unreal team, global shutter, defocusing the screen, narrowing the depth of field, use of atmosphere (practical or on-screen), closing the gap between the screen and the floor, use of foreground elements, supplemental lighting for skintones, extending the frustum 20 to 30% to allow for rendering during camera moves, and brightening the screen outside the frustum to enhance the ambient light, just to name a few. Reset your eyes periodically by walking outside, away from the screen, and bring a laser pointer for pinpointing areas on screens and in the space when communicating with your teams.
Traditionally, productions run out of budget by the time physical production is wrapped, and producers have to find funding to complete post-production. When a production uses an LED stage, all assets must be created and tested in pre-production; therefore, the budget expected to be dedicated to post is needed up front. Because of this, and by nature of the extended timeline, feature films and TV productions have the highest potential to benefit from using the Volume.
This change in production rhythm and expectations has contributed to producers shying away from shooting on the Volume. In addition, studios often find themselves relying on the chroma key set of frames, which allows for more changes—and costs—when compositing the background in post. Because of this, a lot of recent productions are going back to shooting traditional blue or green screens, which offer a more familiar workflow from budgeting to delivery.
The intrinsic issues and limitations of LED walls, conventional light fixtures, camera sensors and processors, spatial trackers, 3D rendering, plate acquisition and implementation and electrical load impose serious time and cost parameters on a cinematographer’s creative intent. Serious and meaningful progress is paramount to justify the immense cost and challenges posed by a virtual production. Fortunately, the compounding result of one technological advancement can have a ripple effect throughout each individual element involved in the virtual production pipeline.
While part of the industry is looking back to traditional blue and green screens, others are now moving to the next step of virtual production by introducing phosphor laser projectors to replace the LED wall. The phosphor laser brings back the age-old process of rear projection but with a few tricks up its sleeve. A phosphor laser is a solid state light produced using a digital light projection system known as a DLP. The laser begins as a blue beam around 445nm, which illuminates a yellow-coated phosphor disk, where the blue passes through filtration, but a new yellow beam is created. This yellow beam is then separated using dichroic coatings into a red and a green. By utilizing this process, white light is emitted from the red, green and blue beams onto a surface with much greater efficiency and luminosity than standard diodes in an LED panel. The laser projector can also house a combination of light emitters such as combining both blue and red lasers in conjunction, or pairing LEDs with the phosphor laser formation. A single DLP chip can average more than 20,000 lumens, with some models reaching 32,000, compared to the 8000 to 9000 lumens achieved through a LED panel. Secondly, projectors can be paired in an array with variable mapping that is able to contour to image surfaces that are three dimensional, allowing for the integration of physical objects inside the soundstage to have a plate image or color projected onto it. The introduction of these new lasers provides new flexibility in arrangement and operation, power consumption efficiency, greater contrast and continued integration with new processes such as subframe capture.
Resources
Zwerman, Susan and Jeffrey A. Okun. The VES Handbook of Virtual Production, Routledge, 2024.
Debevec, Paul, et al. “Light Stage 3: Surrounding Actors with LEDs to Light them with Images of Virtual Sets”, https://www.pauldebevec.com/Research/LS3/, Accessed 03/15/25
“Color Management for Virtual Production – Expert Tips.” YouTube, 6 Sept. 2023, www.youtube.com/watch?v=t1pZrGLsXm4.
Kadner, Noah, et al. “THE VIRTUAL PRODUCTION FIELD GUIDE VOLUME 2.” Epic Games, by Epic Games, 2021.
“Sub-Framing Technology in Virtual Production and XR Broadcast | ROE Visual.” ROE Visual, www.roevisual.com/nl-en/knowledge-and-support/sub-framing-technology-in-virtual-production-and-xr-broadcast.
“Laser Phosphor.” Projector Reviews, 27 Jan. 2022, www.projectorreviews.com/terms/laser-phosphor/.
“Mimik VR.” Kino Flo, 13 Aug. 2024, kinoflo.com/mimik/
“Considerations When Choosing Display Wall Technology.” Christie Digital, Christie Digital, 2015, www.christiedigital.com/help-center/whitepapers?pageNum=2
“Digital Projection Delivers VR Versatility for State-of-the-Art Studio.” Digital Projection, 27 Sept. 2023, www.digitalprojection.com/en-us/case-study/delivering-vr-versatility-for-state-of-the-art-studio/
Hochheim, Frieder and Montes De Oca, Ramiro (Kino Flo Lighting Systems). “Working with the Volume and Developing Technologies”. 19 February 2025. Zoom Online Meeting. Los Angeles, CA.
Procter, David (BSC Member). “Working with the Volume”. 13 March 2025. Google Meet Online Meeting. Los Angeles, CA and London, United Kingdom.
Images
Debevec, Paul, et al. “A Lighting Reproduction Approach to Live-Action Compositing”, SIGGRAPH 2002,07/21/02
“Paul Debevec FMX 2022 Virtual Production: Getting the Lighting Right”, 8/20/22, https://www.youtube.com/watch?v=m-GG92moxnM
“In-Camera VFX with UE4 | SIGGRAPH 2019 | Unreal Engine”, 8/19/19, https://www.youtube.com/watch?v=vyYXLEz0k1Y
