Volumetric photogrammetry — big words, bigger impact on VR

Virtual reality: By this point, most people have at least a rudimentary understanding of what the term means. Once enjoyed and developed for niche techie and gaming audiences, brands have quickly realized VR’s limitless potential, and, as a result, every interest group from interior design to makeup to apparel is jumping on the immersive content bandwagon.

For consumers, VR generally means strapping on a head-mounted display (HMD), stepping into a new world and enjoying the experience. The enveloping nature of VR allows people to explore environments in 360 degrees, but for most, how these immersive worlds are created is a mystery.

Though VR is still in its infancy, “traditional” methods of capturing and transforming footage have emerged. Typically, to shoot 360-degree VR content, a cameraperson employs several cameras rigged in a spherical formation to capture the scene. According to Alicia Millane’s blog entry on The Primacy, “Each camera is mounted at a specific angle so the camera’s field of view will overlap portions of the surrounding cameras’ field of view.” With the overlap, editors should be able to get more seamless footage, without any gaps. Alternatively, professional 360-degree cameras can be purchased, but more or less look and function the same as hand-rigged apparatuses. Once filming is completed, editors stitch together the footage, creating a unified, continuous experience.

In addition to camera formation, camera placement also plays a major role in the end result of a particular piece of immersive content. Depending on what the content creator wants the consumer to experience, camera placement will vary. Are consumers meant to be participants or spectators? Is the scene set low to the ground or up high? Should the rig be set up in the middle of the action or off to the side? Though the creative direction will ultimately determine placement, it is important to note that even with several rigs placed throughout a set, this method creates a more static outcome.

Enter volumetric photogrammetry. A mouthful, for sure, but this method of creating virtual environments could possibly hold the key to the future of VR. Unlike the method mentioned above, there are no takes or shots in volumetric VR that are later edited in post-production. This allows for a much more fluid experience, as the consumer frames the scene and chooses his or her own perspective. Using the volumetric capture method, footage of a real person is recorded from various viewpoints, after which software analyzes, compresses and recreates all the viewpoints of a fully volumetric 3D human.

With volumetric VR explained, photogrammetry’s defining characteristic is the principle of triangulation. As explained in a blog post on Viar360, triangulation involves taking photographs from at least two locations to form lines of sight. “These lines of sight are then mathematically intersected to produce the 3-dimensional coordinates of the points of interest.” This method is used extensively in video games. Case in point, “Star Wars: Battlefront.”

 

As explained by the game’s lead environment artist Andrew Hamilton, “Photogrammetry is essentially the technique of processing still images to produce a high resolution 3D mesh. There are a number approaches to this, but they all involve taking photos of a subject in real-life and running it through photogrammetry software.” Once the images are captured, the software then produces reference points, more or less connecting the dots, resulting in an intricate framework that can be used to construct lower-resolution game-ready environments.

From 8-bit graphics to increasingly realistic interactive worlds, cutting-edge technology has always had a place in video games — but what about other industries? The entertainment world, for example, which serves as a transportive medium, has recently taken note of how employing advanced capture techniques can enhance the viewer experience.

8i, a virtual reality software company focused on volumetric human capture, showcased its documentary, “#100humans” at the Sundance Film Festival in 2016, generating a maelstrom of interest, and helping cast a light on other projects at the forefront of 3D volumetric capture.

“Giant,” a short VR film set in an active war zone and shot with DepthKit, was shown at the festival just this year, confirming interest in the medium and cementing its staying power.

Musicians have also heeded the impact of volumetric capture, as evidenced by Maya Payne’s video for the song Breathe Again. Created by New Zealand’s realityvirtual.co, the video boasts a world’s first superlative and offers viewers true VR cinematography, or, what might be better known as 6 degrees of movement.

In the art world, VRt Ventures, a Los Angeles-based company archiving works of art in true VR, has bridged that gap. MOCA: Kerry James Marshall, explores the celebrated artist’s retrospective exhibition, “Mastry,” and was created utilizing volumetric photogrammetry to capture the air between the walls at MOCA’s Grand Avenue location so viewers could have proper perspective and dimensions.

Immersive experiences utilizing volumetric photogrammetry may convey a much more authentic and realistic environment to the end user. Per VRt Ventures founder, Jacob Koo, “If virtual reality has the chance to reach its full potential, then consumers must feel like they are actually somewhere they cannot be physically. That perception takes VR technology out of the novelty category and makes it something actually useful.”

Featured Image: Yagi Studio/Getty Images