In this blog post, we’ll explore the history, core technologies, and commercialization trends of HMD (head-mounted display) technology.
In a scene from the recent hit movie ‘Kingsman’, the protagonist wears special glasses that allow him to see other people in an empty conference room. These glasses can be seen as an example of augmented reality, which overlays virtual information onto the real world. Taking it a step further, if we could see a world composed entirely of virtual elements, we would be able to experience things that do not exist in reality or that we have never experienced before. Virtual reality technology—which bridges the gap between imagination and reality, offering experiences that are close to reality but not reality itself—is being actively researched due to its vast potential for application across various fields.
Among these, HMD (Head-Mounted Display) technology is considered the virtual reality technology closest to commercialization. Ivan E. Sutherland, a pioneer of virtual reality, proposed that HMDs would serve as a window into the virtual world and developed the first HMD in 1968. The equipment at the time featured small monitors positioned in front of both eyes to display 3D images, and a device connected to the ceiling detected the wearer’s orientation to provide corresponding visuals. Users could also manipulate objects displayed on the screen using a wand held in their hands.
The technology used in this early device can be broadly divided into two categories: one for rendering three-dimensional images, and the other for recognizing the wearer’s line of sight and reflecting it on the screen. To render three-dimensional images, it is necessary to understand how the visual system processes information. Since our eyes detect images focused on the curved two-dimensional surface of the retina via the optic nerve, projecting the appropriate light onto the retina tricks the brain into perceiving three dimensions. Sutherland created a sense of depth using an optical system composed of a miniaturized CRT and complex lenses, and employed tracking that detected ultrasonic waves emitted from the helmet in multiple directions to recognize the wearer’s line of sight. Although the equipment at the time was large and heavy, and the screen response was unnatural, these two technologies became the foundation for the subsequent development of HMDs.
In 1983, Eric Howlett invented the LEEP optical system, which contributed significantly to the commercialization of HMDs. The LEEP optical system is a technology that makes images from a flat panel appear as actual 3D images by passing them through eyepiece lenses on both sides. This method allows for the creation of a sense of depth simply by passing the image from a flat display—the kind we use daily—through an optical system. Subsequently, the development of ultra-compact panel technologies such as LCD, LED, and OLED replaced the heavy, bulky panels, and tracking technologies like gravity sensors and infrared sensors made it easier to track the user’s gaze. Furthermore, as computer processing power increased, allowing for more natural on-screen motion, various developers began creating devices for mass market sale.
A prime example is the Oculus Rift, created by Norwegian developer Palmer Luckey. Conventional HMDs required large, sophisticated lenses to produce high-resolution 3D images, but these complex lenses were difficult to manufacture and expensive. The Oculus Rift increased accessibility to the general public with its affordable price of around $300 and relatively compact size, and at its core lies the fisheye rendering technique.
Fisheye rendering is a method inspired by fisheye lenses; it distorts images in a specific way to provide a wider field of view and a higher sense of resolution. Although our eyes have a wide field of view, we actually obtain much more information from the central part of our vision and relatively less from the periphery. While fisheye rendering distorts the image, it increases the information density in the center, allowing for a wide field of view and high immersion even with inexpensive lenses.
As a result, the Oculus Rift is credited with revolutionizing HMDs, but fisheye rendering is not without its drawbacks. Initially developed for gaming, the Rift excels at providing immersion in a first-person perspective; however, in third-person or external viewpoints, peripheral image distortion can become an issue. In fact, the immersion of a screen viewed from the outside, such as in a movie, may actually decrease. Therefore, there are still many challenges to overcome to achieve a fully immersive virtual reality experience. Nevertheless, the Oculus Rift has become an important stepping stone for the next phase.
Meanwhile, Facebook’s acquisition of Oculus for approximately $2.3 billion in 2014 became a major topic of discussion. This was a sign that the public and investors were showing significant interest in next-generation VR and HMD technology. Facebook CEO Mark Zuckerberg mentioned expansion into various fields beyond gaming, and the subsequently released Oculus Rift DK2 demonstrated several improvements, such as enhanced resolution and software-based image distortion correction. Additionally, various HMD-related devices, such as Sony’s Morpheus, Glyph, and Virtuix Omni, are entering the market.
Currently, virtual reality technology centered on HMDs is experiencing a resurgence. With continued public interest and investment from major capital firms, the possibility that virtual reality experiences matching or exceeding our imaginations will soon become a reality has grown significantly. However, there remains a gap between the fully immersive virtual reality depicted in the media and current technology. To bridge this gap, not only capital investment but also continuous and meticulous development—without resting on current achievements—appears to be necessary.
