VR, AR, and MR explained

Virtual reality, augmented reality, mixed reality: what's the difference between all these different types of unreality? While they're all used in the gaming world in some way or another, each of these categories have their own unique features and uses. Let's break down the major differences between the three.

To get the short answer out of the way first:

Virtual reality is being fully immersed in a virtual world.

Augmented reality involves projecting virtual imagery on top of the real world.

Mixed reality combines virtual and real-world objects to create a hybrid world, or is a blend of AR and VR.

Still confused? Let's have a look at each of the three in detail—and how they're used in the world of gaming.

Virtual Reality

The Vive and roomspace Oculus experiences (including the new wireless, standalone Quest) have a clever approach to letting you move around the VR game world safely. They set up the dimensions of your 'safe' gaming space using IR base stations or inside-out tracking via cameras on the headset itself to track your location within a room. Then, when you're at the edge of that safe area, the headset warns you by displaying a bounding box. By gently reminding you of the real world before you kick your foot on the couch, these integrations of the real world borrow concepts from augmented reality.

Virtual reality is probably the concept you're most familiar with as a PC gamer, with many of the best VR headsets requiring a powerful rig to play properly.

The concept of VR is simple. You fill your visual field with the game's world, viewed through a motion-tracking headset that replaces your entire field of view with a screen. Coupled with headphones, good virtual reality can make you feel completely removed from your physical location.

VR has traditionally been the holy grail of games, the idea of full immersion with no distractions, but it's the hardest to execute technologically. Nintendo's 1995 attempt at a VR headset, the Virtual Boy, is remembered not as a revolution but as one of the biggest flops in gaming history.

VR only recently got its second wind, with 2016 seeing a trio of mainstream headsets launched to the public - the Oculus Rift, HTC Vive and PlayStation VR. These three brands helped massively broaden awareness of VR, offering high visual quality, low latency, and impressive motion tracking at a price point low enough for enthusiastic gamers.

This new generation of VR headsets vary in price and quality, with the PSVR offering a budget option for console gamers, and high-end Vive Pro kits going for $2000 and over (and that's before factoring in the price of a PC powerful enough to run it). However, most of them rely on the same technology to provide their virtual experiences.

VR headsets, or head mounted displays (HMDs), use stereoscopic imagery to trick the brain into seeing a 3D image from 2D images on a screen. To do this, headsets will either use one display for each eye showing a slightly different image, or a single display showing two different image feeds. Headsets rely on high refresh rates to help mitigate nausea (which is most extreme when images stutter or freeze) and to improve immersion.

Beyond the headsets' displays, VR tech relies on a number of sensors, both in the headsets themselves and in the controllers used and positioned within the play space itself. Different manufacturers utilize different sensor technology. The Lighthouse technology deployed in the Valve Index and HTC Vive HMDs uses base stations to sweep a room with invisible lasers that detect the position and motion of the headset and controllers. The sensors sold with the Oculus Rift, on the other hand, are essentially webcams that detect the infrared light coming off the Rift headset and Touch controllers to find their position in 3D space. 

Standalone VR solutions take advantage of tech that the mobile industry has been perfecting for years for use in smartphones. They pack cameras integrated into the headset as well as gyroscopes to measure rotation, magnetometers which find magnetic North to tell which way a device is pointed, and accelerometers, which sense the pull of gravity to figure out orientation and how fast a device is moving through space. Cameras, sensors, or a combination of both provide inside out tracking obviating the need for additional external sensors.

Despite the advanced technology on display, VR still faces a number of notable obstacles. Skyrim's VR port, for example, shows how traversal is a big issue in VR. The game asks players to bunny-hop around its world with its awkward default teleport function, a way of moving that's awkward to explore with and even harder to use during combat. It's no surprise that most players opt out of this control scheme, instead using a regular controller to move. This may be a more familiar control scheme, but it's also liable to cause motion sickness.

https://www.youtube.com/watch?v=TznIBmvKvXs

Augmented Reality

If the idea of virtual reality is to bring you into the game world, think of augmented reality as bringing the game into your world. AR has long been used for gaming, though it doesn't have a massive presence on PC, with most of its hardware and games existing on consoles, handhelds, and mobile devices.

AR is probably the simplest form of virtual reality, often involving nothing more complex than displaying virtual imagery over video feed of the real world. 

While many people immediately think of Pokémon GO, the history of AR gaming goes back much further. AR first came to prominence in games in the early 2000s, when the PlayStation EyeToy used camera images of players in mini-games. The Kinect did the same some years later on the Xbox 360, and Nintendo got in on the action soon after when the 3DS came packaged with a number of AR cards. The 3DS could use these cards to display either images of Nintendo characters sitting on your furniture or play minigames on the various surfaces of your home.

AR was brought fully back into the public consciousness with the release of Pokémon GO in 2016, although the augmented reality elements turned out to be completely optional, more a gimmick than a core part of the gameplay. Since its release, Pokémon GO has updated its technology to support 'AR+', where imagery of Pokémon in the game isn't simply superimposed over your camera feed but 'stuck' to a certain point in the real world.

The original Pokémon GO features AR at its simplest, game elements or graphics that are superimposed over a live video feed from a camera. An added level of complexity uses image recognition technology, motion tracking, and elements like light sensors to position objects and imagery more realistically within a live camera feed. 

The most complex AR tech involves wearable goggles like Microsoft's Hololens to project an image directly onto the real world. The latest version of Hololens shoots lasers at rapidly oscillating mirrors, which then reflect light on displays called waveguides to create holograms. In the past few months, these dedicated AR solutions have been largely repositioned as enterprise devices for large businesses rather consumer/gaming offerings (and a number of companies producing them have folded).

In contrast to virtual reality, AR's biggest weakness is that it's not fully immersive. AR features are often just an optional novelty, rather than being baked in as a central component of gameplay. It's fun to take a photo of Yoshi sitting on top of your dog's head, for example, but the application of AR is often too clunky to make it enjoyable or meaningful.

There may be more promise for gamers in the next generation of augmented reality, however: the technology (sometimes) dubbed 'mixed reality'.

Mixed Reality

Whether AR and MR are truly different often depends on who you ask—many people still use the terms interchangeably. Microsoft has considerably muddied these waters, calling HMDs that blend AR and VR "Windows Mixed Reality" while also referring to Hololens as a mixed reality headset. Thanks, guys.

For our purposes, mixed reality is tech that uses more complex integrations of virtual objects and entities into real world environments. For example, AR may involve pinning an image of a Pikachu to your coffee table, while MR would be complex enough that the Pikachu would disappear from sight if you moved around a corner or put an obstacle in front of it. 

Mixed reality gaming is still more hypothetical than real, however. While some of the demos and promised applications of the latest generation of Hololens sound closer to truly mixed reality than augmented reality, we have yet to see them deployed in the wild. Microsoft showed off the HoloLens at E3 2015, promising it would "change the world of gaming", but HoloLens for gaming never quite eventuated like it was expected to. If you go to Microsoft's homepage for the HoloLens now, it's unambiguously branded as "mixed reality technology for business", even though a small number of games come packaged on the enterprise-edition headsets.

The biggest issues with MR currently are all to do with the technology—as it is still quite new, there are plenty of limitations to what it can do. The HoloLens suffered most from its small field of view and relatively low visual quality, though these are issues that will no doubt improve as the tech is developed further. The new edition HoloLens 2 has already made some broad strides in these areas.

VR, AR, and MR are all distinct technologies within the broader category of virtual reality, all of them with their own unique pros and cons. Because these technologies are all still relatively new, there's some inevitable bleeding together of terminology as manufacturers and developers test the waters and fence off their walled gardens. Of the three, VR is certainly the most dominant in gaming, but it's likely we'll also see MR making inroads into that world in the years to come.