On nomenclature

Ever since the graphics field got expanded by a variety of new gadgets such as the Oculus Rift, Microsoft’s Kinect and HoloLens or the Leap Motion, and new parties moved in to expand their usage (in particular Magic Leap), I have seen a whole brigade of marketeers and fanboys come with them who have simply made stuff up or juggled around with terms they are evidently confused about at best.

The first group likes to revise or invent new terms. For instance, using a different word to hide inefficiencies in a certain method. So instead of having augmenting objects, Microsoft prefers to call them Holograms. Why? Because in a see-through display like the HoloLens, you still can’t render proper black (that is, block light from passing through the lens entirely). Better find a new word for those translucent things on the screen. Additionally there are those who come up with entirely meaningless marketing terms like Cross-Reality (XR) or, for maximum chaos, Merged Reality.

The second is a consistently growing group of people who like to claim one of the following things:

  • Mixed Reality is Augmented Reality and vice versa
  • Mixed Reality = Virtual Reality + Augmented Reality
  • Mixed Reality is a superset of Virtual Reality
  • Mixed Reality = Some new I/O device
  • Mixed Reality and Augmented Reality are unrelated
  • Augmented Reality is the opposite of Virtual Reality

This group is not just simply wrong, but also responsible for the spread of mass-confusion around these terms which have been clearly defined for a long time and which have been used in many research papers.

Mixed Reality

The term Mixed Reality was introduced by Paul Milgram et al. in a 1994 publication called Augmented Reality: A class of displays on the reality-virtuality continuum. It presented the idea that between virtual and real there is a spectrum of different mixtures of both.

From Milgram et al., The Reality-Virtuality Continuum

For instance, there could be a real news-anchor inside a virtual news-room (a thing called Augmented Virtuality). Or a virtual object in a real setting, like Gollum in Lord of the Rings (a.k.a. Augmented Reality). Things don’t necessarily have to look real. The spectrum could be 2D, with one axis defining if something is added or even removed (the latter being called Diminished Reality). The gist of it though is that you can arbitrarily lerp between real and virtual with any degree to define realism.

The paper introduces the term Mixed Reality as follows:

Within this framework it is straightforward to define a generic Mixed Reality (MR) environment as one in which real world and virtual world objects are presented together within a single display, that is, anywhere between the extrema of the RV continuum.

This excludes two realities: For lack of a better term “Real Reality”, and the polar opposite, Virtual Reality. You don’t need a definition for the obvious though, because Mixed Reality implies that you are mixing realities in the first place.

Four stages on the RV continuum. Courtesy Vincent McCurley

There are those who think they might come up with a clever argument that the observer/controller/player of a Virtual Reality simulation is somehow the mixed-in real element and therefore any interaction with Virtual Reality constitutes a Mixed Reality. This is of course silly, since that would include people using a computer and render the term meaningless. It seems that the reasoning behind this line of thinking is that there is a fancy new input device such as the Kinect or the Leap Motion which reads “real gestures”, whereas your old-school mouse or keyboard is just not magical enough.

Conclusion

  • Virtual Reality is NOT Mixed Reality or included by it
  • Virtual Reality is NOT Augmented Reality or included by it
  • Augmented Reality IS a sub-type of Mixed Reality
  • Augmented Reality is NOT the opposite of Virtual Reality
  • Mixed Reality is MORE than Augmented Reality
  • Mixed Reality is NOT a new I/O device
  • AR + VR IS meaningless

Or to make it really simple: If you are not mixing two or more realities, you’re not doing Mixed Reality.

References

  1. Michael Abrash, Why You Won’t See Hard AR Anytime Soon
  2. Gamasutra, Magic Leap’s ‘chief game wizard’ has big ideas for the mixed reality future
  3. The Foundry, VR? AR? MR? Sorry, I’m confused.
  4. Twitter, #MixedReality
  5. Intel, Merged Reality
  6. Milgram et al., Augmented Reality: A class of displays on the reality-virtuality continuum