What is mixed reality?

Mixed reality basically describes the mixing of the “real” and “virtual” worlds through the use of electronic devices. The term “mixed reality” is closely related to the term “augmented reality”. These terms are frequently used interchangeably. The concept of mixed reality has gained new popularity through its use by Microsoft in its “Windows Mixed Reality” product range.

According to this model, mixed reality is everything that does not belong 100% to the “real environment” and is also not 100% virtual. The real environment on the one side and the virtual environment on the other constitute the extreme poles in this model – anything that lies between them and contains both elements is essentially mixed reality. The blending within mixed reality is seamless. The amount of “real” and “virtual” elements in a mixed-reality application is different in every case.

As previously mentioned, the terms mixed reality and augmented reality are often used interchangeably. According to the original definition, though, mixed reality covers significantly more aspects than augmented reality, in that it emphasises the interplay between reality and virtual objects. As a result, objects and/or subjects in the real world can interact with objects and/or subjects in the virtual world. For example, if you were to place a virtual object on a real table, mixed reality applications would register this event and then move the virtual element along with the table if you were to move it. Normal augmented reality apps do not react to such changes, meaning the virtual object then typically either floats in the air or disappears altogether.

Mixed reality is already being used in a wide variety of sectors today – in gaming and in online shops as well as in both industry and military affairs. In many cases, however, the apps being used are just AR apps which only allow real and virtual reality to interact in a limited number of ways. Mixed reality will play a significantly more decisive role in future apps.

Some examples of potential and current mixed reality applications include:

• Mobile games: The real environment is recorded by cameras on tablets and smartphones as a basis for the game. On our screens, we see game characters, objects, and so on embedded in the “real” world. The real environment is mixed with the virtual world on-screen in a way that the virtual elements react directly in relation to the circumstances.
• Online shops for furniture retailers: A smartphone camera records an image of a certain room. A virtual piece of furniture, such as a couch, is then virtually added as a three-dimensional object into the shot. This allows a client using the online shop to imagine what the couch would look like in their room, simplifying the purchase decision. However, the interactivity of many current apps still has a lot of room for improvement.
• Online shops for eyewear retailers: A camera takes a picture of a client’s face. The client then chooses a frame and can view what the glasses would look like on them on their screen. The client’s real face is mixed with the virtual model of the glasses.

In the e-commerce sector, mixed reality applications offer a wide range of possibilities to provide added value for the shopping experience. The more realistic and personalised the intended use of a product can be conveyed, the higher the customer’s trust, which increases the possibility of them making a purchase. Mixed reality can convey a shopping experience when customers use tablets, smartphones, or computers to simulate the experience of entering a real shop – and could even surpass it in some ways. Information about what you need to look out for when creating an online shop can be found in our article “Setting up an online store – the basics”.

Microsoft has given a boost to the idea of mixed reality with Windows 10. The company now offers an entire range of different applications under the “Windows Mixed Reality” section. The product range is based on the reality-virtuality continuum depicted above and is critical of the fact that most augmented reality applications only use a very small portion of the “mixed reality spectrum” – and that, due to inadequate technology, many of the possibilities that the mixing of real and virtual environments opens up are left unused. The video clip below provides a good idea of the approach that Microsoft is taking in relation to the concept of mixed reality:

According to Microsoft, Windows 10 was also designed with the aim of making greater use of the diverse possibilities of mixed reality. Microsoft differentiates between holographic and immersive devices in regard to its hardware for mixed reality applications:

Microsoft believes that the difference between holographic and immersive devices will be replaced by a combination of both approaches in the future.