Imagine starting your day with augmented reality glasses. When you put them on, you can see notifications you missed through the night, the weather for the day, and traffic information as you walk to your car. Once inside your vehicle, you can use your glasses to adjust your AC, volume, and more.

These are some of the things that Microsoft has been working on with the engineers at the German car manufacturer Volkswagen. To get a better look at how some of these futuristic functions would play out, the two companies decided to put the current HoloLens 2 mixed-reality headset to use to see what they could come up with.

Read on to learn more!

The new “moving platform” mode for the HoloLens 2 mixed reality headset allows a user to operate the headset while on a moving vehicle like a boat or large marine vessel. This means that the headset can be used in new ways like training new employees virtually or creating new experiences for things like self-driving cars.

Obviously, Volkswagen is focused on the mobility of their new cars when it comes to the HoloLens 2, but the headset’s capabilities expand across many industries. Inside a conference room, the HoloLens 2 can create a virtual meeting space where remote workers can meet and collaborate. Avatars can be customized for each person, and business information can be viewed and shared in a secure fashion. When it comes to an industry like the medical field, graphs and charts can be viewed in 3D, giving medical providers a life-saving edge inside the hospital.

The HoloLens 2 also gives employees and customers alike a more positive, heightened experience. Volkswagen has stated numerous times in the past few years that they want to add more technologies to their vehicles to make them more convenient, safe, and futuristic. This provides a quality experience for the customers driving their cars, and this kind of outlook is no different for the millions of businesses across the world.

Mixed reality headsets like the HoloLens 2 can be paired with audio and video settings in an office space so the leader of the meeting can control things like volume, the design of the virtual meeting space, the location, design, and movement of holographic, home screen settings, and more.

Augmented reality has been imagined and prophesized for years, and even though consumer-friendly technologies have started becoming more common recently, augmented reality projects have been going on consistently for nearly ten years.

Back in 2015, Volkswagen used mixed reality to teach drivers how to properly race on a track. The training process, which was completed virtually through the HoloLens 2 headset, was similar to how an employer would train a prospective employee. Virtual steps and activities were created for training purposes, but these design and customization features only show off the cutting-edge power of the HoloLens 2 technology.

The team-up between Microsoft and Volkswagen also helped the tech company overcome a barrier that they were experiencing before. The Volkswagen team noticed that when the HoloLens 2 was used inside a moving vehicle, the sensors inside of the headset lost their tracking features, and every hologram that was seen suddenly disappeared. This led Microsoft to go back to the drawing board for their 3D computer vision.

Volkswagen later went to one of Microsoft’s headquarters in Washington so they could help with the new vision of augmented reality. There were several long conversations between both teams, but once Volkswagen pitched their use cases for the HoloLens 2, Microsoft became eager to come up with some solutions so that the HoloLens 2 could be used in more versatile ways.

Fast forward three years later to 2018, and the two teams began to work towards new sensors that would allow the HoloLens 2 to be used in a moving vehicle. The first obstacle they had to overcome was the headset’s sensors. The original sensors measured the HoloLens 2’s motion through visible light and inertia measurement units to gauge the headset’s acceleration and rotation speed. This is how the sensors replicate how humans move and see through the world.

However, there are motion sickness issues with the HoloLens 2 like someone would experience with a moving car or boat. When the coupled HoloLens sensors start working out of sync in a moving environment, the inertia measurement unit and cameras start malfunctioning and overall functions start falling apart.

To fix the motion issues that the HoloLens 2 was facing, the Microsoft team created an algorithm that models the issues between the HoloLens sensors so that they can continue to track. Although this was a useful move, there were still challenges with testing the algorithm in Microsoft’s Swiss headquarters because of their COVID restrictions. The sensors needed to be tested in a car with two people, but the restrictions made this impossible to do.

Instead, a fake moving platform was made so that testing could be continued. Bits of tape were put inside a plastic box so that the HoloLens sensors could find something to track, and buses and trains were used by Microsoft employees to make sure that the holographic inside the headset were still working. Some employees even rode the elevator inside their apartment complexes to make sure that moving platforms were no longer an issue.

Once a prototype was created, more testing was carried out. Boats were rented and taken out into the water and the HoloLens’s head-tracking system was tested to see if it could hold up to the constant motion.

This system was later put into Volkswagen prototypes to see if it could hold up to the movement of a car. Later on, the Volkswagen team created a bunch of demonstrational use cases that took a deeper look into how virtual interfaces could enhance the inside of a moving vehicle or platform.

The moving platform feature was introduced at the start of 2022 but has already had a lot of attraction from boating companies. The employers of these companies have been giving their HoloLens headsets to their employees and connecting them to mechanical experts through Microsoft Dynamics 365 Remote Assist. This app has given boaters access to hundreds of experts across the world so they can see through a person’s HoloLens, share what they are seeing, and figure out how to solve a problem when it arises.

The moving platform feature has now given maritime workers more accessibility while they are out at sea.

This feature can now be used on big ships, but as time goes on, Microsoft is planning on refining the sensors so they can be used in even more moving spaces. Connectivity is also a big thing with virtual technology, and all kinds of ideas are being thrown around about how to make these devices more compatible with your car.

Both the Microsoft and Volkswagen teams believe that augmented reality will become an increasingly important tool for modern living, especially as more compact mixed-reality devices, like smart glasses, come out. Mixed-reality glasses will be a new wave of mixed-reality devices, right after mobile phones and computers.

Mixed-reality devices need to be applicable to most daily tasks, and we’re not too far away from this. The HoloLens 2 puts a new idea in motion for future devices, and with the HoloLens becoming so useful, it’s pushing the envelope for smaller, faster, more useable devices like smart glasses. Once augmented reality becomes more seamless and connected with mobility experiences, new technological opportunities and experiences will only continue to blossom as these technologies improve.

To learn more about current HoloLens 2 and augmented reality products, get in contact with a Ultimate Technologies Group representative today!