MaTrEx

Modules

We planned the solution in a modular fashion, the key modules being:

• Virtual Realy Environment
• The Networking Module
• The IoT module
• User Interface and Dashboard

How they all fit together

• The prototype comprises a room in Virtual Reality that is simple, spacious, and free of distractions.
• The room is shared by a trainer and N trainees. They can remotely join the room through the application.
• A “digital twin”, a 3D model resembling and connected to a machine in the real world, also exists in this room.
• As the trainer interacts with the digital twin, the trainees can observe and also interact with it.
• This can also facilitate ad-hoc remote maintenance by streaming diagnostic data in real-time using IoT.
• Each session can be recorded and replayed for re-training or reference.

Implementation

Virtual Reality Environment

We had to experiment a lot with the size of the VR environment and scale it to make sure it was comfortable for the actors and created a productive learning environment. We built the VR interactions on top of OpenVR for its multi-vendor support, meaning that in the multi-actor scenario, there would be no restriction on the hardware being used. One actor can use a top of the line Occulus Vive while the other can use a budget Windows MR and the application would comply.

The IoT module

For the prototype, since we lacked proper equipment, we made a functioning proof of concept by using a heat-sensor with an Arduino and establishing a connection with unity using MQTT protocol.

User Interface and Dashboard

We implemented a simple user interface and dashboard which would give actors the data in the VR environment in a graphical format.

Features

• Training of (multiple) on-site workers on potentially different sites by remote professionals. Training in VR provides an effective learning experience by training people in the most realistic way possible, without the situation actually happening in real life.
• The database of these sessions can be revisited in VR for future reference. One key advantage of VR is that the sharing of virtual content can lead to more robust replication and representative sampling. A database of standardized content will allow the training of on-site workers more efficiently.
• Ad-hoc maintenance where a remote profession can review the digital twin with real-time data and provide guidance to the on-site workers. These sessions can even be recorded for a particular model for future reference and maintenance.
• A Pooling session where multiple professions get access to the digital twin and each follows their approach which is simulated and the optimal solution is implemented. Since it’s multiplayer, multiple numbers of on-site workers can also work on the model or get trained to work with the necessary parts of the model.

Challenges

• We tested with the few VR headsets as we could get our hands on. We would like to stress test further how many people can feaseably learn in such a test environment.
• Considerable need for storage space for the projects and recording of VR sessions.
• Lack of actual machinery to test with a major constrant. We created a simulated proof of concept which should be feaseable to scale.

Results & Future Work

The experiment serves as a proof of concept that Virtual Reality can serve as a viable platform for demonstration training on the Industrial Scale.

Future plans include conducting more in-depth user testing and analyzing their experience juxtaposed against conventional training methods.

Additionally, the team would like to explore the effect of various kinds of input mappings on the learning experience along with the possibility of the addition of more tactile elements like haptic responses to make the environment more immersive.