(Stranger Team) Augmented Reality Representation of Digital Twins

06/11/201926/01/2023 J Rodriguez

Within the Onesait Platform Revolution initiative, the Stranger Team, made up of Miguel Angel, Jairo and Iñigo and mentored by Raquel and Pedro, opted for Digital Twins Augmented Reality Representation.

What is a Digital Twin?

A Digital Twin is a digital representation of a real-world entity or system that does not act as a replacement for the physical object of the system it represents, but rather as a replica of it, allowing communication (testing, monitoring, command) of this physical device. without having to be near it.

The challenge:

It consists of the creation of an IoT ecosystem in augmented reality, using the capabilities of the Onesait Platform. To do this, they have generated several virtual devices modeled as digital twins, which represent real devices with a state and that generate and react to events. These devices are located virtually by means of a QR code that serves as a public key in a secure link for their control and monitoring, and for the application itself that maintains security with the platform.

3 Digital Twins have been created on the platform:

Thermostat.
Alarm.
Smoke detector.

They created an Android application that represents, from a QR code, the three devices in Augmented Reality, with the information obtained from the corresponding Digital Twin platform. The flow is as follows:

Start the augmented reality application with the android device and perfrom the automatic and secure login on the Onesait Platform.
Detect and locate the devices with QR codes
Within the augmented reality space, click on the smoke detection object and send a request to the smoke detector‘s DT.
This Digital Twin executes its action and the corresponding event is triggered.
Through the orchestration of the FlowEngine, the Fire Alarm‘s digital twin reacts to that previous event and a call to action of the Alarm DT is made that causes a change of state, and that change of state is recorded in its ontology.
A message is inserted into the ST_NotificationMessage ontology along with the deviceToken data. that triggers a sending to the Android device by Google’s Firebase.
This makes the Alarm device react visually in the virtual reality space.
There is a thermostat generating temperatures, and a dashboard graphing everything. It is also possible to see this dashboard as a hub in virtual reality by clicking on the thermostat.

Platform components used:

Identity SSO: The authentication services provided by the platform are used to access the dashboard from the Android application that has been marked as private.
API Manager: Makes available the REST services that implement the functionality.
Flow Engine: Digital Twins Orchestration, this is where the entire flow of the application is defined.
Semantic Models: They created two ontologies that store device registration data, in addition to the ontologies created for the Digital Twins, where all the data provided by them is stored.
Dashboards: The data generated by the Digital Twins is displayed in a dashboard built on the platform.
Persistent: To persist data in ontologies.
Digital Brokers: For the insertion of data in the ontologies.
Microservice: This module could not be used in the challenge and the microservices have been deployed directly in our rancher, but the idea was to use this functionality to manage the microservices.
IoT Broker: For the insertion of data in the ontologies.
Report Engine: Dashboards are exported in image format to be able to embed them in the Android application and view them in AR together with the device.
Digital Twin: The DigitalTwin concept is used to model each of the devices and be able to represent its twin in RA.
Marketplace: Both the Android application and the Digital Twins code have been made available as a resource in the platform market, to make it accessible to potential users.