Writers

KIM Geohwa Senior Surveyor, LEE Eunjoo Surveyor

Digitalization Team

Due to the rapid development of new technologies such as artificial intelligence, big data, and IoT, securing competitiveness by utilizing these technologies across all industries has become a trend. They are being applied in various fields such as manufacturing, aerospace, smart city, construction, medical, and logistics. There is significant movement to introduce new technologies in the shipbuilding and offshore sector. In this study, we applied mixed reality technology to build a ship's digital twin and make it easier for seafarers and inspectors to use.

The architecture of the ship's digital twin is divided into four parts, as shown in Figure 1. Part 1 is the physical model of the digital twin, which corresponds to the target ship of the prototype application. The ship performs the role of collecting internal and external data and delivering it to the cloud system. For this purpose, it must be equipped with IoT sensors that can collect internal and external data, and maritime communication facilities for land connection. Part 2 is a digital model on the cloud that receives data from the ship and implements a digital twin model. Part 3 is an MR system that can visually check the ship's information and information on the cloud. Finally, part 4 is the application field that can provide optimized solutions through the digital model.

Figure 1 Digital Twin Architecture

Implementing 3D models
To increase the intuitiveness of the digital twin system for users, we utilized a 3D digital model similar to the actual ship. To visualize the 3D model of the ship, it is necessary to secure 3D CAD files for the ship and equipment. For equipment models, we used CAD files created during the design phase, and for hull models, we modeled them directly. To run the files in the application, 3D model lightweighting was performed. The 3D model of the vessel to be applied to the digital twin system and the MR system is shown in Figure 2 below.

Figure 2 Small vessel for Digital Twin (left) and 3D model for MR system (right)

Network System 
The ship's communication system and the onshore support center were configured as a single network using a Virtual Private Network (VPN). The ship's communication is composed of Wifi and LTE in parallel in consideration of the unstable sea conditions. The ship's network was connected to the Azure cloud to enable upload of the ship's real-time data. A TCP/IP-based network server and a synchronization server for cooperative work between MR devices were built on board.   

Prototype Implementation 
We implemented a digital twin application on a prototype ship using the cloud. The following figure shows the application using HoloLens 2. You can see the actual ship on the left and the 3D model on the right, as shown in Figure 3. In addition, the real-time information of the motor can be seen on the dashboard, as shown in Figure 4. As a result, the real-time data is delivered to the cloud using the ship's communication network, and the application allows users to check the real-time information of the ship. Users can operate the application using mixed reality-based hololens2 and mobile devices. They can check the 360-degree view and current status of the ship by moving their seats and interface with the application through screen touch and hand tracking. 

Figure 3. Driving a MR-based application / Figure 4. MR-based ship monitoring system

Conclusion 
We have developed a prototype application for the remote monitoring function of an institution among the ship's systems that can be applied to the digital twins analyzed above. For this study, we prepared an 8-meter battery-based electric propulsion ship and implemented a shipboard digital twin architecture using the Azure cloud. The architecture consists of four parts: ship physical model, cloud system, optimization algorithm, and application, and we developed a prototype digital twin system using the architecture and confirmed that the system can be successfully applied.

We are currently developing a synchronization system between users of the MR system. Once this system is established, it is expected to provide fast and accurate information to seafarers and inspectors by applying newly developed applications.