AI Convergence Research Team: 

Lee Gapheon (Deputy Senior Surveyor), Song Sangho(Deputy Senior Surveyor), 

Jo Yeonhwa (Senior Surveyor), Jang Hwasup (General Manager)

1. Introduction

Advances in artificial intelligence (AI) technology are bringing innovation to various industries. KR is developing technology to improve work efficiency and service quality by using artificial intelligence for various tasks such as extracting information from drawings, detecting hull damage, and determining ship equipment failures. Recently, there have been innovative technological developments in the field of natural language processing, and KR is conducting research to apply these technologies to KR rules and tasks.

2. Development of Natural Language Processing

Innovative developments in Natural Language Processing (NLP) begin with the paper “Attention Is All You Need” published by Google in June 2017. In the paper, The Attention mechanism, which pays attention only to the part of the data that needs to be focused on, and the Transformer deep learning algorithm to which this mechanism is applied were announced. And in 2018, Google's BERT model, which used only the Encoder part of a Transformer, and OpenAI's GPT, which also only used the Decoder part of Transformer, were announced.

In the beginning, BERT performed better than GPT, and from this point on, Google Translator's performance improved dramatically. Later, in May 2020, GPT 3, which has excellent performance with 175 billion parameters, became an issue, and KR began research to utilize NLP.

3. Initial Research on NLP

  In the early stages of NLP research, KR attempted to develop fragmentary technology that standardizes and converts user-entered AIS destination values to UN/LOCODE. Furthermore, KR evaluated whether the latest NLP technologies, such as BERT and GPT-3, are applicable to KR operations at a usable level.

Figure 1. Review of NLP Model Selection Suitable for KR Among BERT and GPT3

Figure 2. BERT-based NLP Q&A Prototype

Figure 1 and Figure 2 show that in early 2021, KR identified the characteristics of BERT and GPT3, selected which NLP model was suitable for KR, and developed and tested a Q&A prototype. At this time, BERT supported multiple languages, was free, and could be configured in a secure, closed structure through model downloads. In contrast, GPT-3 was specialized in English, required payment, and was only available through an API, making it relatively more exposed to security risks. Therefore, it was determined that the BERT model was more suitable for KR. The BERT Multilingual model was downloaded, and further trained with an open Korean Q&A database named korQuAD. Based on this model, KR created a Q&A prototype and tested the level of questions that could be answered.

Looking at the example on the left in the Q&A prototype in Figure 2, when given a document about Jupiter, “How many times the volume of Jupiter is the Earth?” If you ask the question, you can get the answer “1300 times.” The example on the right is a document that answers questions about ballast tanks in KR. When asked, “Does transfer to a ballast tank need to be recorded in the ship’s ballast water management record?” the answer was “3.6-additional operational procedure and general remarks [UNK].” In addition to this, various tests were conducted, and the three results obtained through this study are as follows.

1.     When a user question comes in, an artificial intelligence chatbot can be developed by finding the document most similar to it and then finding the answer there.

2.     If you have not learned enough Korean or KR domain words, an unrecognizable token called [UNK] may appear as an answer.

3.     Since the BERT model is specialized in understanding natural language, additional development is required if you want answers generated in the form of sentences (development of sentence generation logic, use of GPT, a specialized natural language generation model, etc.)

4.   Development of a KR chatbot Based on AI 

At the end of 2021, inspired by the financial sector resolving simple inquiries through chatbots on Messenger or the web, we developed an artificial intelligence-based Q&A chatbot limited to some inspection tasks.

Figure 3. Inspection Q&A DB (left), Chatbot Operation Procedure (bottom-right)

 KR created a database of frequently asked questions and answers related to inspection work and created a chatbot operation procedure using the word embedding method that vectorizes words and  a technology based on NLP. Looking at it in detail, a vector DB is constructed by vectorizing the prepared questions and answers DB for each content. Then, when a user asks the question, “What is the remote inspection procedure?”, the artificial intelligence-based KR chatbot service operates by vectorizing the question and showing the user the most similar answer from Vector DB. Looking at it in detail, a vector DB is constructed by vectorizing the prepared questions and answers DB for each content. Then, when a user asks the question, “What is the remote inspection procedure?”, the AI-based KR chatbot service operates by vectorizing the question and showing the user the most similar answer from Vector DB.

Figure 4. KR Chatbot b Based on AI

The front end of the artificial intelligence-based KR chatbot service was developed using ReactJS, the back end was Python, and the NLP module was developed using Azure Cognitive Service. This service has processed 815 Q&As on various tasks such as remote inspections, EDD, ISM, etc. (as of January 2024).

5. The emergence and application of ChatGPT

On November 30, 2022, after the beta version of the AI-based KR chatbot service was distributed, ChatGPT, which fine-tuned GPT3 to be more suitable for conversation, was released. ChatGPT caused a huge stir as soon as it was released and had a great influence on the direction of KR’s research using NLP.

Figure 5. KR Chatbot (left), ChatGPT (middle), KR rules and ChatGPT (right)

After the launch of ChatGPT, KR studied methods that could be used for intelligent search of the classification of steel ship rules. As a result, as shown in Figure 5, KR applied prompt engineering technology to refer to the KR rules when asking questions and providing answers through ChatGPT. The currently developed version is a prototype version that is being examined for applicability, and research and development is being conducted to determine to what extent it can be used. In the future, KR plans to pursue continuous research and development to provide high-quality technical services to customers by utilizing various data and advanced NLP technology, such as international agreements and ship inspection history management in addition to steel ship rules.