Digital voice assistants such as Siri and Alexa have been around for years, but such technology can also have applications in shipping. The Fraunhofer Centre for Maritime Logistics and Services (CML) has developed the “Mar FM” speech recognition system, which is by far not only applicable in the maritime industry.
There is one problem, however. “Only ‘single-use information’ can be exchanged via radio, which is no longer available afterwards,” explains Maximilian Reimann, Team Leader Maritime Operation Management and research associate at Hamburg’s Fraunhofer Centre for Maritime Logistics and Services (CML). “Although radios that allow the last radio message received to be repeated do exist, this doesn’t solve the fundamental problem.” The parties involved are often under a great deal of stress and sometimes overwhelmed by the amount of important information they have to absorb.
New development for the maritime industry
With this in mind, the Fraunhofer CML pursued the idea of developing a support tool for maritime radiotelephony that automatically transcribes and documents the content of radio messages. The challenge is, however, that conventional systems such as Siri and Alexa can only recognise everyday speech.
“The maritime industry, however, uses special vocabulary with lots of technical terms and proper names,” he adds. “In combination with the acoustic conditions on board, conventional voice assistance systems quickly reach their limits, making them unsuitable for use in a maritime context.” In addition, the International Maritime Organisation (IMO) uses the Standard Marine Communication Phrases (SMCP) – standardised language that serves as pre-formulated examples. “These haven’t yet become established in practice, however.”
This is why the CML began developing a speech recognition system specifically for the maritime industry. There were also a few hurdles to overcome here. For example, poor signal quality, loud interference and background noise, different accents, dialects and levels of English all form part of everyday radio communication. Then there are the aforementioned technical terms and abbreviations, set phrases and idioms as well as numerous proper names, such as the names of ships.
The development phase focussed primarily on maritime terminology. “This is why the general vocabulary recognition quality isn’t yet as high as with conventional speech recognition systems,” admits Reimann. “It’s important for us, for example, that it transcribes the correct English spelling of the word ‘buoy’, and that it doesn’t write ‘boy’,” he adds.
“It’s important for us that it transcribes the correct English spelling of the word ‘buoy’, and that it doesn’t write ‘boy’.”
Maximilian Reimann, Team Leader Maritime Operation Management and research associate at Hamburg’s Fraunhofer Centre for Maritime Logistics and Services (CML)
Training data is the key
The AI models used were trained using millions of pieces of audio data to ensure a high level of recognition accuracy. These models were then supplemented with maritime data that the CML generated as part of various research projects. “This allowed us to develop a maritime ‘standard language model’, which is then enriched and optimised using data from our customers,” continues Reimann.
Accessing the data was the most difficult part, and training the models was the most time-consuming part, but things are becoming easier. “In the meantime, around ten hours of audio data is enough for us to adapt a model to a new domain.” This enables audio signals to then be converted automatically into text, i.e. transcribed, and this even works offline. Another advantage is that this makes it possible to evaluate communication content retrospectively, for example to optimise downstream processes.
“Mar FM” already has several customers, although many of them have signed confidentiality agreements. One such customer we are allowed to name is the Vessel Traffic Service Centre (VTS) Fintraffic – the maritime traffic control centre for the entire Finnish sea area. As part of a research contract, VHF maritime radio traffic was developed here using automatic speech recognition, which converts the radio traffic into text form in real time and makes it available to the vessel traffic control centre. The technology was optimised to recognise English, Finnish and Swedish.
“A comparable application doesn’t exist anywhere else,” Reimann confirms. The potential, in the industry and beyond, is seen as correspondingly large. “Applications for the offshore wind industry, the oil and gas sector or aviation are also conceivable,” he adds. “Ultimately, our application works wherever aural information is exchanged.” (cb)
Facts
Fraunhofer Centre for Maritime Logistics and Services CML
Established: 2010
Headquarters: Hamburg
Employees: 100
Field of operation: Innovative solution development for the maritime sector and the maritime supply chain
Main research areas: Nautical and maritime transport, ship and fleet management, port and terminal development and port technologies
