Fraunhofer Center for Maritime Logistics and Services
Fraunhofer Center for Maritime Logistics and Services

Testing NEMO in the ship handling simulator

© Fraunhofer CML

NEMO User Interface

© Fraunhofer CML

NEMO – Onboard Dialog-Based Command Input

Project Description

Project Description and Background

NEMO is the dialog-based human-ship interface developed by Fraunhofer CML (Maritime Informatics Team) for navigation and situational awareness on the bridge. It replaces keyboard input with fast, intuitive voice commands, operates largely offline, and can optionally use voice biometrics for authentication. To reduce the flood of information from numerous displays and warning messages, NEMO prioritizes relevant data and outputs it verbally. Safety is ensured through closed-loop communication (readback), multi-level transcription, and classic NLP methods with a low tendency to produce false positives. Before executing a command, NEMO checks available capacity and the nautical situation (e.g., surrounding vessels) and provides a situational overview including critical distances and the time until the next approach. The interface integrates ECDIS, among other systems, and displays course and speed; training is conducted using SMCP (Standard Marine Communication Phrases) and typical bridge communications. NEMO is being validated in the Fraunhofer CML ship simulator—the goal is to reliably cover a broad range of commands under the supervision of the ship’s command.

How NEMO works:

  1. Audio recording via bridge microphone
  2. Multi-stage real-time transcription
  3. NLP: Intent/entity recognition
  4. Safety logic: Checking capacities/scope for action (nautical situation, surrounding vessels)
  5. Closed-loop confirmation (readback)
  6. Execution/visualization (e.g., ECDIS display) and verbal

Facts at a Glance

Aspect Content
Benefits     Fast, intuitive commands and queries; reduces workload in complex bridge situations
Safety Closed-loop communication (confirmation/readback), capacity and situational checks before execution
Technology     Multi-stage transcription (real-time ASR) + classic NLP methods with low hallucination rates
Training Standard Marine Communication Phrases (SMCP) and typical bridge communications
Integration ECDIS display, NEMO UI with course/speed, microphone input
Operation Runs locally, largely independent of connectivity; voice biometrics option
Development Fraunhofer CML, Maritime Informatics Team; validation in the CML ship simulator environment
Status/ Goal Goal: Understanding “all conceivable” instructions/questions and execution under the supervision of the ship’s command

Project Status

Under development and undergoing simulator validation; goal: broad range of commands, safe execution under supervision