Simulation-based smart testing of autonomous systems for digital assurance
The development and use of autonomous boats and ships is steadily progressing; autonomous boats are already commercially available, and there are many ongoing research projects on autonomous ferries and ships. While most of these systems do not yet operate fully autonomously, even decision support systems are changing the mode of operations compared to traditional shipping practices. To ensure that these (semi-)autonomous systems can safely be introduced to the real world, we need to assure the systems. Assurance refers to obtaining justified confidence that a product (or process) conforms to existing safety, environmental, societal, or regulatory requirements. For example, the IMO is actively developing the regulatory framework for Maritime Autonomous Surface Ships (MASS), for autonomous ships we want to ensure they adhere to the COLREG, and in addition, the European Commission recently introduced the EU AI act. To do assurance of autonomous and AI-enabled systems and applications, i.e. gather evidence for justified confidence in the systems, DNV is developing a digital assurance framework. Methods for gathering such evidence include simulation-based testing, evidence-based argumentation, risk-based approaches, real-world testing, etc. In this lecture, I will introduce concepts for simulation-based testing of autonomous systems, including co-simulation frameworks and smart testing methods, such as using Bayesian Optimization for design of experiments. In addition, I will show use cases from ongoing research projects for testing autonomous navigation systems of ships and ferries.