In the future, systems based on artificial intelligence (AI) and machine learning (ML) will also play an increasingly important role in rail technology. Examples of such technologies include assistance and safety systems that can detect obstacles on the track or improve the positioning of trains. The sensors, components, and vehicle systems used for these applications employ algorithms that learn their functionality based on selected training data and thus exhibit capabilities that cannot currently be achieved by employing classical software design. Entirely new challenges arise for the quality assurance of such systems: On the one hand, the quality and safety requirements in rail traffic must be observed, and on the other hand, the criticality, complexity, and dynamics in the realization of AI-based systems must be dealt with. In particular, the differences between classical software design and the data-intensive optimization processes in machine learning mean that assurance procedures established in classical software development cannot be transferred one-to-one.

The KI-LOK project, therefore, aims to develop new test procedures and methods for safeguarding and certifying AI-supported technologies for safety-critical applications in railroad technology.

In the project, the scientists at Fraunhofer FOKUS are developing a procedure for the risk-based safeguarding of critical system states of ML-based applications through systematic testing. To this end, critical applications are first identified, classified, and evaluated at the system level. Subsequently, known failure classes or vulnerabilities in machine learning are linked to the identified critical system scenarios. Test data is then generated on the basis of specific test patterns.

This approach is based on methods and techniques to be developed in the KI-LOK project. These include, for example, test procedures and test patterns for the systematic generation of tests, techniques for the diversification and systematic mutation of existing scenarios and data sets, or coverage metrics and criteria for the risk-oriented generation of tests.

In addition to Fraunhofer FOKUS, ITPower Solutions GmbH, neurocat GmbH, Thales Deutschland GmbH and Heinrich Heine University Düsseldorf are involved in the project.

The KI-LOK project is funded by the German Federal Ministry for Economic Affairs and Energy (BMWi) in the “New Vehicle and System Technologies” funding program.