How to adopt ISO 26262 to develop safe machine learning algorithms?
Machine learning (ML) algorithms, especially those using Deep Neural Networks, have demonstrated their excellent capabilities to resolve many problems and they improved the quality of several non-safety related products in the past years. Despite the fact that currently designed algorithms using machine learning, are approaching the effectiveness achieved by people, their design and development is so complicated that a number of hardly detectable systematic errors may occur, which may result in unintentional operation of the whole system and injury or death of its users. For example, the use of ML algorithm to correctly mark friends in social media photos will not cause drastic consequences. However, a mistake of the algorithm responsible for locating pedestrians in autonomous vehicles can be tragic in consequences and lead to the death of people. For this reason, despite the fact that the currently designed ML algorithms are very effective, but no standards or norms have been developed so far that would allow for the safe implementation of these algorithms. For example, the safety standard IEC 61508 recommends that artificial intelligence algorithms should not be used at all due to their unpredictability in the process of creating safe electronic systems. ISO 26262 standard for functional safety in the automotive industry does not address the use of ML algorithms at all.
In our work, we analyzed safety lifecycle defined in ISO 26262 standard and proposed how currently defined work products can be adopted to cover specific aspects of machine learning algorithms. The proposed approach was presented in our article “Organization of ML-based product development as per ISO 26262” (https://arxiv.org/abs/1910.05112).
The proposed adaptation of V-model applied for ML algorithms is presented below