Online Verification through Model Checking of Medical Critical Intelligent Systems

Authors

Abstract

Software systems based on Artificial Intelligence (AI) and Machine Learning (ML) are being widely adopted in various scenarios, from online shopping to medical applications. When developing these systems, one needs to take into account that they should be verifiable to make sure that they are in accordance with their requirements. In this work we propose a framework to perform online verification of ML models, through the use of model checking. In order to validate the proposal, we apply it to the medical domain to help qualify medical risk. The results reveal that we can efficiently use the framework to determine if a patient is close to the multidimensional decision boundary of a risk score model. This is particularly relevant since patients in these circumstances are the ones more likely to be misclassified. As such, our framework can be used to help medical teams make better informed decisions.

Related Project

AI4EU - A European AI On Demand Platform and Ecosystem

Conference

Dependable and Secure Machine Learning (DSML 2020) co-located with the 50th IEEE/IFIP International Conference on Dependable Systems and Networks (DSN 2020) , June 2020

PDF File

DOI