Interpretable Anomaly Detection in Cloud Infrastructure Through Explainable Machine Learning

Abstract

Cloud infrastructure is becoming increasingly complex, with distributed and dynamic workloads, microservices-based applications, and multi-tenant environments generating massive amounts of operational telemetry data. Modern machine learning-based anomaly detection approaches are proving effective in such environments, often detecting performance degradations, security breaches, or resource misuse with high precision; however, most such models are black-boxes, providing little or no insight into the rationale behind their conclusions. This paper proposes an interpretable anomaly detection framework which uses explainable machine learning methods, such as the SHAP values and a permutation-based local explainer, to provide human-readable explanations for each anomalous event detected. The framework uses traditional Isolation Forest and One-Class SVM algorithms to identify outliers in the five-dimensional space of CPU, memory, inbound and outbound network throughput, and latency utilization of cloud infrastructure. The model was tested on a controlled cloud-telemetry dataset and was able to detect anomalies with a recall of 1.00, F1-score of 0.77 and ROC-AUC of 1.00 for the Isolation Forest algorithm, and a 0.65 recall and 0.022 false-positive rate for the One-Class SVM. The explainability module added little computational overhead to the model while successfully isolating the features contributing to each anomaly detected. The framework provides potential value in production environments as a tool to understand, qualify, and act upon each anomalous event detected in cloud infrastructure.