On Benchmarking Intrusion Detection Systems in Virtualized Environments

Authors

Abstract

SPEC Research Group - IDS Benchmarking Working Group, Standard Performance Evaluation Corporation (SPEC)
http://research.spec.org/working-groups/ids-benchmarking-working-group.html

Modern intrusion detection systems (IDSes) for virtualized environments are deployed in the virtualization layer with components inside the virtual machine monitor (VMM) and the trusted host virtual machine (VM). Such IDSes can monitor at the same time the network and host activities of all guest VMs running on top of a VMM being isolated from malicious users of these VMs. We refer to IDSes for virtualized environments as VMM-based IDSes. In this work, we analyze state-of-the-art intrusion detection techniques applied in virtualized environments and architectures of VMM-based IDSes. Further, we identify challenges that apply specifically to benchmarking VMM-based IDSes focusing on workloads and metrics. For example, we discuss the challenge of defining representative baseline benign workload profiles as well as the challenge of defining malicious workloads containing attacks targeted at the VMM. We also discuss the impact of on-demand resource provisioning features of virtualized environments (e.g., CPU and memory hot plugging, memory ballooning) on IDS benchmarking measures such as capacity and attack detection accuracy. Finally, we outline future research directions in the area of benchmarking VMM-based IDSes and of intrusion detection in virtualized environments in general.


Find the report here.

Keywords

Intrusion Detection Systems, Benchmarking, Virtualization and Security, Evaluation, Metrics

Subject

IDS Benchmarking

TechReport Number

SPEC-RG-2013-002 v.1.0