Generic Faultloads Based on Software Faults for Dependability Benchmarking

Authors

Abstract

The most critical component of a dependability benchmark is the faultload, as it should represent a repeatable, portable, representative, and generally accepted set of faults. These properties are essential to achieve the desired standardization level required by a dependability benchmark but, unfortunately, are very hard to achieve. This is particularly true for software faults, which surely accounts for the fact that this important class of faults has never been used in known dependability benchmark proposals. This paper proposes a new methodology for the definition of faultloads based on software faults for dependability benchmarking. Our methodology builds on previous published results based on field data and uses a fault injection technique based on executable code mutation to emulate programming errors. Faultload properties such as repeatability, portability and scalability are also analyzed and validated through experimentation using a case study of dependability benchmarking of web servers. We concluded that software fault based faultloads generated using our methodology are appropriate and useful for dependability benchmarking. As our methodology is not tied to any specific software vendor or platform, it can be used to generate faultloads for the evaluation of any software product such as OLTP systems. Another important characteristic of the proposed methodology is the independence of source code availability. This allows us to generate faultloads appropriate for dependability evaluation of generic COTS software modules or systems.

Subject

Software Fault Emulation and Injection

Related Project

DBench - Dependability Benchmarking

Conference

IEEE/IFIP International Conference on Dependable Systems and Networks - Dependable Computing and Communications, DSN-DCCS 2004, September 2004

PDF File