The assessment of the dependability properties of a system (dependability benchmarking) is a critical step when choosing among similar components/products. This paper presents a proposal for the benchmarking of the dependability properties of web-servers. Our benchmark is composed of the three key components: measures, workload, and faultload. We use the SPECWeb99 benchmark as starting point, adopting the workload and performance measures from this performance benchmark, and we added the faultload and new measures related to dependability. We illustrate the use of the proposed benchmark through a case-study involving two widely used web servers (Apache and Abyss) running on top of three different operating systems. The faultloads used encompass software faults, hardware faults and network faults. We show that by using the proposed dependability benchmark it is possible to observe clear differences regarding dependability properties of the web-servers.
Subject
Dependability benchmarking
Related Project
DBench - Dependability Benchmarking
Conference
The 23rd International Conference on Computer Safety, Reliability and Security, SAFECOMP 2004, September 2004
PDF File
Cited by
Year 2012 : 4 citations
1. Amit Sangroya, Damián Serrano, Sara Bouchenak, "Benchmarking Dependability of MapReduce Systems", 31st IEEE International Symposium on Reliable Distributed Systems, Irvine, CA, USA, October 2012.
2. Hajime Fujita, Yutaka Matsuno, Toshihiro Hanawa, Mitsuhisa Sato, Shinpei Kato, Yutaka Ishikawa, "DS-Bench Toolset: Tools for dependability benchmarking with simulation and assurance", 2012 42nd Annual IEEE/IFIP International Conference on Dependable Systems and Networks (DSN 2012), Boston, MA, USA, June 2012.
3. Anatoliy Gorbenko, Vyacheslav Kharchenko, Seyran Mamutov, Olga Tarasyuk, Alexander Romanovsky, “Exploring Uncertainty of Delays as a Factor in End-to-End Cloud Response Time”, 9th European Dependable Computing Conference (EDCC 2012), Sibiu, Romenia, May 2012.
4. Yuhui Chen, Anatoliy Gorbenko, Vyacheslav Kharchenko, and Alexander Romanovsky, “Measuring and Dealing with the Uncertainty of SOA Solutions,” Performance and Dependability in Service Computing: Concepts, Techniques and Research Directions, ed. Valeria Cardellini, Emiliano Casalichio, Kalinka Castelo Branco, Júlio Cezar Estrella, and Francisco José Monaco, IGI Global, ISBN: 1609607945, 2012.
Year 2011 : 2 citations
1. Paulo Véras, “Benchmarking Software Requirements Documentation for Space Application”, PhD Thesis, Instituto Tecnológico de Aeronáutica, São José dos Campos, SP, Brasil, 2011.
Roberto Natella, “Achieving Representative Faultloads in Software Fault Injection”, PhD Thesis, Universita' Degli Studi di Napoli Federico II, Italy, November 2011.
Year 2009 : 2 citations
1. Gustavo M. D. Vieira, Willy Zwaenepoel, Luis E. Buzato, “Dynamic Content Web Applications: Crash, Failover, and Recovery Analysis”, The 39th Annual IEEE/IFIP International Conference on Dependable Systems and Networks, DSN 2009, Lisbon, Portugal, June 29-July 2, 2009.
2. Y. Chen, A. Gorbenko, V. Kharchenko, S. Mamutov, A. Romanovsky, O. Tarasyuk, “Benchmarking Dependability of a System Biology Application”, 14th IEEE International Conference on Engineering of Complex Computer Systems, Potsdam, Germany, June 2009.
Year 2008 : 3 citations
1. Bailey, D., Frank-Schultz, E., Lindeque, P., Temple, J. L., "Three reliability engineering techniques and their application to evaluating the availability of IT systems : An introduction", IBM Systems Journal, ISSN 0018-8670, Vol. 47, no4, 2008.
2. A. Gorbenko, V. Kharchenko, O. Tarasyuk, Y. Chen, A. Romanovsky, “The Threat of Uncertainty in Service-Oriented Architecture”, 2008 RISE/EFTS Joint International Workshop on Software Engineering for Resilient Systems, Newcastle upon Tyne, United Kingdom, November 2008.
3. A. Gorbenko, A. Romanovsky, V. Kharchenko, A. Mikhaylichenko, “Experimenting with exception propagation mechanisms in service-oriented architecture”, 4th international Workshop on Exception Handling, WEH’08, Atlanta, Georgia, November 14, 2008.
Year 2007 : 1 citations
1. Gergely Pintér, “Model Based Program Synthesis and Runtime Error Detection for Dependable Embedded Systems”, PhD Thesis, Department of Measurement and Information Systems, Budapest University of Technology and Economics, Budapest, Hungary, 2007.
Year 2006 : 2 citations
1. L. Strigini et al., “Resilience-Building Technologies: State of Knowledge – ReSIST NoE Deliverable D12”, deliverable of the project: ReSIST: Resilience for Survivability in IST - A European Network of Excellence (ReSIST: Contract Number: 026764), September 2006.
2. S Tixeuil, LM Silva, W Hoarau, “An Overview of Existing Tools for Fault-Injection and Dependability Benchmarking in Grids”, Second CoreGRID Workshop on Grid and Peer to Peer Systems, 2006.
Year 2005 : 2 citations
1. Rogério de Lemos, “Architecting Web Services Applications for Improving Availability”, Lecture Notes in Computer Science, Architecting Dependable Systems III, Springer, ISSN 0302-9743 (Print) 1611-3349 (Online), Volume 3549/2005, 2005.
2. Ali Kalakech, “Étalonnage de la sûreté de fonctionnement des systèmes d’exploitation – Spécifications et mise en œuvre”, PhD Thesis, LAAS-CNRS, Toulouse, France, August 2005.
