This paper presents an experimental study on the emulation of software faults by fault injection. In a first experiment, a set of real software faults has been compared with faults injected by a SWIFI tool (Xception) to evaluate the accuracy of the injected faults. Results revealed the limitations of Xception (and other SWIFI tools) in the emulation of different classes of software faults (about 44% of the software faults cannot be emulated). The use of field data about real faults was discussed and software metrics were suggested as an alternative to guide the injection process when field data is not available. In a second experiment, a set of rules for the injection of errors meant to emulate classes of software faults was evaluated. The fault triggers used seem to be the cause for the observed strong impact of the faults in the target system and in the program results. The results also show the influence in the fault emulation of aspects such as code size, complexity of data structures, and recursive versus sequential execution.
Subject
Software Fault Emulation
Conference
Dependable Systems and Networks Conference â?? DSN-2000, June 2000
Cited by
Year 2013 : 2 citations
K. Jayashree, Sheila Anand, "Web Service Diagnoser Model for managing faults in web services", Computer Standards & Interfaces, Elsevier, July 2013.
Patrick Pape, "A Methodology for Increasing the Dependability of Open Source Software Components", PhD Thesis, Auburn University, Auburn, Alabama, May 2013.
Year 2012 : 6 citations
Piotr Gawkowski, Mariusz Markowski, Grzegorz Smulko, ?ukasz Karolewski, "Fault Injection Techniques Towards Software Quality Assessment", Modern ICT for Evaluation of Business Information Systems - Information Systems in Management XVI, Warsaw University of Life Sciences, Warsaw, Poland, 2012.
Xavier Kauffmann-Tourkestansky, “Analyses sécuritaires de code de carte à puce sous attaques physiques simulées”, PhD Thesis, Université D’Orléans, France, November 2012.
Rim Akrout, "Analyse de vulnérabilités et évaluation de systèmes de détection d'intrusions pour les applications Web", PhD Thesis, Institut National des Sciences Appliquées de Toulouse (INSA Toulouse), Toulouse, France, October 2012.
Joon S. Park, Pratheep Chandramohan, Avinash T. Suresh, Joseph V. Giordano, Kevin A. Kwiat, "Component survivability at runtime for mission-critical distributed systems", The Journal of Supercomputing, September 2012
Thorsten Piper, Stefan Winter, Paul Manns, Neeraj Suri, "Instrumenting AUTOSAR for dependability assessment: A guidance framework", IEEE/IFIP International Conference on Dependable Systems and Networks, DSN 2012, Boston, MA, USA, June 2012.
Domenico Cotroneo, Anna Lanzaro, Roberto Natella, Ricardo Barbosa, "Experimental Analysis of Binary-Level Software Fault Injection in Complex Software", Ninth European Dependable Computing Conference, EDCC-9, Sibiu, Romania, May 2012.
Year 2011 : 6 citations
John Thomson, Andre Guerreiro, Paulo Trezentos, Jeff Johnson, "Package Upgrade Robustness: An Analysis for GNU/Linux Package Management Systems", Open Source Systems: Grounding Research, IFIP Advances in Information and Communication Technology, (Eds. Scott Hissam, Barbara Russo, Manoel de Mendonça Neto, Fabio Kon), Springer Boston, ISBN: 978-3-642-24417-9, 2011.
Roberto Natella, “Achieving Representative Faultloads in Software Fault Injection”, PhD Thesis, Universita' Degli Studi di Napoli Federico II, Italy, November 2011.
Y. Oyama, Y. Hoshi, "A Hypervisor for Injecting Scenario-Based Attack Effects", 2011 IEEE 35th Annual Computer Software and Applications Conference (COMPSAC 2011), Munich, Germany, July 2011.
Gabriela Jacques-Silva, Bugra Gedik, Henrique Andrade, Kun-Lung Wu, Ravishankar K. Iyer, “Fault injection-based assessment of partial fault tolerance in stream processing applications”, 5th ACM international conference on Distributed event-based system, DEBS '11, New York, NY, USA, July 11-15, 2011.
Daniele Esposito, “Strategie per il miglioramento dei log applicativi basate su Software Fault Injection”, Tesi di Laurea, Facoltà di Ingegneria, Universita' degle Studi di Napoli Federico II, 2011.
A. Bondavalli, P. Lollini, A. Bovenzi, M. Colajanni, L. Coppolino, C. Esposito, M. Fixxo, C. di Martino, L. Montecchi, R. Natella, A. Peccia, "D3.1: Modeling and Evaluation: State-of-the-art", deliverable of the project: Dependable Off-The-Shelf based Middleware Systems for Large-scale Complex Critical Infrastructures (DOTS-LCCI Project: PRIN 2008LWRBHF), Italy, March 2011.
Year 2010 : 4 citations
1. S.M. Nematollahzadeh, A. A. Jamshidifar, "Simulation of the Effects of Radiation on a Satellite Memory and Improving Its Fault-Tolerant Ability, Using SIHFT", Book Title: Ultra-Wideband, Short Pulse Electromagnetics 9, ISBN: 978-0-387-77845-7, Springer New York, 2010.
2. Ravishankar K. Iyer, Nithin Nakka, Zbigniew Kalbarczyk, Weining Gu, "Fault Injection, Encyclopedia of Software Engineering, Ch 25, Nov 2010.
3. Hameed, K., Williams, R., Smith, J., "Analytically redundant controllers for fault tolerance: Implementation with separation of concerns," American Control Conference, ACC 2010, B altimore, Maryland, USA, June 30 - July 2, 2010.
4. Roberto Natella, Domenico Cotroneo, “Emulation of Transient Software Faults for Dependability Assessment: A Case Study”, Eigth European Dependable Computing Conference, EDCC 2010, Valencia, Spain, April 2010.
Year 2009 : 4 citations
1. Anderson Nunes Paiva Morais, "injecção de Ataques Baseado em Modelo para Teste de Protocolos de Segurança", MSc Thesis, Instituto de Computação, Universidade Estadual de Campinas (UNICAMP), Campinas, Brazil, 2009.
2. Giuseppe Scafuti, "Valutazione sperimentale di tecniche di testing per software in relazione ai tipi di fault", Tesi de Laurea, Universita degli Studi di Napoli Federico II, Naples, Italy, May 2009.
3. Stefano Rago, “Un approccio per la Software Fault Injection in Sistemi Software Complessi e Distribuiti”, MSc Thesis, Universita' degli Studi di Napoli Federico II, Facoltà de Ingegneria, Italy, 2009.
4. Christopher Dabrowski, “Reliability in grid computing systems”, Concurrency and Computation: Practice and Experience, Wiley InterScience, John Wiley & Sons, 2009.
Year 2008 : 3 citations
1. Gabriella Carrozza, “Software Faults Diagnosis in Complex, OTS-Based, Critical Systems”, tese de doutoramento de Gabriella Carrozza, “FEDERICO II” University of Naples, Naples, Italy, November 2008.
2. Jeffrey Michael Napper, “Robust Multithreaded Applications”, PhD Thesis, Graduate School of The University of Texas at Austin, USA, 2008.
3. Douglas John Buettner, "Designing an Optimal Software Intensive System Acquisition: A Game Theoretic Approach", Phd Thesis, University of Southern California, September 2008.
Year 2007 : 7 citations
1. E. Touloupis, J. A. Flint, V. A. Chouliaras, and D. Ward, “Study of the Effects of SEU-Induced Faults on a Pipeline Protected Microprocessor”, IEEE Transactions on Computers, Vol. 56, issue 12, December 2007.
2. Pardo, J. Campelo, J.C. Ruiz, J.C. Gil, P., "Embedded Software Validation Using On-Chip Debugging Mechanisms", Software engineering of fault tolerant systems (P. Pelliccione, H. Muccini, N. Guelfi, A. Romanovsky Eds), Series on Software Engineering and Knowledge Engineering, World Scientific Publishing Co, Vol. 19, pp 121-149, 2007.
3. Paolo Lollini et al., "D4.1.2. Evaluation methodologies, techniques and tools", deliverable of the project: Highly dependable ip-based networks and services Project (HIDENETS Project: IST-FP6-STREP- 26979), December 2007.
4. Andréas Johansson, “Robustness Evaluation of Operating Systems”, tese de doutoramento de Andréas Johansson, Vom Fachbereich Informatik der Technischen Universit, University of Darmstad, Germany 2007.
5. S. Bruning, S. Weißleder, M. Malek, “A fault taxonomy for service-oriented architecture”, Humboldt-Universit¨at zu Berlin, Tech. Rep. 215, 2007.
6. Joon S. Park, Joseph Giordano, "Software component survivability in information warfare", Cyber Warfare and Cyber Terrorism, Information Science Reference, IGI-Gloabl, ISBN: 1591409918. 2007.
7. Joon Park, “Dynamic Hybrid Component Test For Mission-Critical Distributed Systems”, Technical Report AFRL-IF-RS-TR-2007-149, Syracuse University, Air Force Research Laboratory, Information Directorate, Rome Research Site, Rome, New York,USA, June 2007.
Year 2006 : 6 citations
1. V.V. Phoha, S. Phoha, A. Ray, K.S. Balagani, A.U. Nadgar, R. Varanasi, "Embedded Soft Sensing for Anomaly Detection in Mobile Robotic Networks", Sensor Network Operations, IEEE Press/Wiley, ISBN 0-387-19826-1, eds. S. Phoha, T. LaPorta and C. Griffin, Chapter 10, 2006.
2. Juan Pardo, José-Carlos Campelo, Juan-Carlos Ruiz, Pedro Gil, "Temporal Characterization of Embedded Systems Using Nexus", Sixth European Dependable Computing Conference, EDCC-6, Coimbra, Portugal, October 18-20, 2006.
3. Paolo Lollini et al., "D4.1.1. Identification and development of evaluation methodologies, techniques and tools", deliverable of the project: Highly dependable ip-based networks and services Project (HIDENETS Project: IST-FP6-STREP- 26979), December 2006.
4. L. Strigini et al., “Resilience-Building Technologies: State of Knowledge – ReSIST NoE Deliverable D12”, deliverable of the project: ReSIST: Resilience for Survivability in IST - A European Network of Excellence (ReSIST: Contract Number: 026764), September 2006.
5. Richard Torkar, “Towards Automated Software Testing: Techniques, Classifications and Frameworks”, PhD Thesis, Blekinge Institute of Technology, Sweden, 2006.
6. Cheng-Chun Tu, “Modeling Component Failures in Embedded Systems”, Master's Thesis, International Master’s Program in Dependable Computer Systems, Chalmers University of Technology, Department of Computer Science and Engineering, Goteborg, Sweden, 2006.
Year 2005 : 11 citations
1. Ali Kalakech, “Étalonnage de la sûreté de fonctionnement des systèmes d’exploitation – Spécifications et mise en œuvre”, PhD Thesis, LAAS-CNRS, Toulouse, France, August 2005.
2. Arnaud Albinet, “Caractérisation de Systèmes D’exploitation en Présence de Pilotes Défaillants”, PhD Thesis, Laboratoire d’Analyse et d’Architecture des Systèmes (LAAS-CNRS), Toulouse, France, 2005.
3. N. Looker, M. Munro, J. Xu, "Simulating Errors in Web Services", International Journal of Simulation: Systems, Science & Technology, Vol. 5 No. 5 ISSN 1473-804x, 2005.
4. Joon S. Park, Pratheep Chandramohan, Ganesh Devarajan, Joseph Giordano, “Trusted Component Sharing by Runtime Test and Immunization for Survivable Distributed Systems”, Security and Privacy in the Age of Ubiquitous Computing: IFIP TC11 20th International Information Security Conference, Chiba, Japan, May 30-June 1, 2005.
5. Ruben Alexandersson, Peter Öhman, “A technique for fault tolerance assessment of COTS”, International Conference on Computer Safety, Reliability and Security (SAFECOMP2005), Fredrikstad, Norway, September 2005.
6. J.-C. Ruiz, J. Pardo, J.-C. Campelo, P. Gil, “On-chip Debugging-based Fault Emulation for Robustness Evaluation of Embedded Software Components”, IEEE 11th International Symposium Pacific Rim Dependable Computing (PRDC2005), Changsha, Hunan, China, December 2005.
7. Wu Ji, JIA Xiaoxia, Liu Chang, Y Haiyan, Liu Chao, “Finds in Testing Experiments for Model Evaluation”, Tsinghua Science and Technology Journal, Vol.10 No.3 P.298-303, 2005.
8. Jonas Myhrman, “Studying Fault Injection in WCDMA Base Station Processors Using Simics Simulator”, tese de mestrado de Jonas Myhrman, Chalmers University of Technology, Gothenburg, Sweden, 2005.
9. M. F. Lau, Y. T. Yu, “An extended fault class hierarchy for specification-based testing”, ACM Transactions on Software Engineering and Methodology (TOSEM), Volume 14 , Issue 3 , July 2005.
10. J. Pardo, J. C. Ruiz, J.-C. Campelo, P. Gil, "A Non-Intrusive SWIFI Technique for RTOS Robustness Testing" 5th European Dependable Computing Conference, EDCC5, Budapest, Hungary, April 2005.
11. D. Coppit, J. M. Haddox-Schatz, “On the Use of Specification-Based Assertions as Test Oracles”, 29th Annual IEEE/NASA Software Engineering Workshop (SEW’05), Greenbelt, MA, USA, April 2005.
Year 2004 : 10 citations
1. S. Bagchi, Z. Kalbarczyk, R. Iyer, and Y. Levendel. "Design and Evaluation of Preemptive Control Signature (PECOS) Checking", IEEE Transactions on Computers, September 2004.
2. Ana – Elena Rugina, “Etalonnage de la sûreté de fonctionnement de systèmes d’exploitation”, Stage de Recherche, Institut National Polytechnique de Toulouse, Ecole Nationale Supérieure d’Electrotechnique, d’Electronique, d’Informatique, d’Hydraulique Et des Télécommunications, 2004.
3. M. Hiller, A. Jhumka, N. Suri, “EPIC: Profiling the Propagation and Effect of Data Errors in Software”, IEEE Trans. on Computers, 53(5):512–530, May 2004.
4. Marco Serafini, “Una Metodologia per Modellizzare e Valutare la Dependability di Sistemi Safety-Critical”, PhD thesis, Universit a degli Studi di Firenze (University of Florence), Italy, 2004.
5. G. Pintér and I. Majzik, “Impact of Statechart Implementation Techniques on the Effectiveness of Fault Detection Mechanisms”, Proceedings of the 30th IEEE EUROMICRO Conference (EUROMICRO’04), Rennes – France, September 1-3, 2004.
6. H. R. Zarandi, S. G. Miremadi, S. Hessabi, A. Ejlali, “A Mixed-Mode Simulation-based Environment to Test and Dependability Assessment of HDL Models”, in the Proceedings of International Conference on Embedded Systems and Applications (ESA 2004), USA, June 2004.
7. D. Xu, D. Berleant, G. Takle, and Z. Pan, “A better understanding of the effects of software defects in weather simulation”, WRF/MM5 User’s Workshop poster session, Boulder, June 2004.
8. Nik Looker, Malcolm Munro, Jie Xu, “Testing Web Services”, The 16th IFIP International Conference on Testing of Communicating Systems, Oxford, UK, March 17-19, 2004.
9. Éric Marsden, “Caractérisation de la Sûreté de Fonctionnement de Systèmes à base d’Intergiciel”, PhD Thesis, Laboratoire d’Analyse et d’Architecture des Systèmes du CNRS, Toulouse, France, February 2004.
10. Gergely Pinter and Istvan Majzik, “Impact of statechart implementation techniques on the effectiveness of fault detection mechanisms” In Proceedings of the 30th EUROMICRO Conference, IEEE Computer Society Press, 2004.
Year 2003 : 10 citations
1. Pedro Yeste Pérez, “Contribución a la Validación de la Confiabilidad en los Sistemas Empotrados Tolerantes a Fallos”, tese de doutoramento de Pedro Yeste Pérez, Universidad Politécnica de Valencia, Departamento de Informática de Sistemas Y Computadores, Valencia, Spain, 2003.
2. A. Pataricza and G. Pintér “Data Mining in Fault Injection”, In Proc. DDECS 2003. IEEE Workshop on Design and Diagnostics of Electronic Circuits and Systems, Poznan, Poland, April, 2003.
3. H. R. Zarandi, S. G. Miremadi and A. R. Ejlali, "Fault Injection into Verilog Models for Dependability Evaluation of Digital Systems," in the Proceedings of the International Symposium on Parallel and Distributed Computing (ISPDC 2003), IEEE-CS Press, Slovenia, October 2003.
4. Pedro Yuste, Juan-Carlos Ruiz-Garcia, Lenin Lemus, Pedro J. Gil, “Non-intrusive Software-Implemented Fault Injection in Embedded Systems", First Latin-American Symposium on Dependable Computing, LADC 2003, São Paulo, Brazil, October 21-24, 2003.
5. Shu Xiao, Lijun Deng, Sheng Li, Xiangrong Wang, “Integrated TCP/IP Protocol Software Testing for Vulnerability Detection”, 2003 International Conference on Computer Networks and Mobile Computing, ICCNMC’03, Shanghai, China, October 20-23, 2003.
6. Jean Arlat, Yves Crouzet, Johan Karlsson, Peter Folkesson, Emmerich Fuchs, Günther H. Leber, “Comparison of Physical and Software-Implemented Fault Injection Techniques,” IEEE Transactions on Computers Archive, Volume 52, Issue 9, September de 2003.
7. Tahar Jarboui, “Surete de Fonctionnement de Systemes Informatiques – Étalonage et Representativite des Fautes”, PhD Thesis, LAAS-CNRS, Toulouse, France, May 2003.
8. Shu Xiao, Sheng Li, Xiangrong Wang, Lijun Deng, “Fault-oriented Software Robustness Assessment for Multicast Protocols”, Second IEEE International Symposium on Network Computing and Applications, Cambridge, Massachusetts, USA, April 16-18, 2003.
9. András Pataricza, Gergely Pinter, “Data Mining in Fault Injection”, 6th IEEE International Workshop on Design and Diagnostics of Electronic Circuits and Systems, DDECS-2003, Poznan, Poland, April 14-16, 2003
10. Ravi Iyer, "Design of Reliable Systems and Networks", ECE 442/CS 436, Course Papers, Spring 2003, Center for Reliable and High-Performance Computing, Coordinated Science Laboratory, University of Illinois at Urbana-Champaign, Urbana, IL, USA, 2003. (http://courses.ece.uiuc.edu/ece442/papers.htm).
Year 2002 : 13 citations
1. Tahar Jarboui, Jean Arlat, Yves Crouzet, Karama Kanoun, T. Marteau, "Analysis of the Effect of Real and Injected Software Faults: Linux as a Case Study", Pacific Rim International Symposium on Dependable Computing, PRDC2002, Tsukuba, Japan, December 16-18, 2002.
2. Eric Marsden, Jean-Charles Fabre, Jean Arlat, “Dependability of CORBA Systems: Service Characterization by Fault Injection”, 21st IEEE Symposium on Reliable Distributed Systems, SRDS02, Osaka University, Suita, Japan, October 13-16, 2002.
3. Manuel Rodriguez, “Technologie d’empaquetage pour la surete de fonctionnement des systemes temps-reel”, PhD Thesis, LAAS-CNRS, Toulouse, France, July 2002.
4. B. Floering, B. Brothers, Zbigniew Kalbarczyk, Ravishankar K. Iyer, “An Adaptive Architecture for Monitoring and Failure Analysis of High-Speed Networks”, 2002 IEEE/IFIP International Conference on Dependable Systems and Networks, DSN 2002, Bethesda, Maryland, USA, June 23-26, 2002.
5. Manuel Rodriguez, Arnaud Albinet, Jean Arlat, "MAFALDA-RT, a tool for dependability assessment of real-time systems", IEEE/IFIP International Conference on Dependable Systems and Networks, DSN 2002, Bethesda, Maryland, USA, June 23-26, 2002.
6. Tahar Jarboui, Jean Arlat, Yves Crouzet, Karama Kanoun, "Experimental analysis of the errors induced into linux by three fault injection techniques", EEE/IFIP International Conference on Dependable Systems and Networks, DSN 2002, Bethesda, Maryland, USA, June 23-26, 2002.
7. Eliane Martins, Cecilia Rubira, Nelson Leme, "Jaca: a reflective fault injection tool based on patterns", EEE/IFIP International Conference on Dependable Systems and Networks, DSN 2002, Bethesda, Maryland, USA, June 23-26, 2002.
8. Jean Arlat, Yves Crouzet, "Faultload representativeness for dependability benchmarking", IEEE/IFIP International Conference on Dependable Systems and Networks, DSN 2002, Bethesda, Maryland, USA, June 23-26, 2002.
9. Eric Marsden, Jean-Charles Fabre, Jean Arlat, "Characterization approaches for CORBA systems by fault injection", IEEE/IFIP International Conference on Dependable Systems and Networks, DSN 2002, Bethesda, Maryland, USA, June 23-26, 2002.
10. Jean Arlat, “From Experimental Assessment of Fault-Tolerant Systems to Dependability Benchmarking”, 16th International Parallel and Distributed Processing Symposium, IPDPS 2002, Fort Lauderdale, Florida, USA, April 15-19, 2002.
11. Jean Arlat, Jean-Charles Fabre, Manuel Rodriguez, Frédéric Salles, "Dependability of COTS microkernel-based systems", IEEE Transactions on Computers, 51 (2): pp. 138-163, February 2002.
12. Jean Arlat, “From Experimental Assessment of Fault-Tolerant Systems to Dependability Benchmarking”, Proceedings of the International Parallel and Distributed Processing Symposium (IPDPS’02), 2002.
13. Jean Arlat, "From Fault Injection Experiments to Dependability Benchmarking", Workshop "Challenges and Directions for Dependable Computing, 41st Meeting of IFIP WG 10.4, Saint John, US, January 4-8, 2002.
Year 2001 : 5 citations
1. Eric Marsden, Jean-Charles Fabre, “Failure Mode Analysis of CORBA Service Implementations”, Middleware 2001: IFIP/ACM International Conference on Distributed Systems Platforms Heidelberg, Heidelberg, Germany, November 12-16, 2001.
2. Eric Marsden, Jean-Charles Fabre, “Failure analysis of an ORB in presence of faults”, DSoS. IST-1999-11585. Dependable Systems of Systems, DSoS Project (IST-1999-11585), Deliverable IC3, October 2001.
3. Maitrayi Sabaratnam, “Improving Dependability in Database Management Systems”, PhD Thesis, University of Trondheim, Trondheim, Norway, November 2001.
4. Laurent Réveillère, Gilles Muller, “Improving Driver Robustness: an Evaluation of the Devil Approach”, IEEE/IFIP International Dependable Systems and Networks Conference, DSN 2001, Gothenburg, Sweden, July 1-4, 2001.
5. Elisa Nakagawa, José C. Maldonado, "Software-Fault Injection Based on Mutant Operators", IX Brazilian Symposium on Fault-Tolerant Computing, SCTF 2001, Florianópolis, Santa Catarina, Brazil, March 5-7, 2001.
