Search-Based Software Testing Based on Information Theory

Software Engineering Seminar,  Friday, Oct 27, 1pm, ENGR Room 4201

Robert Feldt
Professor
Department of Computer Science and Engineering
Chalmers University of Technology

Abstract:

Software testing has progressed in recent years and we have a multitude of new methods to automate the creation and execution of tests. The more general trends of agile development and continuous integration/deployment have also lead to increased use of testing techniques in industrial practice. However, key concepts in software testing still lack clear definitions and formalizations, which hinders progress. For example, it is intuitive to any tester that test cases should differ to better cover both intended and actual software behavior. It has been less clear how to define such diversity and how to achieve it in practice. In this talk we show how concepts from information theory can be applied to give universal and general metrics for diversity of test information and how, coupled with search and optimization, these metrics can help create diverse test suites. Empirical evaluation shows that such highly diverse test sets give higher structural and fault coverage even when only applied to test inputs. We also briefly present recent ideas for how to build on these results to formalize and automate other testing concepts such as boundary value and robustness testing.