Don’t Go Down the Rabbit Hole: Reprioritizing Enumeration for Property-Based Testing
Combinatorial testing is a popular software engineering technique for effectively testing programs that operate on parameters drawn from small, finite domains (such as configuration options), by providing a principled way of systematically exploring interactions between them. Property-based testing, on the other hand, is a widely used technique for testing the correctness of functional programs that usually operate on large, infinite domains (such as algebraic data types), by randomly generating or enumerating inputs.
In this paper, we show how to extend and apply ideas from the narrow scope of combinatorial testing techniques to the broader scope of property-based testing applications. In particular, we develop a novel way of pruning the input search space while still ensuring that a diverse set of constructor patterns appear among the set of generated tests. In our implementation, we integrate a state-of-the-art enumeration-based property-based testing framework, LazySearch, with a state-of-the-art combinatorial testing tool, NIST’s ACTS, and demonstrate how it can significantly speed up the effectiveness of testing—up to more than 20$\times$ in the case of a prior System F case study from the literature.
Sat 9 SepDisplayed time zone: Pacific Time (US & Canada) change
11:00 - 12:30
|falsify: Internal Shrinking Reimagined for Haskell
Edsko de Vries Well-Typed LLPDOI
|Don’t Go Down the Rabbit Hole: Reprioritizing Enumeration for Property-Based Testing
Segev Elazar Mittelman University of Maryland, College Park, Aviel Resnick University of Pennsylvania, Ivan Perez NASA Ames Research Center, Alwyn Goodloe NASA Langley Research Center, Leonidas Lampropoulos University of Maryland, College ParkDOI
|Lightning Talks II