Show simple item record

dc.contributor.authorLin, Chu-Ti
dc.contributor.authorTang, Kai-Wei
dc.contributor.authorWang, Jiun-Shiang
dc.contributor.authorKapfhammer, Gregory M.
dc.date.accessioned2017-11-09T13:34:11Z
dc.date.available2017-11-09T13:34:11Z
dc.date.issued2017-05-24
dc.identifier.citationLin, C.T., Tang, K.W., Wang, J.S., and Kapfhammer, G.M. (2017). Empirically evaluating Greedy-based test suite reduction methods at different levels of test suite complexity. Science of Computer Programming 150(1): 1-25. doi: 10.1016/j.scico.2017.05.004en_US
dc.identifier.issn0167-6423
dc.identifier.urihttp://hdl.handle.net/10456/45331
dc.description.abstractTest suite reduction is an important approach that decreases the cost of regression testing. A test suite reduction technique operates based on the relationship between the test cases in the regression test suite and the test requirements in the program under test. Thus, its effectiveness should be closely related to the complexity of a regression test suite - the product of the number of test cases and the number of test requirements. Our previous work has shown that cost-aware techniques (i.e., the test suite reduction techniques that aim to decrease the regression test suite's execution cost) generally outperform the others in terms of decreasing the cost of running the regression test suite. However, the previous empirical studies that evaluated cost-aware techniques did not take into account test suite complexity. That is, prior experiments do not reveal if the cost-aware techniques scale and work effectively on test suites with more test cases and more test requirements. This means that prior experiments do not appropriately shed light on how well test suite reduction methods work with large programs or test suites. Therefore, this paper focuses on the Greedy-based techniques and empirically evaluates the additional Greedy and two cost-aware Greedy techniques - at different levels of test suite complexity - from various standpoints including the cost taken to run the regression test suite, the time taken to reduce the test suites, the total regression testing costs, the fault detection capability, the fault detection efficiency, and the common rates of the representative sets. To the best of our knowledge, none of the previous empirical studies classify a considerable number of test suites according to their complexity. Nor do any prior experiments evaluate the test suite reduction techniques, in terms of the aforementioned criteria, at different levels of test suite complexity. This paper represents the first such attempt to carry out this important task. Based on the empirical results, we confirm the strengths and weaknesses of the cost-aware techniques and develop insights into how the cost-aware techniques' effectiveness varies as the test suite complexity increases. (C) 2017 Elsevier B.V. All rights reserved.en_US
dc.description.sponsorshipMinistry of Science and Technology, Taiwan (MOST 103-2221-E-415-010 & MOST 104-2628-E-415-001-MY3)en_US
dc.language.isoen_USen_US
dc.publisherElsevieren_US
dc.relation.ispartofScience of Computer Programmingen_US
dc.relation.isversionofhttps://doi.org/10.1016/j.scico.2017.05.004en_US
dc.rightsPublished articles are subject to copyright, contact the publisher or reprint author for further use of this material. Post-prints will be made available for public use after a 48 month embargo.en_US
dc.subjectSoftware testingen_US
dc.subjectRegression testingen_US
dc.subjectTest suite reductionen_US
dc.subjectTest suite complexityen_US
dc.titleEmpirically evaluating Greedy-based test suite reduction methods at different levels of test suite complexity.en_US
dc.description.versionPublished articleen_US
dc.contributor.departmentComputer Scienceen_US
dc.description.embargoThe post-print (if available) will be made available to the public after 05/24/2019.en_US
dc.citation.volume150en_US
dc.citation.issue1en_US
dc.citation.spage1en_US
dc.citation.epage25en_US
dc.identifier.doi10.1016/j.scico.2017.05.004
dc.contributor.avlauthorKapfhammer, Gregory M.


Files in this item

Thumbnail
Thumbnail

This item appears in the following Collection(s)

Show simple item record