Random testing is a software testing technique through which programs are tested by generating and executing random inputs. Because of its unstructured nature, it is difficult to determine when to stop a random testing process. Faults may be missed if the process is stopped prematurely, and resources may be wasted if the process is run too long. In this article, we propose two promising termination criteria, “All Equivalent” (AEQ) and “All Included in One” (AIO), applicable to random testing. These criteria stop random testing once the process has reached a code-coverage-based saturation point after which additional testing effort is unlikely to provide additional effectiveness. We model and implement them in the context of a general random testing process composed of independent random testing sessions. Thirty-six experiments involving GUI testing and unit testing of Java applications have demonstrated that the AEQ criteria is generally able to stop the process when a code coverage equal or very near to the saturation level is reached, while AIO is able to stop the process earlier in cases it reaches the saturation level of coverage. In addition, the performance of the two criteria has been compared against other termination criteria adopted in the literature.
Developing and Evaluating Objective Termination Criteria for Random Testing / Tramontana, Porfirio; Amalfitano, Domenico; Amatucci, Nicola; Memon, Atif; Fasolino, Anna Rita. - In: ACM TRANSACTIONS ON SOFTWARE ENGINEERING AND METHODOLOGY. - ISSN 1049-331X. - 28:3(2019), pp. 1-52. [10.1145/3339836]
Developing and Evaluating Objective Termination Criteria for Random Testing
Tramontana, Porfirio
;Amalfitano, Domenico;Amatucci, Nicola;Fasolino, Anna Rita
2019
Abstract
Random testing is a software testing technique through which programs are tested by generating and executing random inputs. Because of its unstructured nature, it is difficult to determine when to stop a random testing process. Faults may be missed if the process is stopped prematurely, and resources may be wasted if the process is run too long. In this article, we propose two promising termination criteria, “All Equivalent” (AEQ) and “All Included in One” (AIO), applicable to random testing. These criteria stop random testing once the process has reached a code-coverage-based saturation point after which additional testing effort is unlikely to provide additional effectiveness. We model and implement them in the context of a general random testing process composed of independent random testing sessions. Thirty-six experiments involving GUI testing and unit testing of Java applications have demonstrated that the AEQ criteria is generally able to stop the process when a code coverage equal or very near to the saturation level is reached, while AIO is able to stop the process earlier in cases it reaches the saturation level of coverage. In addition, the performance of the two criteria has been compared against other termination criteria adopted in the literature.File | Dimensione | Formato | |
---|---|---|---|
ACM Tosem.pdf
solo utenti autorizzati
Descrizione: Articolo principale
Tipologia:
Documento in Post-print
Licenza:
Accesso privato/ristretto
Dimensione
4.35 MB
Formato
Adobe PDF
|
4.35 MB | Adobe PDF | Visualizza/Apri Richiedi una copia |
I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.