Investigating the robustness of locators in template-based Web application testing using a GUI change classification model

GUI-based test-cases generated by Capture and Replay tools suffer from the well-known fragility problem: they may break even if small layout changes are operated in a Web application, without modifying the app functionality. An approach based on the automatic injection of HTML tag attributes named hooks in the source code of Web templates has been recently proposed to solve this problem. Such hooks allow the unique identification of GUI items to be located during test case execution. This technique showed its effectiveness in a preliminary validation study, where it allowed to significantly reduce the number of test case locator breakages in regression testing of student-made Web applications. This paper presents a further validation study where we compared the robustness of hook-based test cases against state-of-the-art and state-of-the-practice techniques for locating GUI objects. We proposed a three dimensional model for classifying different types of layout changes and used it to define a benchmark of realistic changes. Thanks to the model, we systematically compared the robustness of test cases generated by different techniques with respect to specific types of changes and studied the relationship between fragility issues and types of changes in different test case generation techniques.