Repairing Test Script Commands
Testing is of vital importance in software engineering. Of particular interest is functional GUI testing in which an application's user interface is exercised along requirement scenarios. An exclusively manual activity in the past, it has recently seen the arrival of test automation tools such as HP Quick Test Pro, Robotium and Selenium. These tools execute so-called test scripts which are executable implementations of the traditional requirements scenarios. Test scripts consist of commands that simulate the user's interactions with the GUI (e.g., button clicks and key presses) and of assertions that compare the observed state of the GUI (e.g., the contents of its text fields) with the expected one. Studies suggest that these test scripts are very brittle to changes of the system under test. This brings about the problem of maintaining these software as the system under test evolves.
The goal of this thesis is to help web-developers maintaining their test scripts by providing repair suggestions as the system under test evolves. In this thesis we focus on an intermediate representation of test scripts used by Selenium: Selenese commands. A Selenese command simply consists in a command name and two parameters -- e.g.: ("type", "id=telephone", "(555) 666-7066"). An example of repair could simply be to update the DOM identifier from telephone to phone. Finding such repairs can be formulated as a search problem. First, the previous version of the system under test should be recorded as it is executed with the test commands to repair. Second, these test commands should be transformed so that the current version of the system under test sticks as closely as possible to the previously recorded execution. Finding the best repair may requires to run the current system under test multiple times.
- Provide a trace recorder based on the provided instrumentation platform.
- Develop a trace alignment algorithm to assess how good is a repair of Selenese commands.
- Identify practically relevant repairs category and come up with a search strategy for each one of them.
- Validate the overall approach on preferably real-world web-applications, orelse convincing examples.