Repairing Test Script Commands

Printer-friendly version

Good knowledge of JavaScript.


Testing is of vital importance in software engineering. Of particular interest is functional GUI testing in which an application's user interface is exercised along requirement scenarios. An exclusively manual activity in the past, it has recently seen the arrival of test automation tools such as HP Quick Test Pro, Robotium and Selenium. These tools execute so-called test scripts which are executable implementations of the traditional requirements scenarios. Test scripts consist of commands that simulate the user's interactions with the GUI (e.g., button clicks and key presses) and of assertions that compare the observed state of the GUI (e.g., the contents of its text fields) with the expected one. Studies suggest that these test scripts are very brittle to changes of the system under test. This brings about the problem of maintaining these software as the system under test evolves.

Goal & Research Activities: 

The goal of this thesis is to help web-developers maintaining their test scripts by providing repair suggestions as the system under test evolves. In this thesis we focus on an intermediate representation of test scripts used by Selenium: Selenese commands. A Selenese command simply consists in a command name and two parameters -- e.g.: ("type", "id=telephone", "(555) 666-7066"). An example of repair could simply be to update the DOM identifier from telephone to phone. Finding such repairs can be formulated as a search problem. First, the previous version of the system under test should be recorded as it is executed with the test commands to repair. Second, these test commands should be transformed so that the current version of the system under test sticks as closely as possible to the previously recorded execution. Finding the best repair may requires to run the current system under test multiple times.

To gain time, the student is expected to use a JavaScript instrumentation platform developed at SOFT. The instrumentation platform enables the user to control the execution of a JavaScript program by providing trap functions -- demonstration here. We foresee the following research activities:

  1. Provide a trace recorder based on the provided instrumentation platform.
  2. Develop a trace alignment algorithm to assess how good is a repair of Selenese commands.
  3. Identify practically relevant repairs category and come up with a search strategy for each one of them.
  4. Validate the overall approach on preferably real-world web-applications, orelse convincing examples.