Coen De Roover

End-to-end (E2E) testing is a software validation approach that simulates realistic user scenarios throughout the entire workflow of an application. In the context of web applications, E2E testing involves two activities: Graphic User Interface (GUI) testing, which simulates user interactions with the web app’s GUI through web browsers, and performance testing, which evaluates system workload handling. Despite its recognized importance in delivering high-quality web applications, the availability of large-scale datasets featuring real-worldE2E web tests remains limited, hindering research in the field.To address this gap, we present E2EGit, a comprehensive dataset of non-trivial open-source web projects collected on GITHUB that adopt E2E testing. By analyzing over 5,000web repositories across popular programming languages (JAVA,JAVASCRIPT, TYPESCRIPT and PYTHON), we identified 472repositories implementing 43,670 automated Web GUI tests with popular browser automation frameworks (SELENIUM, PLAY-WRIGHT, CYPRESS, PUPPETEER), and 84 repositories that featured 271 automated performance tests implemented leveraging the most popular open-source tools (JMETER, LOCUST). Among these, 13 repositories implemented both types of testing for a total of 786 Web GUI tests and 61 performance tests.

Infrastructure as Code is an emerging paradigm to automate the configuration of cloud infrastructures. Infrastructure code often processes secret information, such as passwords or private keys. Mishandling such secrets can lead to information disclosure vulnerabilities, yet existing efforts to detect them rely on pattern matching of parameter and variable names, causing false positives and negatives due to suboptimal string patterns. This paper aims to address these limitations by assessing the effectiveness of traditional Machine Learning (ML) and transformer-based Language Model (LM) classifiers to predict sensitive module parameters in Ansible, one of the most popular IaC tools. We collect a dataset of over 160,000 Ansible module parameters and their documentation, containing more than 16,000 parameters that expect secret data. Then, we train several ML algorithms and find that the Random Forest algorithm performs best, achieving 93.5% precision but limited recall (72.7%). In parallel, we evaluate multiple pretrained zero-shot language models, which achieve a recall of up to 90.4% at the expense of a lower precision of up to 88.5%. We subsequently fine-tune the language models, resulting in nearly perfect precision (99.8%) and recall (99.8%) on the ground truth dataset. We compare the best performing ML and LM classifiers to two baselines that use string patterns. We find that the ML classifier achieves a performance comparable to the two baselines, while the fine-tuned LM outperforms all approaches. A qualitative comparison reveals that the approaches are complementary to the baselines, motivating future work to use prediction models to reduce false positives in reports generated by inexpensive baselines. However, we also find that the fine-tuned LM misses several secrets caused by noise in the dataset, highlighting the importance of fine-tuning on a high-quality ground truth.

The recently introduced WebAssembly standard aims to be a portable compilation target, enabling the cross-platform distribution of programs written in a variety of languages. We propose an approach to slice WebAssembly programs in order to enable applications in reverse engineering, code comprehension, and security among others. Given a program and a location in that program, program slicing produces a minimal version of the program that preserves the behavior at the given location. Specifically, our approach is a static, intra-procedural, backwards slicing approach that takes into account WebAssembly-specific dependences to identify the instructions of the slice. To do so it must correctly overcome the considerable challenges of performing dependence analysis at the binary level. Furthermore, for the slice to be executable, the approach needs to ensure that the stack behavior of its output complies with WebAssembly’s validation requirements. We implemented and evaluated our approach on a suite of 8,386 real-world WebAssembly binaries, finding that the average size of the 97,728,325 slices computed is 46% of the original code, an improvement over the 60% attained by related work on slicing ARM binaries. To gain a more qualitative understanding of the slices produced by our approach, we compared them to 1,956 source-level slices of benchmark C programs. This inspection helps to illustrate the slicer’s strengths and to uncover potential future improvements.

Design-by-contract is a development best practice that requires the interactions between software components to be governed by precise specifications, called contracts. Contracts often take the form of pre- and post-conditions on function definitions, and are usually translated to (frequently redundant) run-time checks. So-called soft contract verifiers have been proposed to reduce the run-time overhead introduced by such contract checks by verifying parts of the contracts ahead of time, while leaving those that cannot be verified as residual runtime checks. In the state of the art, static analyses based on the Abstracting Abstract Machines (AAM) approach to abstract interpretation have been proposed for implementing such soft verifiers. However, these approaches result in whole-program analyses which are difficult to scale. In this paper, we propose a scalable summary-based compositional analysis for soft contract verification, which summarises both the correct behaviour and erroneous behaviour of all functions in the program using symbolic path conditions. Information from these summaries propagates backwards through the call graph, reducing the amount of redundant analysis states and improving the overall performance of the analysis. This backwards flow enables path constraints associated with erroneous program states to flow to call sites where they can be refuted, whereas in the state of the art they can only be refuted using the information available at the original location of the error. To demonstrate our improvements in both precision and performance compared to the state-of-the-art, we implemented our analysis in a framework called MAF (short for Modular Analysis Framework) — a framework for the analysis of higher-order dynamic programming languages. We conducted an empirical study and found an average performance improvement of 21%, and an average precision improvement of 38.15%.

Selecting an appropriate library for reuse within a vast software ecosystem can be a daunting task. A list of features for each library, i.e., a short description of the functionality that can be reused with code examples that illustrate its usage, may alleviate this problem. In this paper, we propose a data-driven approach that uses both the code snippets and the accompanying natural language descriptions from Stack Overflow posts to produce a list of features of a given library. Each extracted feature corresponds to a cluster of API classes and methods considered related based on attributes of the Stack Overflow posts in which they appear. We evaluated the approach considering seven Maven libraries and compared the resulting features against library descriptions from cookbook-like tutorials. The approach achieves an average accuracy of 67% across the seven libraries for the tutorial-like features. For at least 73% of the features extracted by the approach but missing from the documentation, we found a matching library usage in a corpus of GitHub projects. These results suggest that our clusters represent library features, which paves the way to better tool support for documenting software libraries and for selecting a library in an ecosystem.

Code templates are a convenient means to specify source code snippets of interest. However, despite a low learning curve, it may not always be evident to write templates that produce only the desired matches. A template may be too general and result in false positives, or too specific and produce false negatives. We assist the users of Ekeko/X, our template-based search and transformation tool for Java using a two-component approach: First, we present a suite of mutation operators to simplify editing templates. Some operators only modify a single component of a template, whereas others can automate high-level operations involving multiple components, such as abstracting away the declaration of a method and all calls to it using metavariables. Second, we introduce a genetic search algorithm that uses our suite of mutation operators and can recommend a sequence of changes to a template, such that it matches with a desired set of code snippets. The approach is evaluated by letting it infer a template that matches all instances of a design pattern, given one instance as a starting template.

We present an approach to detect function purity in JavaScript. A function is pure if none of its applications cause observable side-effects. The approach is based on a pushdown flow analysis that besides traditional control and value flow also keeps track of write effects. To increase the precision of our purity analysis, we combine it with an intraprocedural analysis to determine freshness of variables and object references. We formalize the core aspects of our analysis, and discuss our implementation used to analyze several common JavaScript benchmarks. Experiments show that our technique is capable of detecting function purity, even in the presence of higher-order functions, dynamic property expressions, and prototypal inheritance.