The work addresses a central challenge in modern software engineering: To ensure that updates do not unintentionally slow down a program, developers continuously perform performance tests (microbenchmarks). In practice, however, this benchmarking is extremely computationally and time-intensive.

Improving Efficiency in Software Development

The award-winning research team has developed a novel, intelligent approach to optimize the execution of these benchmarks. Instead of running the benchmarks according to a rigid time budget, the new system dynamically determines—based on statistical criteria (Minimum Detectable Effect)—when sufficient data is available for a reliable result. The benchmark is stopped precisely when the sought-after performance change can be proven—or reliably ruled out.

Less Effort, Consistent Reliability

Evaluation using large open-source projects shows that this approach can drastically reduce the resource-intensive effort required for benchmarking. Thanks to sound statistical methods, the system continues to deliver highly precise and reliable results. The publication thus makes an important contribution to more efficient and sustainable software development.