Automatic Configuration of Benchmark Sets for Classical Planning

Torralba, Álvaro and Seipp, Jendrik and Sievers, Silvan. (2020) Automatic Configuration of Benchmark Sets for Classical Planning. In: HSDIP 2020: Proceedings of the 12th Workshop on Heuristics and Search for Domain-Independent Planning (HSDIP). pp. 58-66.

[img] PDF - Published Version

Official URL: https://edoc.unibas.ch/81333/

Downloads: Statistics Overview


The benchmarks from previous International Planning Competitions are commonly used to evaluate new planning algorithms. Since this set has grown organically over the years, it has several flaws: it contains duplicate tasks, unsolvable tasks, trivially solvable domains, and domains with modelling errors. Also, diverse domain sizes complicate aggregating results. Most importantly, however, the range of task difficulty is very small in many domains. We propose an automated method for creating benchmarks that solves these issues. To find a good scaling in difficulty, we automatically configure the parameters of benchmark domains. We show that the resulting benchmark set improves empirical comparisons by allowing to differentiate between planners more easily.
Faculties and Departments:05 Faculty of Science > Departement Mathematik und Informatik > Informatik > Artificial Intelligence (Helmert)
UniBasel Contributors:Seipp, Jendrik and Sievers, Silvan
Item Type:Conference or Workshop Item, refereed
Conference or workshop item Subtype:Conference Paper
Note:Publication type according to Uni Basel Research Database: Conference paper
Related URLs:
edoc DOI:
Last Modified:23 Feb 2021 13:20
Deposited On:23 Feb 2021 13:20

Repository Staff Only: item control page