Neural Network Heuristic Functions for Classical Planning: Reinforcement Learning and Comparison to Other Methods

Ferber, Patrick and Geißer, Florian and Trevizan, Felipe and Helmert, Malte and Hoffmann, Jörg. (2021) Neural Network Heuristic Functions for Classical Planning: Reinforcement Learning and Comparison to Other Methods. PRL Workshop – Bridging the Gap Between AI Planning and Reinforcement Learning.

[img] PDF - Published Version

Official URL: https://edoc.unibas.ch/84326/

Downloads: Statistics Overview


How can we train neural network (NN) heuristic functions for classical planning, using only states as the NN input? Prior work addressed this question by (a) supervised learning and/or (b) per-domain learning generalizing over problem in- stances. The former limits the approach to instances small enough for training data generation, the latter to domains and instance distributions where the necessary knowledge generalizes across instances. Clearly, reinforcement learning (RL) on large instances can potentially avoid both difficul- ties. We explore this here in terms of three methods drawing on previous ideas relating to bootstrapping and approximate value iteration, including a new bootstrapping variant that es- timates search effort instead of goal distance. We empirically compare these methods to (a) and (b), aligning three differ- ent NN heuristic function learning architectures for cross- comparison in an experiment of unprecedented breadth in this context. Key lessons from this experiment are that our meth- ods and supervised learning are highly complementary; that per-instance learning often yields stronger heuristics than per- domain learning; and that LAMA is still dominant but is out- performed by our methods in one benchmark domain .
Faculties and Departments:05 Faculty of Science > Departement Mathematik und Informatik > Informatik > Artificial Intelligence (Helmert)
UniBasel Contributors:Ferber, Patrick and Helmert, Malte
Item Type:Other
Publisher:PRL Workshop
Number of Pages:9
Note:Publication type according to Uni Basel Research Database: Other publications
Related URLs:
edoc DOI:
Last Modified:22 Sep 2021 12:35
Deposited On:22 Sep 2021 12:33

Repository Staff Only: item control page