Knowledge Engineering Competition ICKEPS

List of Papers


Stefan Edelkamp, University of Dortmund, Gemany
Jeremy Frank, NASA Ames, U.S.A.


Knowledge Engineering (KE) for AI Planning has been defined as the process that deals with the acquisition, validation and maintenance of planning domain models, and the selection and optimization of appropriate planning machinery to work on them. Hence, KE processes support the planning process: they comprise all of the off-line, knowledge-based aspects of planning that are to do with the application being built, and any on-line processes that cause changes in the planners domain model. We expect the competition to encourage the development of tools across the whole KE area including domain modeling, heuristic acquisition, planner-domain matching, and domain knowledge validation.

The International Competition on Knowledge Engineering for Planning and Scheduling (ICKEPS) was planned as a continuing bi-annual event, hosted at the International Conference on Automated Planning and Scheduling (ICAPS). The objectives of the competition were to promote tools for knowledge acquisition and domain modeling, to accelerate knowledge engineering research in AI, and to encourage the development of software platforms that promise more rapid, accessible, and effective ways to construct reliable and efficient systems.

The 2nd ICKEPS has been hosted at ICAPS-2007. It builds on the previous competition, held in 2005. The 2005 competition featured tools with different strengths, which were demonstrated on different planning domains, and for which the final rankings were based solely on the authors' demonstration of the tools to the committee. As such, the rankings were quite subjective. For the 2007 competition, we decided to imposed a new knowledge engineering scenario, in which quantitative data instead of subjective reviews could be made available.

The competition in 2007 uses simulations of the planning and scheduling domains as the basis of the competition. These simulations were made available via a web service. The simulations are described in a short text document, including a description of the simulation API. Competitors retrieve problem instances, submit plans for each instance, and receive feedback describing the quality of the plan as well as any errors. Each participants interaction with simulators is logged, and the logs form significant inputs for the tool evaluation.

The competition was open to authors of a tool, an integrated tools environment or tools platform, where the tool helps knowledge engineering for AI P&S purposes in at least one of the following categories: knowledge formulation (the acquisition and encoding of domain structure and/or control heuristics) planner configuration (fusing application knowledge with a P&S engine) validation of the domain model (e.g. using visualization, analysis, re-formulation) or validation of the P&S system as a whole (e.g. using plan/schedule visualization) knowledge refinement and maintenance (e.g. through automated learning/training, or a mixed initiative P&S process).

In order to separate ICKEPS from the other bi-annual event, the International Planning Competition (IPC), which measures performance on planning benchmarks, stand-alone planners/schedulers were not eligible at ICKEPS, but could be a part of a tool. To foster further progress in research, we additionally imposed that competition entries have be distributable without any associated fees; the competition is otherwise open to all participants.

The simulators themselves were of medium to high fidelity over a wide range of domains, and were provided by both commercial and academic partners. Adventium Labs provided the Behavioral Adversary Modeling System, Palo Alto Research Center (with assistance from Arizona State University) provided the On-Line Manufacturing domain, the Space Telescope Science Institute provided the Ground-based Astronomical Telescope Simulation, the Australian National University provided the Power Supply Restoration simulation, and the University of Dortmund and University of Twente provided the Graph Transformation System domain.

Unfortunately, the number of registered participants to the competition was too small for a fair comparison. We only received three entries (plus one showcase). Despite this, the main contribution of ICKEPS remains the web interface of the simulators, which will serve as a long-term challenge for the planning and scheduling community.

This volume includes an overview paper that describes the simulation infrastructure of the competition. The main body of the proceedings is dedicated to describe the simulator systems. The presentation uncover the modeling challenges imposed by the specific domains. We also included the technical papers of the competitor systems, including the showcase system. Therefore, the set of contributions includes improvements of previous competitors ItSimple and Gipo, a new planning framework based on OWL ontologies, and a constraint reasoning modeling support system.


Roman Barták, Charles University, Czech Republic
Lee McCluskey, University of Huddersfield, United Kingdom
Stefan Edelkamp, University of Dortmund, Germany (co-chair)
Jeremy Frank, NASA Ames, U.S.A. (co-chair)
Robert P. Goldman, SIFT, U.S.A.
Hector Geffner, Departamento de Tecnologia UPF, Spain
Robert Hawkins, Space Telescope Science Institute, Baltimore, U.S.A.
Wheeler Ruml, Palo Alto Research Center (PARC), U.S.A.

List of Papers

Competition Infrastructure

Stefan Edelkamp, Jeremy Frank, and Mark Kellershoff


Mark Boddy and Hazel Shackleton

J. Benton, Minh B. Do, and Wheeler Ruml

Robert Hawkins

Sylvie Thiebaux

Stefan Edelkamp and Arend Rensink

Competitors/Show Cases

Eric Bouillet, Mark Feblowitz, Zhen Liu, Anand Ranganathan and Anton Riabov

Tiago S. Vaquero, Victor Romero, Fernando M. Sette, Flavio Tonidandel and Jose Reinaldo Silva

Ron M. Simpson

Debdeep Banerjee and Ralph Becket

List of Authors

Banerjee, D.
Becket, R.
Benton, J.
Boddy, Mark
Bouillet, Eric
Do, Minh B.
Edelkamp, Stefan
Feblowitz, Mark
Frank, Jeremy
Hawkins, Robert
Kellershoff, Mark
Liu, Zhen
Ranganathan, Anand
Rensink, Arendt
Riabov, Anton
Romero, Victor
Ruml, Wheeler
Shackleton, Hazel
Silva, Jose Reinaldo
Simpson, Ron
Thiebaux, Sylvie
Tonidandel, Flavio
Vaquero, Tiago
Zhen, Liu

Ome © Marjorie Mikasen 2005