Description

Selector [8] is an ensemble-based automated algorithm configuration method. It queries the integrated algorithm configuration (AC) methods to suggest configuations for the target algorithm. There are more suggestions made than will be evaluated. The set of suggestions is reduced by a learned model. For this, features computed from the suggested configurations are used. The selected configurations are then evaluated in tournaments. The results of the tournaments are stored and used as feedback for the suggesting AC methods.

The infrastructure regarding target algorithm evaluations and data of Selector is implemented with ray. Currently, there are three model based suggesters implemented based on:

  • CPPL [4]

  • GGA [1]

  • SMAC [7]

Additionally the following suggesters are implemented:

  • Default parameter [3, 5]

  • Random parameter [7]

  • Latin Hypercube sampling [9]

  • GGA graph crossover [6]

The selection is based on an iterative scoring selection mechanism [2]. Find the source code at Selector Github.

References

[1]

C. Ansótegui, Y. Malitsky, Horst Samulowitz, M. Sellmann, and K. Tierney. Model-based genetic algorithms for algorithm configuration. In International Joint Conferences on Artificial Intelligence Organization (IJCAI). 2015.

[2]

Carlos Ansótegui, Meinolf Sellmann, Tapan Shah, and Kevin Tierney. Learning to optimize black-box functions with extreme limits on the number of function evaluations. In Dimitris E. Simos, Panos M. Pardalos, and Ilias S. Kotsireas, editors, Learning and Intelligent Optimization, 7–24. Cham, 2021. Springer International Publishing.

[3]

Carlos Ansótegui, Meinolf Sellmann, and Kevin Tierney. A gender-based genetic algorithm for the automatic configuration of algorithms. In Principles and Practice of Constraint Programming, 142–157. 09 2009.

[4]

Adil El Mesaoudi-Paul, Dimitri Weiß, Viktor Bengs, Eyke Hüllermeier, and Kevin Tierney. Pool-based realtime algorithm configuration: a preselection bandit approach. In Ilias S. Kotsireas and Panos M. Pardalos, editors, Learning and Intelligent Optimization, 216–232. 2020.

[5]

Frank Hutter, Holger H. Hoos, Kevin Leyton-Brown, and Thomas Stützle. Paramils: an automatic algorithm configuration framework. Journal of Artificial Intelligence Research (JAIR), pages 267–306, 2009.

[6]

Frank Hutter, Manuel López-Ibáñez, Chris Fawcett, Marius Lindauer, Holger Hoos, Kevin Leyton-Brown, and Thomas Stützle. Aclib: a benchmark library for algorithm configuration. In International Conference on Learning and Intelligent Optimization (LION), 36–40. 02 2014.

[7] (1,2)

Marius Thomas Lindauer, Katharina Eggensperger, Matthias Feurer, Andr'e Biedenkapp, Difan Deng, Caroline Benjamins, René Sass, and Frank Hutter. Smac3: a versatile bayesian optimization package for hyperparameter optimization. Journal of Machine Learning Research, 23:54:1–54:9, 2022.

[8]

Dimitri Weiss, Elias Schede, and Kevin Tierney. Selector: ensemble-based automated algorithm configuration. under revision, 2025.

[9]

Timgates42. scikit-optimize. 2020.