Search strategies¶

Warning

This section is under construction

A search strategy is an algorithm that finds well-performing pipelines in a given search space. Most search strategies are applicable to a variety of search spaces, but often require some specific characteristics. For example, bayesian optimization requires probabilistic spaces, because it uses the probabilistic distribution as the means to explore the space.

The other ingredient of a search strategy, once a search space is defined, is a performance metric to optimize. Most search strategies are compatible with multiple metrics, but some have pre-requisites as well. For example, gradient descent methods requires the performance metric to be differentiable with respect to the search space.

Sampling-based methods¶

Random search¶

Examples: AutoGOAL , Azure ML , H2O AutoML , Hyperopt-sklearn , KNIME AutoML , TransmogrifAI , Vertex AI .

Grid search¶

Examples: H2O AutoML .

Hill climbing¶

Examples: Hyperopt-sklearn , KNIME AutoML .

Gradient descent¶

Examples:

Evolutionary search¶

Examples: AutoGOAL , RECIPE , TPOT .

Bayesian optimization¶

Examples: Auto-Pytorch , Auto-Sklearn , Auto-WEKA , AutoKeras , Azure ML , Hyperopt-sklearn , KNIME AutoML , Vertex AI .

Constructive methods¶

Monte Carlo tree search¶

Examples: ML-Plan .

Reinforced learning¶

Examples:

Meta-learning¶

Portfolio-based methods¶

Examples: Auto-Pytorch .

Warm starting¶

Examples: Auto-Sklearn .