While you are experimenting with Azure autoML service, you need to configure the Tuning Hyperparameters feature.
You want autoML to try running experiments varying the number_of_hidden_layers parameter of a neural network algorithm, as well as the batch_size.
Which is NOT a valid configuration for the autoML runs?
Click on the arrows to vote for the correct answer
A. B. C. D.Answer: A.
Option A is CORRECT because the Bayesian sampling method can be used with choice, uniform and quniform space definition, but it cannot be combined with early termination option.
Option B is incorrect becausethe search space can be defined by a quniform distribution; random sampling can be used to select parameter combinations; Bandit policy as a termination strategy can be a valid option.
Option C is incorrect because the search space can be defined as a range of values; random sampling can be used to select parameter combinations; MedianStopping can be a valid option as a termination strategy.
This is a valid option.
Option D is incorrect because for discrete hyperparameters, the search space can be a choice of values, grid sampling can be used to try all possible combinations; if the number of combinations is not too high, the early termination might not be necessary, so this is a valid option.
Reference:
Azure AutoML is a service that enables users to build, train, and deploy machine learning models on Azure with ease. It also provides a range of features to help users optimize their models, such as the Tuning Hyperparameters feature, which allows users to configure and run experiments with different hyperparameter configurations.
In this scenario, the user wants to vary two hyperparameters, number_of_hidden_layers and batch_size, using the Tuning Hyperparameters feature. The user also wants to try different configurations for the search space, sampling, and early termination.
Option A: Search space: normal; sampling: Bayesian; early termination: Yes
This configuration is valid. The normal search space is a continuous distribution that is commonly used for neural network hyperparameters. Bayesian sampling is an intelligent sampling method that uses previous trials to guide the search for new hyperparameters. Early termination is enabled, which allows AutoML to stop runs early if they are not showing promising results.
Option B: Search space: quniform; sampling: random; early termination: BanditPolicy
This configuration is also valid. The quniform search space is a discrete uniform distribution, which is suitable for batch_size. Random sampling is a simple method that randomly selects hyperparameters for each run. BanditPolicy is an early termination policy that stops runs if they are not showing improvement compared to the best run so far.
Option C: Search space: range; sampling: random; early termination: MedianStoppingPolicy
This configuration is valid as well. The range search space is a continuous distribution, which is suitable for number_of_hidden_layers. Random sampling is used to randomly select hyperparameters, and MedianStoppingPolicy is an early termination policy that stops runs if the median performance of the previous runs is not improving.
Option D: Search space: choice; sampling: grid; early termination: None.
This configuration is not valid. The choice search space is used for categorical hyperparameters, but number_of_hidden_layers is a continuous hyperparameter. Grid sampling is a method that tests all possible combinations of hyperparameters in a predefined range, which can be computationally expensive. Finally, early termination is not enabled, which means that AutoML will not stop runs early if they are not showing promising results.
In conclusion, option D is the answer to the question. It is not a valid configuration for the Tuning Hyperparameters feature of Azure AutoML.