examprep7

Exam Prep Section #7 2 The English Alphabet Goes α , | β | , C(ross-validation) 2. Brandon is using stochastic gradient descent to optimize a model with 1,000 parameters that can detect vehicles and pedestrians in images. However, Brandon is having trouble choosing hyperparameters such as the learning rate, α , and the batch size, β . (a) Suppose Brandon employs k-fold cross-validation to compute optimal values of the hyperparameters α and | β | . Which of the following is true? ⃝ A. k-fold cross-validation will not yield useful information for selecting α or | β | , and is instead intended only for selecting the regularization parameter λ for regularized models. ⃝ B. k-fold cross-validation may yield suboptimal hyperparameters for α and | β | . ⃝ C. k-fold cross-validation cannot output an optimal set of hyperparameters if the empirical risk is non-convex. ⃝ D. To use cross-validation to find the optimal α or | β | , the loss must be differentiable with respect to α and | β | . (b) Regardless of your answer to the previous question, suppose Brandon uses k-fold cross-validation with α chosen from 0.1, 0.2, 0.4 and | β | chosen from 32, 64, 128. The average cross-validated loss is shown in the below table for each combination of α and | β | . | β | α 0.1 0.2 0.4 32 0.0022 0.7031 0.0370 64 0.0051 0.9075 0.0471 128 0.0018 0.6007 0.0157 What is the most optimal pair of α and | β | ?