The table also reports LDA (Chapter 4) and multiclass logistic regression. Although packages such as glmnet can handle multiclass logistic regression, they are quite slow on this large dataset. It is much faster and quite easy to fit such a model using the keras software. We just have an input layer and output layer, and omit the hidden layers!

modellr <- keras_model_sequential() %>%
   layer_dense(input_shape = 784, units = 10,
               activation = "softmax")

summary(modellr)

Model: "sequential_2"
_____________________________________________________________________
Layer (type)                   Output Shape               Param #    
=====================================================================
dense_5 (Dense)                (None, 10)                 7850       
=====================================================================
Total params: 7,850
Trainable params: 7,850
Non-trainable params: 0
_____________________________________________________________________

We fit the model just as before.

modellr %>% compile(loss = "categorical_crossentropy",
                    optimizer = optimizer_rmsprop(), metrics = c("accuracy"))
modellr %>% fit(x_train, y_train, epochs = 30,
                batch_size = 128, validation_split = 0.2)
modellr %>% predict_classes(x_test) %>% accuracy(g_test)
[1] 0.9273

Questions

• Define a multiclass logistic regression model. Store it in modellr.
• Compile the model below with the following parameters
  • Use the categorical_crossentropy as the loss function
  • Use the optimizer_adam() as you optimizer
  • Also track the AUC as a metric (you can pass tf$keras$metrics$AUC() as an argument).
• Fit the logistic regression on the training data for 10 epoch and a batch size of 128. Use 20% of the training data for validation.
• Store the predictions on the test data in lrpred.
• Store the AUC of the neural network in lrauc. Use the function pROC::auc(pROC::roc(drop(response), as.numeric(drop(predictor)))), with response and predictor obtained in the previous steps.