Lab6 - Deep Neural Networks (DNN)

Course: ITM 390 004 - Machine Learning
Lecturer: Sothea HAS, PhD

Objective: This lab is designed for you to apply Deep Neural Networks in image classification task. You will try to explore how different choices of hyperparameters such as learning_rate, batch_size, actiavation function, number of epoches, architecture of the network influences the model performance.

1. Fashion MNIST Dataset

Fashion-MNIST is a dataset of Zalando’s article images—consisting of a training set of 60, 000 examples and a test set of 10, 000 examples. Each example is a 28x28 grayscale image, associated with a label from 10 classes. Zalando intends Fashion-MNIST to serve as a direct drop-in replacement for the original MNIST dataset for benchmarking machine learning algorithms. It shares the same image size and structure of training and testing splits, similar to the hand-written digit Minst dataset.

A. Import the dataset into python environment and print the first 12 training items (3 rows and 4 columns) of this dataset with title corresponding to their actual item name (you can find the true label of each item here: https://www.kaggle.com/datasets/zalando-research/fashionmnist).

  • Check if the classes are balanced or not?

  • Preprocess the data and target by:

    • Normalize the input scale.
    • Perform one-hot encoding on the target.
import kagglehub
import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
# To do

B. Build a DNN with your choice of hyperparameters to classify the iterms (You can use your favorite Python modules such as Keras or Pytorch to build the model).

  • What’s the purpose of the learning curve (train and validation curves)? Plot the learning curve of the training phase.

  • Compute the confusion matrix of the prediction vs actual label for the test dataset, then compute suitable test metrics of the trained model.

# To do

C. Vary the hyperparameters such as mini-batch, number of epochs, penalty strength,… of the network to improve the performance of the network.

  • Plot the learning curves of each training phase (when you make change to new hyperparameters, you should retrain the data and plot a new learning curve).

  • Select a final choice of all hyperparameters and retrain the network. Compute confusion matrix and test metrics.

  • Plot some misclassified items.

# To do

D. Classical ML Models

  • Implement and fine-tune some classical ML models we have studied such as:
    • \(k\)-NN
    • Random Forest
    • XGBoost…

to classify the items of the FashionMnist dataset above. - Compute the confusion matrix of each method and evaluate its test metrics then compare to DNN model above. - Conclude.

# To do

2. Cybersecurity Intrusion Dectection Dataset (Optional/Exploration)

This Cybersecurity Intrusion Detection Dataset is designed for detecting cyber intrusions based on network traffic and user behavior. The description of the data is available here: https://www.kaggle.com/datasets/dnkumars/cybersecurity-intrusion-detection-dataset.

A. Import the data into the environment.

# To do

B. Preprocess, clean and split the data into training and testing for fitting ML models.

# To do

C. Build and fine-tune classical ML models and DNN to detect intrusion activities. Evaluate test metrics and compare.

References

\(^{\text{📚}}\) Deep Learning, Ian Goodfellow. (2016)..
\(^{\text{📚}}\) Hands-on ML with Sklearn, Keras & Tensorflow, Aurélien Geron (2017)..
\(^{\text{📚}}\) Heart Disease Dataset.
\(^{\text{📚}}\) Backpropagation, 3B1B.