Lab8 - Deep Neural Networks (DNN)

Course: Advanced Machine Learning
Lecturer: Sothea HAS, PhD

Objective: This lab is designed for you to apply Deep Neural Networks in image classification task. You will try to explore how different choices of hyperparameters such as learning_rate, batch_size, actiavation function, number of epoches, architecture of the network influences the model performance.

1. Fashion MNIST Dataset

Fashion-MNIST is a dataset of Zalando’s article images—consisting of a training set of 60, 000 examples and a test set of 10, 000 examples. Each example is a 28x28 grayscale image, associated with a label from 10 classes. Zalando intends Fashion-MNIST to serve as a direct drop-in replacement for the original MNIST dataset for benchmarking machine learning algorithms. It shares the same image size and structure of training and testing splits, similar to the hand-written digit Minst dataset.

A. Import the dataset into python environment and print the first 12 training items (3 rows and 4 columns) of this dataset with title corresponding to their actual item name (you can find the true label of each item here: https://www.kaggle.com/datasets/zalando-research/fashionmnist).

  • Check if the classes are balanced or not?

  • Preprocess the data and target by:

    • Normalize the input scale.
    • Perform one-hot encoding on the target.
import kagglehub
import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
# To do
import kagglehub

# Download latest version
path = kagglehub.dataset_download("zalando-research/fashionmnist")

print("Path to dataset files:", path)
data_test = pd.read_csv(path + '/fashion-mnist_test.csv')
data_train = pd.read_csv(path + '/fashion-mnist_train.csv')
X_train, y_train = data_train.iloc[:,1:], data_train['label']
X_test, y_test = data_test.iloc[:,1:], data_test['label']
X_train.head()
Path to dataset files: /Users/hassothea/.cache/kagglehub/datasets/zalando-research/fashionmnist/versions/4
pixel1 pixel2 pixel3 pixel4 pixel5 pixel6 pixel7 pixel8 pixel9 pixel10 ... pixel775 pixel776 pixel777 pixel778 pixel779 pixel780 pixel781 pixel782 pixel783 pixel784
0 0 0 0 0 0 0 0 0 0 0 ... 0 0 0 0 0 0 0 0 0 0
1 0 0 0 0 0 0 0 0 0 0 ... 0 0 0 0 0 0 0 0 0 0
2 0 0 0 0 0 0 0 5 0 0 ... 0 0 0 30 43 0 0 0 0 0
3 0 0 0 1 2 0 0 0 0 0 ... 3 0 0 0 0 1 0 0 0 0
4 0 0 0 0 0 0 0 0 0 0 ... 0 0 0 0 0 0 0 0 0 0

5 rows × 784 columns

%ls {path}
fashion-mnist_test.csv   t10k-images-idx3-ubyte   train-images-idx3-ubyte
fashion-mnist_train.csv  t10k-labels-idx1-ubyte   train-labels-idx1-ubyte

B. Build a DNN with your choice of hyperparameters to classify the iterms (You can use your favorite Python modules such as Keras or Pytorch to build the model).

  • What’s the purpose of the learning curve (train and validation curves)? Plot the learning curve of the training phase.

  • Compute the confusion matrix of the prediction vs actual label for the test dataset, then compute suitable test metrics of the trained model.

# To do

C. Vary the hyperparameters such as mini-batch, number of epochs, penalty strength,… of the network to improve the performance of the network.

  • Plot the learning curves of each training phase (when you make change to new hyperparameters, you should retrain the data and plot a new learning curve).

  • Select a final choice of all hyperparameters and retrain the network. Compute confusion matrix and test metrics.

  • Plot some misclassified items.

# To do

D. Classical ML Models

  • Implement and fine-tune some classical ML models we have studied such as:
    • \(k\)-NN
    • Random Forest
    • XGBoost…

to classify the items of the FashionMnist dataset above. - Compute the confusion matrix of each method and evaluate its test metrics then compare to DNN model above. - Conclude.

# To do

2. Cybersecurity Intrusion Dectection Dataset (Optional/Exploration)

This Cybersecurity Intrusion Detection Dataset is designed for detecting cyber intrusions based on network traffic and user behavior. The description of the data is available here: https://www.kaggle.com/datasets/dnkumars/cybersecurity-intrusion-detection-dataset.

A. Import the data into the environment.

# To do

B. Preprocess, clean and split the data into training and testing for fitting ML models.

# To do

C. Build and fine-tune classical ML models and DNN to detect intrusion activities. Evaluate test metrics and compare.

References

\(^{\text{📚}}\) Deep Learning, Ian Goodfellow. (2016)..
\(^{\text{📚}}\) Hands-on ML with Sklearn, Keras & Tensorflow, Aurélien Geron (2017)..
\(^{\text{📚}}\) Heart Disease Dataset.
\(^{\text{📚}}\) Backpropagation, 3B1B.