TP5 - Clustering

Exploratory Data Analysis & Unsuperivsed Learning
Lecturer: Dr. HAS Sothea
TP: Mr. UANN Sreyvi

Objective: Clustering is technique of ML and Data Analysis used to group similar data points together. The goal is to partition a dataset into distinct subsets, or clusters, such that data points within each cluster are more similar to each other than to those in other clusters. This practical class aims to enhance your understanding of two different clustering algorithms, including their strengths and weaknesses.

The Jupyter Notebook for this TP can be downloaded here: TP5-Clustering.

1. Kmeans Algorithm

We will begin with a toy example using simulated dataset.

a. Implement a function, simulateData(K, N), that generates a synthetic dataset for K-Means clustering. The dataset must consist of K distinct clusters, each containing N observations drawn from a 2D or 3D multivariate normal distribution. Select a value for K where \(K\in\{3,…,8\}\) and visualize the output to verify that the cluster centers are sufficiently separated.

# To do

b. We are trying to detect the number of clsuter \(k\) using Within-class variance:

  • Check the equality: Within-class variation + Between-class variation = Total variation.
  • Perform Kmeans algorithm using KMeans from sklearn.cluster module with different numbers of clusters and compute within-class variation or each case.
  • Draw the values of within-class variances as a function of number of cluster.
  • What do you observe?
from sklearn.cluster import KMeans
# To do

c. Can you propose a systematic approach to approximate the most suitable number of clusters?

  • Run your code \(30\) times on the same data, how many times did you get the number of clusters right? Why?
  • Try to set argument n_init = 5 in KMeans module then use the previous method to approximate the optimal number of clusters. This time, within \(30\) runs, how many times do you get the number of clusters right? Explain why.
# To do

d. Compute and visualize Silhouette Coefficient for each number of clusters considered above. Conclude.

from sklearn.metrics import silhouette_score
# To do

2. Hirachical clustering

Unlike Kmeans algrithm, Hirachical clustering or hcluster does not require a prior number of clusters. It iteratively merges (agglomerative or bottom up approach) into less and less clusters starting from each points being a cluster on its own, or separate the data point (divisive or top-down approach) to create more and more clusters starting from one clsuter containing all data points.

a. Apply Hirachical cluster on the previously simulated dataset.

# To do

b. Plot the associated Dendrograms of the resulting groups.

# To do

c. Can you decide the most suitable number of clusters from the previous dendrogram?

# To do

3. Real dataset

Now apply both algorithms on Kaggle California Housing Prices.

  • Preprocess the data: missing values, encoding the categorical data, standardization…

  • Detect the optimal number of clusters for this dataset.

  • Perform PCA and visualize the obtained result.

  • Interpret each cluster.

# To do

Further Readings