TP8 - Linear Regression

Course: INF-604: Data Analysis I
Lecturer: Sothea HAS, PhD

Objective: In this lab, you will reproduce the result we have done in linear and logistic regression. Moreover, you will explore beyond what we have done including using multiple inputs variables, compare their performances and feature engineering to further elevate the model performance.

I. Linear Regression

We will work with Abalone dataset, with detailed information available at UCI Machine Learning Repository. Now, find a way to import the dataset into our environment.

1. Data Preprocessing and EDA

A. Univariate Analysis:

  • Check and modify columns with wrong data type if there is any.
  • Perform statistical summary on all columns. What’s wrong with column Height? How would you solve this problem?
  • Visualize the each quantitative column using boxplot to detect potential outliers. Take note of those problematic columns.
# To do

B. Bivariate Analysis:

  • Compute both Pearson and Spearman correlation matrices on numerical columns. Comment these correlation matrices.
  • Create pair plot (pair-by-pair scatterplot) using sns.pairplot() from seaborn library on all quantitative variables. Should outliers/high leverage points be removed?
  • Is column Sex useful for predicting the target Rings? Provide evidence.
# To do

2. Simple Linear Regression (SLR)

  • Perform 80%-20% data splitting using train_test_split method of sklearn.model_selection. The linear regression model will be built using the 80% part leaving the 20% part for testing how good the model is.
  • Pick one input you think is most suitable and build linear regression model to predict Rings using LinearRegression from sklearn.linear_models.
  • Draw scatterplot of the selected input vs Rings. Add the fitted line to the existing scatterplot.
  • Compute \(R^2\) on the training data and check the residual of the model. Explain the result.
  • Compute \(R^2\) on the test data and add them to the existing scatterplot of fitted line.
  • Check if your selected input is significant in predicting at confidence level of \(95\%\).
  • Interpret the model.
# To do

3. Multiple Linear Regression (MLR)

A. All at once:

  • Build two multiple linear regression models:
    • full_lr: using all inputs columns.
    • eda_lr using some selected columns that you think are suitable.
  • Compare \(R^2_{\text{adj}}\) of the two models on training data and explain the result.
  • Check the residuals of the two models. Explain the result.
  • Compute \(R^2\), Mean Squared Error (MSE) and Mean Absolute Percentage Error (MAPE) from all three models (SLR, full_lr and eda_lr) on the test data.
  • Are all inputs significantly related with the target Rings?
# To do

B. Polynomial Features (Optional)

  • Return to your choice in SLR, now create input data consisting of that input (call it \(X\)) and its square (\(X^2\)). Build multiple linear regression to predict Rings using \(X\) and \(X^2\) together.
  • Draw the scatterplot and the fitted curve.
  • Compute \(R^2\) and \(R^{2}_{\text{adj}}\) on the training data. Then compute \(R^2\), MSE and MAPE on the test data. Is it better than SLR?
  • Check the residual and explain.
  • Conclude your findings: which one do you think is the best model among the four models?
# To do

Further readings