By Randley Morales

Problem Statement

Business Context

Workplace safety in hazardous environments like construction sites and industrial plants is crucial to prevent accidents and injuries. One of the most important safety measures is ensuring workers wear safety helmets, which protect against head injuries from falling objects and machinery. Non-compliance with helmet regulations increases the risk of serious injuries or fatalities, making effective monitoring essential, especially in large-scale operations where manual oversight is prone to errors and inefficiency.

To overcome these challenges, SafeGuard Corp plans to develop an automated image analysis system capable of detecting whether workers are wearing safety helmets. This system will improve safety enforcement, ensuring compliance and reducing the risk of head injuries. By automating helmet monitoring, SafeGuard aims to enhance efficiency, scalability, and accuracy, ultimately fostering a safer work environment while minimizing human error in safety oversight.

Objective

As a data scientist at SafeGuard Corp, you are tasked with developing an image classification model that classifies images into one of two categories:

• With Helmet: Workers wearing safety helmets.
• Without Helmet: Workers not wearing safety helmets.

Data Description

The dataset consists of 631 images, equally divided into two categories:

• With Helmet: 311 images showing workers wearing helmets.
• Without Helmet: 320 images showing workers not wearing helmets.

Dataset Characteristics:

• Variations in Conditions: Images include diverse environments such as construction sites, factories, and industrial settings, with variations in lighting, angles, and worker postures to simulate real-world conditions.
• Worker Activities: Workers are depicted in different actions such as standing, using tools, or moving, ensuring robust model learning for various scenarios.

Installing and Importing the Necessary Libraries

!pip install tensorflow[and-cuda] numpy==1.26.5 -q

ERROR: Ignored the following versions that require a different python version: 1.21.2 Requires-Python >=3.7,<3.11; 1.21.3 Requires-Python >=3.7,<3.11; 1.21.4 Requires-Python >=3.7,<3.11; 1.21.5 Requires-Python >=3.7,<3.11; 1.21.6 Requires-Python >=3.7,<3.11
ERROR: Could not find a version that satisfies the requirement numpy==1.26.5 (from versions: 1.3.0, 1.4.1, 1.5.0, 1.5.1, 1.6.0, 1.6.1, 1.6.2, 1.7.0, 1.7.1, 1.7.2, 1.8.0, 1.8.1, 1.8.2, 1.9.0, 1.9.1, 1.9.2, 1.9.3, 1.10.0.post2, 1.10.1, 1.10.2, 1.10.4, 1.11.0, 1.11.1, 1.11.2, 1.11.3, 1.12.0, 1.12.1, 1.13.0, 1.13.1, 1.13.3, 1.14.0, 1.14.1, 1.14.2, 1.14.3, 1.14.4, 1.14.5, 1.14.6, 1.15.0, 1.15.1, 1.15.2, 1.15.3, 1.15.4, 1.16.0, 1.16.1, 1.16.2, 1.16.3, 1.16.4, 1.16.5, 1.16.6, 1.17.0, 1.17.1, 1.17.2, 1.17.3, 1.17.4, 1.17.5, 1.18.0, 1.18.1, 1.18.2, 1.18.3, 1.18.4, 1.18.5, 1.19.0, 1.19.1, 1.19.2, 1.19.3, 1.19.4, 1.19.5, 1.20.0, 1.20.1, 1.20.2, 1.20.3, 1.21.0, 1.21.1, 1.22.0, 1.22.1, 1.22.2, 1.22.3, 1.22.4, 1.23.0, 1.23.1, 1.23.2, 1.23.3, 1.23.4, 1.23.5, 1.24.0, 1.24.1, 1.24.2, 1.24.3, 1.24.4, 1.25.0, 1.25.1, 1.25.2, 1.26.0, 1.26.1, 1.26.2, 1.26.3, 1.26.4, 2.0.0, 2.0.1, 2.0.2, 2.1.0, 2.1.1, 2.1.2, 2.1.3, 2.2.0, 2.2.1, 2.2.2, 2.2.3, 2.2.4, 2.2.5, 2.2.6, 2.3.0, 2.3.1, 2.3.2, 2.3.3, 2.3.4, 2.3.5)
ERROR: No matching distribution found for numpy==1.26.5

import tensorflow as tf
print("Num GPUs Available:", len(tf.config.list_physical_devices('GPU')))
print(tf.__version__)

Num GPUs Available: 0
2.19.0

Note:

• After running the above cell, kindly restart the notebook kernel (for Jupyter Notebook) or runtime (for Google Colab) and run all cells sequentially from the next cell.

• On executing the above line of code, you might see a warning regarding package dependencies. This error message can be ignored as the above code ensures that all necessary libraries and their dependencies are maintained to successfully execute the code in this notebook.

import os
import random
import numpy as np                                                                               # Importing numpy for Matrix Operations
import pandas as pd
import seaborn as sns
import matplotlib.image as mpimg                                                                 # Importing pandas to read CSV files
import matplotlib.pyplot as plt                                                                  # Importting matplotlib for Plotting and visualizing images
import math                                                                                      # Importing math module to perform mathematical operations
import cv2


# Tensorflow modules
import keras
import tensorflow as tf
from tensorflow.keras.preprocessing.image import ImageDataGenerator                              # Importing the ImageDataGenerator for data augmentation
from tensorflow.keras.models import Sequential                                                   # Importing the sequential module to define a sequential model
from tensorflow.keras.layers import Dense,Dropout,Flatten,Conv2D,MaxPooling2D,BatchNormalization # Defining all the layers to build our CNN Model
from tensorflow.keras.optimizers import Adam,SGD                                                 # Importing the optimizers which can be used in our model
from sklearn import preprocessing                                                                # Importing the preprocessing module to preprocess the data
from sklearn.model_selection import train_test_split                                             # Importing train_test_split function to split the data into train and test
from sklearn.metrics import confusion_matrix
from tensorflow.keras.models import Model
from keras.applications.vgg16 import VGG16                                                       # Importing confusion_matrix to plot the confusion matrix

# Display images using OpenCV
from google.colab.patches import cv2_imshow

#Imports functions for evaluating the performance of machine learning models
from sklearn.metrics import confusion_matrix, f1_score,accuracy_score, recall_score, precision_score, classification_report
from sklearn.metrics import mean_squared_error as mse                                            # Importing cv2_imshow from google.patches to display images

# Ignore warnings
import warnings
warnings.filterwarnings('ignore')

# Set the seed using keras.utils.set_random_seed. This will set:
# 1) `numpy` seed
# 2) backend random seed
# 3) `python` random seed
tf.keras.utils.set_random_seed(812)

Data Overview

Loading the data

from google.colab import drive
drive.mount('/content/drive')

Mounted at /content/drive

# Load the image file of the dataset
images = np.load("/content/drive/MyDrive/Computer Vision/images_proj.npy")

# Load the labels file of the dataset
labels = pd.read_csv("/content/drive/MyDrive/Computer Vision/Labels_proj.csv")

print(f"Image data shape: {images.shape}")
print(f"Labels data shape: {labels.shape}")

Image data shape: (631, 200, 200, 3)
Labels data shape: (631, 1)

Observations:

Dataset Dimensions & Structure:

• Sample Size: The dataset is relatively small, containing 631 images. This is confirmed by the shape output (631, ...) for both images and labels.

• Image Resolution: The images are 200x200 pixels.

• Color Channels: The images have 3 channels, indicating they are color images (RGB/BGR).

• Consistency: The number of labels (631) matches the number of images perfectly, so there is no missing data alignment issue at this stage.

Exploratory Data Analysis

Plot random images from each of the classes and print their corresponding labels.

# Converting the images from BGR to RGB using cvtColor function of OpenCV
for i in range(len(images)):
  images[i] = cv2.cvtColor(images[i], cv2.COLOR_BGR2RGB)

# Get indices of each class
helmet_indices = np.where(labels == 1)[0]
no_helmet_indices = np.where(labels == 0)[0]

# Choose 10 random images from each class
helmet_samples = np.random.choice(helmet_indices, 10, replace=False)
no_helmet_samples = np.random.choice(no_helmet_indices, 10, replace=False)

# Combine them into one array (20 images total)
selected_indices = np.concatenate([helmet_samples, no_helmet_samples])

# OPTIONAL: Shuffle so they are not grouped
np.random.shuffle(selected_indices)

# Create a 4x5 subplot grid
fig, axes = plt.subplots(4, 5, figsize=(15, 12))
axes = axes.flatten()

# Display each selected image
for i, idx in enumerate(selected_indices):
    axes[i].imshow(images[idx])
    title = "Worker WITH Helmet" if labels.loc[idx, 'Label'] == 1 else "Worker WITHOUT Helmet"
    axes[i].set_title(title)
    axes[i].axis('equal')

plt.tight_layout()
plt.show()

Observations:

1. Task Identification

• Binary Classification: Based on the variable names in the helmet_indices and no_helmet_indices, the goal of this project is Helmet Detection.

Classes:

• Class 1: Helmet (Indices where label is 1)

• Class 0: No Helmet (Indices where label is 0)

2. Data Preprocessing

• Color Space Conversion: The code explicitly converts images from BGR to RGB (cv2.cvtColor(..., cv2.COLOR_BGR2RGB)).

  • Observation: This suggests the original data was likely saved or processed using OpenCV (which defaults to Blue-Green-Red). This conversion is crucial because if you plotted these images using libraries like Matplotlib without this step, the colors would look inverted (e.g., red objects would appear blue).

• File Formats:

  • The images are loaded from a .npy file (NumPy binary), which is faster and more efficient for loading large array data compared to reading individual image files (JPG/PNG).

  • The labels are loaded from a standard .csv file.

3. Significant Scale and Crop Discrepancy (Potential Bias)

The most critical observation is a distinct difference in how the two classes are framed:

• "Worker WITHOUT Helmet" (Class 0): These images appear to be tight crops of faces. They are zoomed in significantly, often cutting off the forehead or chin.

• "Worker WITH Helmet" (Class 1): These images are generally zoomed out, showing the upper body, shoulders, and significant background context (construction sites, machinery, etc.).

Why this matters: This is a major risk for machine learning models. My model might accidentally learn to classify images based on "zoom level" or "amount of background clutter" rather than the actual presence of a helmet. It might assume that "Face filling the whole frame = No Helmet" and "Small face with background = Helmet."

4. Visual Diversity

• Helmet Colors: The "With Helmet" class shows good diversity in helmet colors (Yellow, Red, White/Blue), which will help the model generalize so it doesn't associate "Helmet" with just the color "Yellow."

• Lighting: There is significant variation in lighting, from bright outdoor sunlight to darker indoor/tunnel environments.

4. Code Logic

• Balanced Sampling: The code explicitly selects 10 random images from the "helmet" list and 10 from the "no_helmet" list. This is good practice for visualization because it ensures you see a representative sample of both classes, even if the underlying dataset is imbalanced.

Checking for class imbalance

# Create a count plot
plt.figure(figsize=(6, 4))
ax = sns.countplot(x=labels.iloc[:, 0], palette=['red', 'green'])

# Add exact counts on top of bars
for p in ax.patches:
    ax.annotate(f'{int(p.get_height())}', (p.get_x() + p.get_width() / 2, p.get_height()),
                ha='center', va='bottom', fontsize=10, )

# Add labels
plt.xlabel("Class Labels", fontsize=12)
plt.ylabel("Number of Images", fontsize=12)
plt.title("Count of Images per Class", fontsize=14)
plt.xticks(ticks=[0, 1], labels=["Without Helmet (0)", "With Helmet (1)"])  # Rename x-axis labels

# Show plot
plt.show()

Observations:

Class Balance

The dataset exhibits a near-perfect balance between the two classes, with a distribution of approximately 50.7% versus 49.3%. Consequently, no resampling techniques (oversampling or undersampling) are required to address class imbalance.

• Without Helmet (0): 320 images

• With Helmet (1): 311 images

• Total = 631 (matches your earlier shapes)

Data Preprocessing

# Function to plot the original and processed images side by side
def grid_plot(img1,img2,gray=False):
    fig, axes = plt.subplots(1, 2, figsize=(12, 5))
    axes[0].imshow(img1)
    axes[0].set_title('Original Image')
    axes[0].axis('off')

    if gray:
      axes[1].imshow(img2,cmap='gray')
    else:
      axes[1].imshow(img2)
    axes[1].set_title('Processed Image')
    axes[1].axis('off')

    plt.show()

Converting images to grayscale

images_gray = []
for i in range(len(images)):
    img_gray = cv2.cvtColor(images[i], cv2.COLOR_BGR2GRAY)  # Convert to grayscale
    images_gray.append(img_gray)

# Display a sample Original vs Grayscale image
n = random.randint(0, len(images_gray) - 1)

orig = images[n]
gray = images_gray[n]

plt.figure(figsize=(8, 4))

# Original image
plt.subplot(1, 2, 1)
plt.imshow(orig)
plt.title("Original")
plt.axis('equal')

# Grayscale (preprocessed) image
plt.subplot(1, 2, 2)
plt.imshow(gray, cmap='gray')
plt.title("Grayscale (Preprocessed)")
plt.axis('equal')

plt.show()

# Choose how many original+gray pairs to include
num_pairs = 4   # change this value as you like

# Build a list that alternates: [orig1, gray1, orig2, gray2, ...]
mixed_images = []
for i in range(num_pairs):
    mixed_images.append(images[i])        # original
    mixed_images.append(images_gray[i])   # grayscale

cols = 4
rows = math.ceil(len(mixed_images) / cols)

plt.figure(figsize=(12, 3 * rows))

for idx, img in enumerate(mixed_images):
    plt.subplot(rows, cols, idx + 1)
    if idx % 2 == 1:   # grayscale images at odd indices
        plt.imshow(img, cmap='gray')
        plt.title(f"Grayscale (Preprocessed)")
    else:
        plt.imshow(img)
        plt.title(f"Original")
    plt.axis("equal")

plt.tight_layout()
plt.show()

Splitting the dataset

# Split the data into train, validation, and test sets with stratification to maintain class balance.
X_train, X_temp, y_train, y_temp = train_test_split(np.array(images),labels , test_size=0.3, random_state=42, stratify=labels)

# Further split the temp set equally into validation and test sets, again with stratification.
X_val, X_test, y_val, y_test = train_test_split(X_temp,y_temp , test_size=0.5, random_state=42,stratify=y_temp)

# Print the shapes of the resulting splits to verify the split sizes.
print(X_train.shape,y_train.shape)
print(X_val.shape,y_val.shape)
print(X_test.shape,y_test.shape)

(441, 200, 200, 3) (441, 1)
(95, 200, 200, 3) (95, 1)
(95, 200, 200, 3) (95, 1)

Observations:

1. Robust Splitting Strategy (70-15-15 Split)

The code implements a standard and effective 3-way split for machine learning:

• Training Set: 441 images (~70%) — Used to teach the model.

• Validation Set: 95 images (~15%) — Used to tune hyperparameters and check for overfitting during training.

• Test Set: 95 images (~15%) — Used for the final unbiased evaluation.

2. Data Integrity Check

• The shapes printed in the output (441, 95, 95) sum up exactly to 631, which confirms that no data was lost or mishandled during the splitting process.

Data Normalization

# Normalizing the image pixels
X_train_normalized = X_train.astype('float32')/255.0
X_val_normalized = X_val.astype('float32')/255.0
X_test_normalized = X_test.astype('float32')/255.0

Observations:

1. Data Normalization

• Rescaling: The code rescales the pixel intensity values from the range [0, 255] to [0, 1] by dividing by 255.0.

• Why this is important: Neural networks converge much faster and more stably when input values are small (between 0 and 1). Large integer inputs (like 255) can lead to unstable gradients, making the model hard to train.

• Precision: Converting to float32 is the standard for Deep Learning. It offers sufficient precision for training while using half the memory of the default float64, which is important for GPU performance.

Model Building

Model Evaluation Criterion

Utility Functions

# defining a function to compute different metrics to check performance of a classification model built using statsmodels
def model_performance_classification(model, predictors, target):
    """
    Function to compute different metrics to check classification model performance

    model: classifier
    predictors: independent variables
    target: dependent variable
    """

    # checking which probabilities are greater than threshold
    pred = model.predict(predictors).reshape(-1)>0.5

    target = target.to_numpy().reshape(-1)


    acc = accuracy_score(target, pred)  # to compute Accuracy
    recall = recall_score(target, pred, average='weighted')  # to compute Recall
    precision = precision_score(target, pred, average='weighted')  # to compute Precision
    f1 = f1_score(target, pred, average='weighted')  # to compute F1-score

    # creating a dataframe of metrics
    df_perf = pd.DataFrame({"Accuracy": acc, "Recall": recall, "Precision": precision, "F1 Score": f1,},index=[0],)

    return df_perf

def plot_confusion_matrix(model,predictors,target,ml=False):
    """
    Function to plot the confusion matrix

    model: classifier
    predictors: independent variables
    target: dependent variable
    ml: To specify if the model used is an sklearn ML model or not (True means ML model)
    """

    # checking which probabilities are greater than threshold
    pred = model.predict(predictors).reshape(-1)>0.5

    target = target.to_numpy().reshape(-1)

    # Plotting the Confusion Matrix using confusion matrix() function which is also predefined tensorflow module
    confusion_matrix = tf.math.confusion_matrix(target,pred)
    f, ax = plt.subplots(figsize=(10, 8))
    sns.heatmap(
        confusion_matrix,
        annot=True,
        linewidths=.4,
        fmt="d",
        square=True,
        ax=ax
    )
    plt.show()

Model 1: Simple Convolutional Neural Network (CNN)

# Initializing Model
model_1 = Sequential()

# Convolutional layers
# 🔹 Use your actual image size here, e.g. (128, 128, 3) or (64, 64, 3)
model_1.add(Conv2D(32, (3, 3), activation='relu', padding="same", input_shape=(200, 200, 3)))
model_1.add(MaxPooling2D((4, 4), padding='same'))

model_1.add(Conv2D(64, (3, 3), activation='relu', padding="same"))   # 64 filters, 3x3 kernel, ReLU
model_1.add(MaxPooling2D((2, 2), padding='same'))                    # 2x2 pooling

model_1.add(Conv2D(128, (3, 3), activation='relu', padding="same"))  # 128 filters, 3x3 kernel, ReLU

# Flatten and Dense layers
model_1.add(Flatten())
model_1.add(Dense(4, activation='relu'))

# For binary classification → 1 neuron with sigmoid
model_1.add(Dense(1, activation='sigmoid'))  # output layer

# Compile with Adam Optimizer
opt = Adam(learning_rate=0.001)  # typical starting learning rate

model_1.compile(optimizer=opt, loss='binary_crossentropy', metrics=["accuracy", "Precision"])

# Summary
model_1.summary()

Model: "sequential"

┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┓
┃ Layer (type)                    ┃ Output Shape           ┃       Param # ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━┩
│ conv2d (Conv2D)                 │ (None, 200, 200, 32)   │           896 │
├─────────────────────────────────┼────────────────────────┼───────────────┤
│ max_pooling2d (MaxPooling2D)    │ (None, 50, 50, 32)     │             0 │
├─────────────────────────────────┼────────────────────────┼───────────────┤
│ conv2d_1 (Conv2D)               │ (None, 50, 50, 64)     │        18,496 │
├─────────────────────────────────┼────────────────────────┼───────────────┤
│ max_pooling2d_1 (MaxPooling2D)  │ (None, 25, 25, 64)     │             0 │
├─────────────────────────────────┼────────────────────────┼───────────────┤
│ conv2d_2 (Conv2D)               │ (None, 25, 25, 128)    │        73,856 │
├─────────────────────────────────┼────────────────────────┼───────────────┤
│ flatten (Flatten)               │ (None, 80000)          │             0 │
├─────────────────────────────────┼────────────────────────┼───────────────┤
│ dense (Dense)                   │ (None, 4)              │       320,004 │
├─────────────────────────────────┼────────────────────────┼───────────────┤
│ dense_1 (Dense)                 │ (None, 1)              │             5 │
└─────────────────────────────────┴────────────────────────┴───────────────┘

 Total params: 413,257 (1.58 MB)

 Trainable params: 413,257 (1.58 MB)

 Non-trainable params: 0 (0.00 B)

history_1 = model_1.fit(
    X_train_normalized, y_train,
    epochs=20,              # number of epochs
    validation_data=(X_val_normalized, y_val),
    shuffle=True,
    batch_size=128,          # batch size
    verbose=2
)

Epoch 1/20
4/4 - 21s - 5s/step - Precision: 0.4627 - accuracy: 0.4649 - loss: 0.6986 - val_Precision: 0.0000e+00 - val_accuracy: 0.5053 - val_loss: 0.7471
Epoch 2/20
4/4 - 17s - 4s/step - Precision: 0.0000e+00 - accuracy: 0.5079 - loss: 0.6745 - val_Precision: 0.0000e+00 - val_accuracy: 0.5053 - val_loss: 0.6903
Epoch 3/20
4/4 - 18s - 4s/step - Precision: 0.9730 - accuracy: 0.6667 - loss: 0.6274 - val_Precision: 0.6528 - val_accuracy: 0.7368 - val_loss: 0.6177
Epoch 4/20
4/4 - 19s - 5s/step - Precision: 0.8366 - accuracy: 0.9002 - loss: 0.4113 - val_Precision: 1.0000 - val_accuracy: 0.9579 - val_loss: 0.3200
Epoch 5/20
4/4 - 17s - 4s/step - Precision: 0.9851 - accuracy: 0.9501 - loss: 0.2409 - val_Precision: 0.9592 - val_accuracy: 0.9789 - val_loss: 0.1130
Epoch 6/20
4/4 - 21s - 5s/step - Precision: 0.9773 - accuracy: 0.9841 - loss: 0.0642 - val_Precision: 0.9792 - val_accuracy: 0.9895 - val_loss: 0.0334
Epoch 7/20
4/4 - 18s - 5s/step - Precision: 0.9907 - accuracy: 0.9887 - loss: 0.0428 - val_Precision: 0.9792 - val_accuracy: 0.9895 - val_loss: 0.0469
Epoch 8/20
4/4 - 22s - 5s/step - Precision: 1.0000 - accuracy: 0.9864 - loss: 0.0372 - val_Precision: 0.9792 - val_accuracy: 0.9895 - val_loss: 0.0766
Epoch 9/20
4/4 - 17s - 4s/step - Precision: 0.9819 - accuracy: 0.9909 - loss: 0.0147 - val_Precision: 0.9792 - val_accuracy: 0.9895 - val_loss: 0.0658
Epoch 10/20
4/4 - 17s - 4s/step - Precision: 1.0000 - accuracy: 0.9955 - loss: 0.0087 - val_Precision: 0.9792 - val_accuracy: 0.9895 - val_loss: 0.0465
Epoch 11/20
4/4 - 21s - 5s/step - Precision: 0.9954 - accuracy: 0.9955 - loss: 0.0127 - val_Precision: 0.9792 - val_accuracy: 0.9895 - val_loss: 0.1730
Epoch 12/20
4/4 - 18s - 4s/step - Precision: 1.0000 - accuracy: 1.0000 - loss: 0.0019 - val_Precision: 0.9792 - val_accuracy: 0.9895 - val_loss: 0.0510
Epoch 13/20
4/4 - 17s - 4s/step - Precision: 1.0000 - accuracy: 1.0000 - loss: 0.0025 - val_Precision: 0.9792 - val_accuracy: 0.9895 - val_loss: 0.0348
Epoch 14/20
4/4 - 18s - 4s/step - Precision: 1.0000 - accuracy: 1.0000 - loss: 0.0021 - val_Precision: 0.9792 - val_accuracy: 0.9895 - val_loss: 0.0880
Epoch 15/20
4/4 - 20s - 5s/step - Precision: 1.0000 - accuracy: 1.0000 - loss: 0.0018 - val_Precision: 0.9792 - val_accuracy: 0.9895 - val_loss: 0.1079
Epoch 16/20
4/4 - 29s - 7s/step - Precision: 1.0000 - accuracy: 1.0000 - loss: 0.0012 - val_Precision: 0.9792 - val_accuracy: 0.9895 - val_loss: 0.0915
Epoch 17/20
4/4 - 31s - 8s/step - Precision: 1.0000 - accuracy: 1.0000 - loss: 1.7766e-04 - val_Precision: 0.9792 - val_accuracy: 0.9895 - val_loss: 0.0768
Epoch 18/20
4/4 - 18s - 4s/step - Precision: 1.0000 - accuracy: 1.0000 - loss: 8.6354e-05 - val_Precision: 0.9792 - val_accuracy: 0.9895 - val_loss: 0.0669
Epoch 19/20
4/4 - 17s - 4s/step - Precision: 1.0000 - accuracy: 1.0000 - loss: 1.2064e-04 - val_Precision: 0.9792 - val_accuracy: 0.9895 - val_loss: 0.0631
Epoch 20/20
4/4 - 22s - 6s/step - Precision: 1.0000 - accuracy: 1.0000 - loss: 1.7029e-04 - val_Precision: 0.9792 - val_accuracy: 0.9895 - val_loss: 0.0658

plt.plot(history_1.history['accuracy'])
plt.plot(history_1.history['val_accuracy'])
plt.title('Model Accuracy')
plt.ylabel('Accuracy')
plt.xlabel('Epoch')
plt.legend(['Train', 'Validation'], loc='upper left')
plt.show()

model_1_train_perf = model_performance_classification(model_1, X_train_normalized,y_train)

print("Train performance metrics")
print(model_1_train_perf)

14/14 ━━━━━━━━━━━━━━━━━━━━ 12s 856ms/step
Train performance metrics
   Accuracy  Recall  Precision  F1 Score
0       1.0     1.0        1.0       1.0

plot_confusion_matrix(model_1,X_train_normalized,y_train)

14/14 ━━━━━━━━━━━━━━━━━━━━ 5s 326ms/step

model_1_valid_perf = model_performance_classification(model_1, X_val_normalized,y_val)

print("Validation performance metrics")
print(model_1_valid_perf)

3/3 ━━━━━━━━━━━━━━━━━━━━ 2s 511ms/step
Validation performance metrics
   Accuracy    Recall  Precision  F1 Score
0  0.989474  0.989474   0.989693  0.989474

plot_confusion_matrix(model_1,X_val_normalized,y_val)

3/3 ━━━━━━━━━━━━━━━━━━━━ 1s 328ms/step

Vizualizing the predictions

# The indices you want to inspect
indices = [12, 21, 33, 41]

plt.figure(figsize=(8, 8))

for i, idx in enumerate(indices):
    # Plot image
    plt.subplot(2, 2, i + 1)
    plt.imshow(X_val[idx])
    plt.axis("equal")

    # Make prediction for the corresponding index
    pred = model_1.predict(X_val_normalized[idx].reshape(1, 200, 200, 3))[0][0]
    predicted_label = 1 if pred > 0.5 else 0

    # True label
    true_label = y_val.iloc[idx]

    # Set title with prediction + truth
    plt.title(f"Index {idx}\nPred: {predicted_label} | True: {true_label}")

plt.tight_layout()
plt.show()

1/1 ━━━━━━━━━━━━━━━━━━━━ 0s 47ms/step
1/1 ━━━━━━━━━━━━━━━━━━━━ 0s 46ms/step
1/1 ━━━━━━━━━━━━━━━━━━━━ 0s 50ms/step
1/1 ━━━━━━━━━━━━━━━━━━━━ 0s 46ms/step

Observations:

1. What I’m seeing in my CNN architecture:

• Input: (200, 200, 3) RGB images.

• Backbone: 3 conv blocks (32 → 64 → 128 filters, 3×3, ReLU, padding="same").

• Pooling choice: my first MaxPool is (4×4), which is a very aggressive downsample:

200×200 → 50×50 immediately. This can work, but it can also throw away fine details early.

• Big flatten: after the last conv I have 25×25×128 = 80,000 features → Flatten().

• This creates a classic “CNN → huge vector → dense” pattern that often overfits on small datasets.

• Dense head is tiny: Dense(4) then Dense(1) sigmoid.

• Interestingly, most of my parameters are in that Dense(4) because of the huge flatten:

  • Dense(4) params ≈ 80,000×4 + 4 = 320,004

  • Total params ≈ 413,257

So even though the head looks small, the model still has a lot of capacity.

2. Training behavior (from my logs/plots)

• Very fast convergence: by around epoch 4–6 my validation accuracy is already ~0.98–0.99, and then it stays flat.

• Training becomes perfect: my training accuracy reaches 1.0, my training loss goes extremely close to 0, and my training confusion matrix is perfect (224 TN, 217 TP, 0 FP, 0 FN).

• Validation is almost perfect: my validation confusion matrix shows 47 TN, 47 TP, 1 FP, 0 FN.

So my only error is one false positive.

This matches my validation precision/recall being ~0.989.

3. Important “sanity” observations (potential red flags)

• My dataset is small (based on the confusion matrices):

  • Train size = 224 + 217 = 441

  • Val size = 47 + 1 + 47 = 95, With image models, results can look amazing on small splits even when real-world generalization is weaker.

• Epoch 1–2 behavior: my val_precision = 0 while my val_accuracy ≈ 0.505.

That usually happens when my model predicts almost everything as the negative class early on (or produces no positive predictions at a 0.5 threshold).

• Then it “snaps” into a good solution after a couple epochs.

• Possible leakage / overly-easy split: getting 1.0 training accuracy with only 1 validation mistake is great, but it’s also a pattern I might see when:

  • near-duplicates exist across my train/val sets,

  • my split wasn’t stratified/grouped properly,

  • some preprocessing leaks label information,

  • or the task is simply extremely easy (strong visual shortcuts).

Model 2: (VGG-16 (Base))

# Standard VGG16 definition
vgg_model = VGG16(weights='imagenet', include_top=False, input_shape=(200, 200, 3))

vgg_model.summary()

Downloading data from https://storage.googleapis.com/tensorflow/keras-applications/vgg16/vgg16_weights_tf_dim_ordering_tf_kernels_notop.h5
58889256/58889256 ━━━━━━━━━━━━━━━━━━━━ 1s 0us/step

Model: "vgg16"

┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┓
┃ Layer (type)                    ┃ Output Shape           ┃       Param # ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━┩
│ input_layer_1 (InputLayer)      │ (None, 200, 200, 3)    │             0 │
├─────────────────────────────────┼────────────────────────┼───────────────┤
│ block1_conv1 (Conv2D)           │ (None, 200, 200, 64)   │         1,792 │
├─────────────────────────────────┼────────────────────────┼───────────────┤
│ block1_conv2 (Conv2D)           │ (None, 200, 200, 64)   │        36,928 │
├─────────────────────────────────┼────────────────────────┼───────────────┤
│ block1_pool (MaxPooling2D)      │ (None, 100, 100, 64)   │             0 │
├─────────────────────────────────┼────────────────────────┼───────────────┤
│ block2_conv1 (Conv2D)           │ (None, 100, 100, 128)  │        73,856 │
├─────────────────────────────────┼────────────────────────┼───────────────┤
│ block2_conv2 (Conv2D)           │ (None, 100, 100, 128)  │       147,584 │
├─────────────────────────────────┼────────────────────────┼───────────────┤
│ block2_pool (MaxPooling2D)      │ (None, 50, 50, 128)    │             0 │
├─────────────────────────────────┼────────────────────────┼───────────────┤
│ block3_conv1 (Conv2D)           │ (None, 50, 50, 256)    │       295,168 │
├─────────────────────────────────┼────────────────────────┼───────────────┤
│ block3_conv2 (Conv2D)           │ (None, 50, 50, 256)    │       590,080 │
├─────────────────────────────────┼────────────────────────┼───────────────┤
│ block3_conv3 (Conv2D)           │ (None, 50, 50, 256)    │       590,080 │
├─────────────────────────────────┼────────────────────────┼───────────────┤
│ block3_pool (MaxPooling2D)      │ (None, 25, 25, 256)    │             0 │
├─────────────────────────────────┼────────────────────────┼───────────────┤
│ block4_conv1 (Conv2D)           │ (None, 25, 25, 512)    │     1,180,160 │
├─────────────────────────────────┼────────────────────────┼───────────────┤
│ block4_conv2 (Conv2D)           │ (None, 25, 25, 512)    │     2,359,808 │
├─────────────────────────────────┼────────────────────────┼───────────────┤
│ block4_conv3 (Conv2D)           │ (None, 25, 25, 512)    │     2,359,808 │
├─────────────────────────────────┼────────────────────────┼───────────────┤
│ block4_pool (MaxPooling2D)      │ (None, 12, 12, 512)    │             0 │
├─────────────────────────────────┼────────────────────────┼───────────────┤
│ block5_conv1 (Conv2D)           │ (None, 12, 12, 512)    │     2,359,808 │
├─────────────────────────────────┼────────────────────────┼───────────────┤
│ block5_conv2 (Conv2D)           │ (None, 12, 12, 512)    │     2,359,808 │
├─────────────────────────────────┼────────────────────────┼───────────────┤
│ block5_conv3 (Conv2D)           │ (None, 12, 12, 512)    │     2,359,808 │
├─────────────────────────────────┼────────────────────────┼───────────────┤
│ block5_pool (MaxPooling2D)      │ (None, 6, 6, 512)      │             0 │
└─────────────────────────────────┴────────────────────────┴───────────────┘

 Total params: 14,714,688 (56.13 MB)

 Trainable params: 14,714,688 (56.13 MB)

 Non-trainable params: 0 (0.00 B)

# Making all the layers of the VGG model non-trainable. i.e. freezing them
for layer in vgg_model.layers:
    layer.trainable = False

model_2 = Sequential()

# Adding the convolutional part of the VGG16 model from above
model_2.add(vgg_model)

# Flattening the output of the VGG16 model
model_2.add(Flatten())

# Adding a dense output layer
model_2.add(Dense(1, activation='sigmoid'))

opt = Adam(learning_rate=0.0001) # 1. Define the learning rate

# Compile model
model_2.compile(optimizer=opt, loss=keras.losses.BinaryCrossentropy(), metrics=["accuracy"]) # 2. Define the metric

# Generating the summary of the model
model_2.summary()

Model: "sequential_1"

┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┓
┃ Layer (type)                    ┃ Output Shape           ┃       Param # ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━┩
│ vgg16 (Functional)              │ (None, 6, 6, 512)      │    14,714,688 │
├─────────────────────────────────┼────────────────────────┼───────────────┤
│ flatten_1 (Flatten)             │ (None, 18432)          │             0 │
├─────────────────────────────────┼────────────────────────┼───────────────┤
│ dense_2 (Dense)                 │ (None, 1)              │        18,433 │
└─────────────────────────────────┴────────────────────────┴───────────────┘

 Total params: 14,733,121 (56.20 MB)

 Trainable params: 18,433 (72.00 KB)

 Non-trainable params: 14,714,688 (56.13 MB)

train_datagen = ImageDataGenerator()

# Epochs
epochs = 20
# Batch size
batch_size = 128

history_2 = model_2.fit(train_datagen.flow(X_train_normalized,y_train,
                                      batch_size=batch_size,
                                      seed=42,
                                      shuffle=False),
                    epochs=epochs,
                    steps_per_epoch=X_train_normalized.shape[0] // batch_size,
                    validation_data=(X_val_normalized,y_val),
                    verbose=1)

Epoch 1/20
3/3 ━━━━━━━━━━━━━━━━━━━━ 217s 77s/step - accuracy: 0.6094 - loss: 0.6528 - val_accuracy: 0.7895 - val_loss: 0.5805
Epoch 2/20
3/3 ━━━━━━━━━━━━━━━━━━━━ 107s 41s/step - accuracy: 0.8421 - loss: 0.5498 - val_accuracy: 0.8421 - val_loss: 0.5525
Epoch 3/20
3/3 ━━━━━━━━━━━━━━━━━━━━ 230s 83s/step - accuracy: 0.8663 - loss: 0.5282 - val_accuracy: 0.9263 - val_loss: 0.4740
Epoch 4/20
3/3 ━━━━━━━━━━━━━━━━━━━━ 98s 20s/step - accuracy: 0.9453 - loss: 0.4617 - val_accuracy: 0.9474 - val_loss: 0.4495
Epoch 5/20
3/3 ━━━━━━━━━━━━━━━━━━━━ 235s 60s/step - accuracy: 0.9664 - loss: 0.4197 - val_accuracy: 0.9789 - val_loss: 0.3839
Epoch 6/20
3/3 ━━━━━━━━━━━━━━━━━━━━ 139s 41s/step - accuracy: 0.9844 - loss: 0.3694 - val_accuracy: 0.9789 - val_loss: 0.3642
Epoch 7/20
3/3 ━━━━━━━━━━━━━━━━━━━━ 182s 60s/step - accuracy: 0.9929 - loss: 0.3429 - val_accuracy: 1.0000 - val_loss: 0.3126
Epoch 8/20
3/3 ━━━━━━━━━━━━━━━━━━━━ 138s 41s/step - accuracy: 0.9922 - loss: 0.2939 - val_accuracy: 1.0000 - val_loss: 0.2976
Epoch 9/20
3/3 ━━━━━━━━━━━━━━━━━━━━ 176s 76s/step - accuracy: 0.9955 - loss: 0.2666 - val_accuracy: 1.0000 - val_loss: 0.2582
Epoch 10/20
3/3 ━━━━━━━━━━━━━━━━━━━━ 135s 41s/step - accuracy: 1.0000 - loss: 0.2415 - val_accuracy: 1.0000 - val_loss: 0.2466
Epoch 11/20
3/3 ━━━━━━━━━━━━━━━━━━━━ 220s 81s/step - accuracy: 1.0000 - loss: 0.2208 - val_accuracy: 1.0000 - val_loss: 0.2161
Epoch 12/20
3/3 ━━━━━━━━━━━━━━━━━━━━ 98s 20s/step - accuracy: 1.0000 - loss: 0.2062 - val_accuracy: 1.0000 - val_loss: 0.2072
Epoch 13/20
3/3 ━━━━━━━━━━━━━━━━━━━━ 205s 76s/step - accuracy: 1.0000 - loss: 0.1884 - val_accuracy: 1.0000 - val_loss: 0.1832
Epoch 14/20
3/3 ━━━━━━━━━━━━━━━━━━━━ 112s 41s/step - accuracy: 1.0000 - loss: 0.1552 - val_accuracy: 1.0000 - val_loss: 0.1761
Epoch 15/20
3/3 ━━━━━━━━━━━━━━━━━━━━ 178s 61s/step - accuracy: 1.0000 - loss: 0.1537 - val_accuracy: 1.0000 - val_loss: 0.1576
Epoch 16/20
3/3 ━━━━━━━━━━━━━━━━━━━━ 96s 20s/step - accuracy: 1.0000 - loss: 0.1416 - val_accuracy: 1.0000 - val_loss: 0.1520
Epoch 17/20
3/3 ━━━━━━━━━━━━━━━━━━━━ 174s 75s/step - accuracy: 1.0000 - loss: 0.1277 - val_accuracy: 1.0000 - val_loss: 0.1375
Epoch 18/20
3/3 ━━━━━━━━━━━━━━━━━━━━ 106s 24s/step - accuracy: 1.0000 - loss: 0.1275 - val_accuracy: 1.0000 - val_loss: 0.1332
Epoch 19/20
3/3 ━━━━━━━━━━━━━━━━━━━━ 221s 60s/step - accuracy: 1.0000 - loss: 0.1174 - val_accuracy: 1.0000 - val_loss: 0.1217
Epoch 20/20
3/3 ━━━━━━━━━━━━━━━━━━━━ 98s 20s/step - accuracy: 1.0000 - loss: 0.1057 - val_accuracy: 1.0000 - val_loss: 0.1183

plt.plot(history_2.history['accuracy'])
plt.plot(history_2.history['val_accuracy'])
plt.title('Model Accuracy')
plt.ylabel('Accuracy')
plt.xlabel('Epoch')
plt.legend(['Train', 'Validation'], loc='upper left')
plt.show()

model_2_train_perf = model_performance_classification(model_2,X_train_normalized,y_train)

print("Train performance metrics")
print(model_2_train_perf)

14/14 ━━━━━━━━━━━━━━━━━━━━ 204s 14s/step
Train performance metrics
   Accuracy  Recall  Precision  F1 Score
0       1.0     1.0        1.0       1.0

plot_confusion_matrix(model_2,X_train_normalized,y_train)

14/14 ━━━━━━━━━━━━━━━━━━━━ 191s 14s/step

model_2_valid_perf = model_performance_classification(model_2, X_val_normalized,y_val)

print("Validation performance metrics")
print(model_2_valid_perf)

3/3 ━━━━━━━━━━━━━━━━━━━━ 44s 15s/step
Validation performance metrics
   Accuracy  Recall  Precision  F1 Score
0       1.0     1.0        1.0       1.0

plot_confusion_matrix(model_2,X_val_normalized,y_val)

3/3 ━━━━━━━━━━━━━━━━━━━━ 41s 14s/step

Visualizing the prediction:

# The indices you want to inspect
indices = [2, 15, 24, 36]

plt.figure(figsize=(8, 8))

for i, idx in enumerate(indices):
    # Plot image
    plt.subplot(2, 2, i + 1)
    plt.imshow(X_val[idx])
    plt.axis("equal")

    # Make prediction for the corresponding index
    pred = model_2.predict(X_val_normalized[idx].reshape(1, 200, 200, 3))[0][0]
    predicted_label = 1 if pred > 0.5 else 0

    # True label
    true_label = y_val.iloc[idx]

    # Set title with prediction + truth
    plt.title(f"Index {idx}\nPred: {predicted_label} | True: {true_label}")

plt.tight_layout()
plt.show()

1/1 ━━━━━━━━━━━━━━━━━━━━ 0s 401ms/step
1/1 ━━━━━━━━━━━━━━━━━━━━ 0s 415ms/step
1/1 ━━━━━━━━━━━━━━━━━━━━ 0s 404ms/step
1/1 ━━━━━━━━━━━━━━━━━━━━ 0s 395ms/step