One of the useful applications of eigenvalues and eigenvectors is the dimensionality reduction algorithm called Principal Component Analysis, or PCA for short.

Here we will be applying PCA on an image dataset to perform image compression. We will be using a portion of the Cat and dog face dataset from Kaggle. In particular, we will be using the cat images.

To apply PCA on any dataset you will begin by defining the covariance matrix. After that you will compute the eigenvalues and eigenvectors of this covariance matrix. Each of these eigenvectors will be a principal component. To perform the dimensionality reduction, you will take the 𝑘 principal components associated to the 𝑘 biggest eigenvalues, and transform the original data by projecting it onto the direction of these principal components (eigenvectors).

Load Images

Begin by loading the images and transforming them to black and white.

import os
import cv2

def load_images_from_directory(directory_path):
    images = []
    for filename in os.listdir(directory_path):
        print(filename)
        file_path = os.path.join(directory_path, filename)
        img = cv2.imread(file_path, cv2.IMREAD_GRAYSCALE)
        images.append(img)
    return images

directory_path = '/Users/senthilp/Desktop/CatDog'
imgs = load_images_from_directory(directory_path)

height, width = imgs[0].shape
print(f'\\nYour dataset has {len(imgs)} images of size {height}x{width} pixels\\n')

Your dataset has 55 images of size 64x64 pixels

Go ahead and plot one image to see what they look like. We can use the colormap 'gray' to plot in black and white.

import matplotlib.pyplot as plt
plt.imshow(imgs[15], cmap='gray')

Untitled

When working with images, you can consider each pixel as a variable. In order to apply PCA for dimensionality reduction we will need to flatten each image into a single row vector. We can do this using the reshape function from NumPy.

The resulting array will have 55 rows, one for each image, and 64x64=4096 columns.

imgs_flatten = np.array([im.reshape(-1) for im in imgs])
print(f'imgs_flatten shape: {imgs_flatten.shape}')

imgs_flatten shape: (55, 4096)

Get the covariance matrix

If you consider each pixel (column) as a variable, and each image (rows) as an observation you will have 55 observations of 4096 variables.

In order to get the covariance matrix you first need to center the data by subtracting the mean for each variable (column).

def center_data(Y):
    mean_vector = np.mean(Y, axis=0)
    mean_matrix = Y - mean_vector    
    return mean_matrix

X = center_data(imgs_flatten)
plt.imshow(X[15].reshape(64,64), cmap='gray')

Untitled