In this article, we're about to dive into the fascinating world of image filters or kernels. We'll uncover how these mathematical constructs can work wonders in blurring, sharpening, outlining, and even adding an embossed effect to the features within an image—all through the power of mathematics and coding.
Let's kick things off.
To get started, we import essential libraries including numpy and matplotlib. Moreover, we bring in certain functions from the skimage and scipy.signal library to aid in our exploration.
import numpy as np
import matplotlib.pyplot as plt
from skimage.io import imread, imshow
from skimage.color import rgb2gray
from skimage.transform import rescale
from scipy.signal import convolve2d
However, before we proceed, let's clarify the concept of a filter or kernel.
In essence, these matrices are utilized to enact various image effects upon an image. This achievement is made possible through a mathematical process called convolution. This procedure involves summing up each pixel's value in the image while considering its neighboring pixels, and this sum is determined by the values within the kernel. To paint a clearer picture of this concept, let's provide an illustrative example.
The resulting image's pixel values are derived by the multiplication of each kernel value with its corresponding input image pixel value. This operation continues iteratively until the kernel has traversed the entire input image, performing this multiplication at each step. The following visual representation provides a clearer understanding of this process:
With a grasp of the fundamentals behind filters and convolution, let's put this theory into action using a real image.
For this practical illustration, we'll employ a photograph featuring an adorable Golden Retriever:
my_dog = imread('pb_doggo.jpg')
plt.imshow(my_dog, cmap='gray');
To ensure that the effects of the filters and kernels are visually evident, let us rescale the image down to 10% of its original size.
r_scaled = rescale(my_dog[:,:,0], 0.10)
g_scaled = rescale(my_dog[:,:,1], 0.10)
b_scaled = rescale(my_dog[:,:,2], 0.10)
my_dog_scaled = np.stack([r_scaled, g_scaled, b_scaled], axis=2)
my_dog_gray = rescale(rgb2gray(my_dog), 0.10)
We have also defined a function that will apply the convolution function in all channels of the image, as shown below:
def rgb_convolve2d(image, kernel):
red = convolve2d(image[:,:,0], kernel, 'valid')
green = convolve2d(image[:,:,1], kernel, 'valid')
blue = convolve2d(image[:,:,2], kernel, 'valid')
return np.stack([red, green, blue], axis=2)
Now, let’s try to apply the identity filter to the image of the dog.
identity = np.array([[0, 0, 0],
[0, 1, 0],
[0, 0, 0]])
conv_im1 = rgb_convolve2d(my_dog_scaled, identity)
fig, ax = plt.subplots(1,2, figsize=(12,5))
ax[0].imshow(identity, cmap='gray')
ax[1].imshow(abs(conv_im1), cmap='gray');
As expected, nothing happens! As the filter’s name suggests, the identity kernel will return the input image itself.
Now, let’s try edge detection filters on the grayscale image of the dog.
# Edge Detection1
kernel1 = np.array([[0, -1, 0],
[-1, 4, -1],
[0, -1, 0]])
# Edge Detection2
kernel2 = np.array([[-1, -1, -1],
[-1, 8, -1],
[-1, -1, -1]])
# Bottom Sobel Filter
kernel3 = np.array([[-1, -2, -1],
[0, 0, 0],
[1, 2, 1]])
# Top Sobel Filter
kernel4 = np.array([[1, 2, 1],
[0, 0, 0],
[-1, -2, -1]])
# Left Sobel Filter
kernel5 = np.array([[1, 0, -1],
[2, 0, -2],
[1, 0, -1]])
# Right Sobel Filter
kernel6 = np.array([[-1, 0, 1],
[-2, 0, 2],
[-1, 0, 1]])
kernels = [kernel1, kernel2, kernel3, kernel4, kernel5, kernel6]
kernel_name = [‘Edge Detection#1’, ‘Edge Detection#2’,
‘Bottom Sobel’, ‘Top Sobel’,
‘Left Sobel’, ‘Right Sobel’]
figure, axis = plt.subplots(2,3, figsize=(12,10))
for kernel, name, ax in zip(kernels, kernel_name, axis.flatten()):
conv_im1 = convolve2d(my_dog_gray,
kernel[::-1, ::-1]).clip(0,1)
ax.imshow(abs(conv_im1), cmap=’gray’)
ax.set_title(name)
Observing the outcomes displayed in the generated images, we can discern that edge detection effectively identifies areas in which there exists a distinct alteration in intensity or color. A higher value within the results denotes a pronounced change, whereas a lower value signifies a more gradual transition. Additionally, the Sobel operators exhibit a parallel functionality to edge detection, albeit with a notable distinction—these operators exhibit sensitivity to particular directions. As an example, the lower Sobel operator accentuates edges in the lower portion of the object, while the upper Sobel operator emphasizes the opposite.
Now, let's venture into experimenting with alternative kinds of kernel operators applied to the initial image featuring our canine companion.
# Sharpen
kernel7 = np.array([[0, -1, 0],
[-1, 5, -1],
[0, -1, 0]])
# Emboss
kernel8 = np.array([[-2, -1, 0],
[-1, 1, 1],
[ 0, 1, 2]])
# Box Blur
kernel9 = (1 / 9.0) * np.array([[1, 1, 1],
[1, 1, 1],
[1, 1, 1]])
# Gaussian Blur 3x3
kernel10 = (1 / 16.0) * np.array([[1, 2, 1],
[2, 4, 2],
[1, 2, 1]])
# Gaussian Blur 5x5
kernel11 = (1 / 256.0) * np.array([[1, 4, 6, 4, 1],
[4, 16, 24, 16, 4],
[6, 24, 36, 24, 6],
[4, 16, 24, 16, 4],
[1, 4, 6, 4, 1]])
# Unsharp masking 5x5
kernel12 = -(1 / 256.0) * np.array([[1, 4, 6, 4, 1],
[4, 16, 24, 16, 4],
[6, 24, -476, 24, 6],
[4, 16, 24, 16, 4],
[1, 4, 6, 4, 1]])
kernels = [kernel7, kernel8, kernel9, kernel10, kernel11, kernel12]
kernel_name = [‘Sharpen’, ‘Emboss’, ‘Box Blur’,
‘3x3 Gaussian Blur’, ‘5x5 Gaussian Blur’,
‘5x5 Unsharp Masking’]
figure, axis = plt.subplots(2,3, figsize=(12,10))
for kernel, name, ax in zip(kernels, kernel_name, axis.flatten()):
conv_im1 = rgb_convolve2d(my_dog_scaled,
kernel[::-1, ::-1]).clip(0,1)
ax.imshow(abs(conv_im1), cmap=’gray’)
ax.set_title(name)
It's safe to assume that most of us are well-acquainted with these filters, given their common presence in smartphone camera applications. Hence, I won't delve into an extensive explanation of the functions of each filter. Instead, I'll shed light on how the sharpening and 5x5 unsharp masking filters remarkably enhanced the image quality derived from the input image. This serves as a testament to the convolutional filters' ability to manipulate the relatively constrained information inherent in the provided images.
It's worth noting that when utilizing scipy's convolve2d function, a crucial point to remember is that the kernel is inverted before applying the convolution process. This piece of information holds significance, particularly for filters with directional specificity, such as the Sobel operator. Consequently, the kernels must be inverted before employing the convolve2d function. An interesting tidbit: If you're familiar with the tensorflow library, it's noteworthy that the convolution operation within that framework doesn't involve kernel inversion before application.
In summary:
We've ventured into the realm of using convolutional filters to preprocess images and attain our intended outcomes. Through the process of convolution, we've harnessed the ability to implement a range of effects on images, including edge detection, blurring, sharpening, and embossing. With this exploration, we've gained insight into the foundational workings behind the operations our smartphones employ to achieve these image transformations.
Comments
Post a Comment