Activity # 10 – Image Enhancement by Histogram Manipulation

Though digital cameras incorporate different features to imitate the function and response of human eyes to light and color, digital cameras are able to overcome some of its limitations. The sensors of digital cameras are able to pick up information that our eyes are not sensitive enough to detect, which leads us to..

– Ma’am Jing.

Images can contain information that we can uncover by manipulating its image histogram. For this activity, we’ll mainly be using these images:

Fig 1. Test images a and b.

Fig 1(a) was taken outside at night with a few hanging light sources in the corners of the room, so naturally there are a lot of dark areas and shadows in the image. Most of the people in the image were also wearing dark-colored clothing, resulting to darkened areas in our image. Fig 1(b) is an image of a child behind an illuminated pumpkin head in a very dark room during Halloween.

The first step is to obtain their gray level probability distribution function (PDF), which is the same as its image histogram normalized by the total number of pixels. Using the imhist() function in Scilab, we have the PDF of our images:

Fig 2. Probability distribution functions of a and b, respectively.

This PDF shows how many pixels have a corresponding gray level value. As you can see, majority of the pixels in a have pixel values in the range of 10-50, which can be considered as “dark” pixels. Similarly, but more obviously, the PDF of b reflects the original image in which most of the pixels in the image have very low pixel values.

For the second step, if the gray levels r of our image has a PDF p(r), then its cumulative distribution function (CDF) T(r) is given by

where g is a dummy variable. This CDF gives us the probability that g will take a value less than or equal to r. We can obtain the CDF of our image from the PDF using the cumsum() function in Scilab:

Fig 3. Cumulative distribution functions of a and b

Since the CDF is the cumulative sum of the frequency at its previous values, they are increasing with the gray level values r.

Now, what does this all have to do with enhancing our image? Well, if we can change our CDF to something that decreases the probability of finding dark pixels, then we may be able to uncover some of the hidden details in our image. We can do this by mapping our r values to a new set of gray level values z according to a desired CDF G(z), such that

If you’re confused, this image clearly summarizes how this works:

Fig 4. Summary of steps to enhance an image by histogram manipulation.

For each pixel in our image, we have to get its gray level value r (1), evaluate T(r) (2), match T(r) with G(z) (3), and finally get the new z value corresponding to the value of  G(z) (4).

I. Linear CDF

Fig 5. Linear function as desired CDF

To try it out, let our desired CDF be a linear function as shown in Fig 5. The coefficient is configured to match the gray level range of 0 to 255. Here is a Scilab code implementation of the procedures we outlined:

Fig 6. Scilab code for Histogram manipulation with linear G(z)

Lines 6 generates the image histogram of our images using the imhist() function, with the output containing the number of pixels and the bins. We can obtain the y-values of the PDF in line 7 by dividing the pixel count per gray level value by the total amount of pixels. Lastly, line 8 generates the CDF of our gray scale image using the calculated PDF. Lines 10-12 define the range and y-values of the desired CDF T(r) which is the linear function previously given. Line 14 calculates the values of the original CDF per pixel with an increment of 1 since the indexing starts at 1 instead of 0. Line 15 interpolates the new z values by given the desired CDF in the range of values in Tr  matrix. This corresponds to steps 3 and 4 in our outline. We can now compare the original grayscale images to their enhanced versions by histogram manipulation:

Fig 7. Original gray scale images (top row) and their enhanced versions (bottom row) using a linear function as the desired CDF

Immediately we can see that most the brightness of the images were increased to varying degrees. For a, we can see the manifestation of the histogram manipulation on the brightness of their clothes. You can more clearly see the creases and folds in the shirt of the guy in the leftmost part as well as the pants of the tallest guy in the picture. One good example of uncovering “hidden” details in a is the appearance of the circular necklace worn by the girl with the blue top. Did you notice that before? On the other hand, the effect is quite obvious for b. Despite the fuzzy image (this is a picture from a smart phone), you can now clearly see the face of the child, even his hairdo! Also, would you have known that there was a picture frame in the background if it wasn’t for our histogram manipulation? Cool, huh?

However, you can see a drawback in b, where the brightly lit portions of the image became brighter in the enhanced version, reducing the clarity of the previously observed detail. This suggests that our technique does not discriminate between dark and bright regions, as it tries to increase the pixel value of all pixels in the image.

Before we move on, let’s also investigate the CDF of our new histogram equalized images:

Fig 8. Original CDF of a and b (top row) and the CDF of the histogram equalized images (bottom row)

For a, we see that the modified CDF is almost linear, with the exception of a few small ripples near the limits of the function. This is the result most likely because the original CDF also resembles a linear curve aside from the bump in the lower pixel values. Meanwhile, the modified CDF of b showcases step-like features for pixel values from 0 up to around 210, before settling to a linear line. Though this may not look like a linear function, you can easily fit a linear trend line through the plot similar to the desired linear CDF.

II. Nonlinear CDF: Sigmoid Function

Now that we have achieved our initial goal, we can go further by trying out a new function for our CDF: the sigmoid function

Relax! It’s just a nonlinear function similar to the response of human eyes. The sigmoid function is given by

and it looks like this:

Fig 9. Sigmoid function

So what do we do now? Well, we can actually just replace the part of the code that specifies the desired CDF by declaring a sigmoid function. Note however that the given equation works for a curve that is centered at x = 0 with a range from -10 to 10. So we need to tweak the parameters in the equation by adding a translation term to the input variable which can then be normalized. Here’s what I got:

Fig 10. Original Gray scale images of a and b (top row) and the histogram manipulated images (bottom row) using a sigmoid function as the desired CDF.

Both of the modified images are somewhat “hazy”, like there is a gray overlay on both images. In terms of detail, the result in a is similar to that of the linear CDF, however you can notice that there are some parts of the image that looks distorted at corrupted, such as the left most light source as well as a few parts in the right side of the picture. This may be due to problems in the interpolation process. For b, you can also uncover the face of the child as well as the picture frame in the background, but there is a sense of “control” in the image. The details in the face of the pumpkin head can also be resolved more clearly as compared to the results from the linear CDF.

III. Histogram Manipulation on an RGB image

Now that we have tried different CDFs in manipulating our gray scale images, we can now proceed on trying this technique on RGB images. Histogram manipulation on RGB images differ from that of grayscale images since the image has three color channels. Instead of directly applying our technique on each color channel, we will first need to convert the color channels from RGB to the rgI color space (or the Normalized chromaticity coordinates NCC). You can see my blog post on Activity 7 for more details regarding this. Upon conversion, we can now apply our technique on the I channel, which corresponds to the intensity or brightness of the image.

The details of the technique are quite similar, the only differences being that the maximum value of the channel amounts to 765 and that after mapping, we need to retrieve the modified RGB channels using the modified I channel. I opted to use the linear function as the desired CDF since it was able to modify the image cleanly without errors while retaining the details. Applying this to our two original images, we get:

Fig 11. Original images (top row) and their histogram manipulated version (bottom row) using a linear function as the desired CDF

The general quality of the result is similar to that in the grayscale versions. For a, a slight increase in brightness was noticeable as the modified image looks the same as the original. However, you can see the hidden details in the image that we have previously uncovered in Fig. 7. As for b, I was a bit surprised on the color of the modified image since there is a hint of blue in the background which was not present before. Also, the colors are spread out, such as the lighting from the features of the pumpkin head. In terms of function, we were still able to recover the hidden details in the image as before but with more details such as how the face is illuminated by the pumpkin.

IV. Conclusion

At first, I though that this activity would be quite easy, since the technique used was mostly mechanical. However, it took me a long time to get the right outputs due to my use of for loops that resulted to VERY LONG runtime for a single image. Combined with faulty handling of the data types and normalization of matrices, it almost came to the point of giving up. Which is why I have Roland, Harold and Angelo to thank for providing suggestions and tweaks for the code. It was through Harold that I learned how to perform the technique using matrix operations, which greatly reduced the program running time.

In the end, I was able to do the activity, and I was actually amazed and proud of the results that I was able to produce. I’m also proud that I was able to avoid giving up on this. All in all, I would give myself an 11/10 for this activity since there is certainly room for improvement in how I can do the activity.