Activity 7 Enhancement in the Frequency Domain

In the past few activities we familiarized ourselves with the use and applications of the 2-dimensional Fourier Transform. Along with the basic properties of the 2-dimensional FT,we also learned and observed the effects of imaging with convolutions and correlations. When we think about it, the past few activities were exercises geared to build on our skills for manipulation in the inverse space or frequency domain. In this activity we utilize the skills we learned to do image enhancements in inverse space.

Most of the time, when we do image processing and analysis, it is more appropriate to do manipulations in the frequency domain. For example, unwanted repetitive patterns over an image would be very difficult to remove without loss of important information. Such tasks can be easily handled when by going into the frequency domain. As we have seen in the previous activities, repetitive patterns are just represented by a few points or lines in inverse space and removing them becomes much simpler. In this activity we will be doing similar tasks in enhancing images of fingerprints, paintings, etc.

The first part of this activity is again aimed to further familiarize ourselves with inverse space. Here we compare the FT of different structures, namely: single points, circles, squares, and gaussians. We can see on Figure 1 below shows the Fourier transform of two single pixel dots compared with the Fourier transforms of two circles of different radius. As expected the Ft of the single pixel dots are multiple parallel lines just like a sinusoid in activity 6. On the other hand the FT of the two circles are very similar to the multiplication of the FT of the two dots and that of a circle (airy disk). When we think about it the image of these two circles is just like the convolution of the two dots and a circle of the same radius. Hence in inverse space the result should be the product of their FT, as seen in figure 1. Increasing the radius of the circle simply results to a decrease in the size of the FT pattern, similar to our results in previous activities.

Figure 1. 1st column: Two single pixel dots and its FT.
2nd-5th column: Two circles of varying radius with FT

Figures 2 and 3 below are just like figure 1 except that the objects are squares and gaussians respectively. Again as expected the FT patterns are simply the product of the FT of the two dots and the objects. That is the FT of a square is two perpendicular sinc patterns and for a gaussian the FT is also a gaussian. The behavior of the FT with varying object size is still the same (inversely proportional).

Figure 2. 1st column: Two single pixel dots and its FT.
2nd-5th column: Two squares of varying side length with FT


Figure 3. 1st column: Two single pixel dots and its FT. 2nd-5th column:
Two gaussians of varying spread (standard deviation/sigma) with FT

Next we explore the difference in the FT of an inverted gaussian. Figure 4 shows the gaussian and inverted gaussian patterns with different standard deviations. When we look at their normalized FT we see no difference at all so instead we compare their Real and Imaginary parts.

Figure 4. Double Gaussian pattern of different spreads and their inverted counterparts.

Figure 4 and 5 shows the comparison of the real and imaginary parts of the gaussian and inverted gaussian FT's. First let me say that though the images look like they are just flat gray images there are actually fine detail around the center which can be seen when the images are enlarged. Still even with very close inspection there are hardly any difference with the gaussian and inverted gaussian. But this is actually expected since inverting the gaussian simply involves multiplying by -1 which can be factored out of the transform. This simply results to a switching of the high and low values of the FT Real and Imaginary parts which is, in this case, hardly distinguishable.

Figure 5. Real part of the Fourier transform of the gaussian and inverted gaussian. The large image corresponds to the first gaussian and inverse gaussian pattern (figure 4) and the other three pairs are for the succeeding gaussian patterns.


Figure 6. Imaginary part of the Fourier transform of the gaussian and inverted gaussian. The large image corresponds to the first gaussian and inverse gaussian pattern (figure 4) and the other three pairs are for the succeeding gaussian patterns.

The second part of this activity involves enhancing a fingerprint image using inverse space filtering. We do this by applying a mask to remove the 'unwanted' regions in the FT of the finger print. Then we take the inverse Fourier transform of the masked FT and hope to obtain clearer and sharper fingerprint pattern.

Figure 7 shows the original fingerprint image and its Fourier transform. Looking at the original image we see that aside from the fingerprint a background pattern is present. This background pattern is probably the grain of the paper which only interferes with the fingerprint pattern. We will remove this background by applying a mask on the FT of the image. To create the mask we must first observe the FT pattern and determine which regions most likely contribute to the fingerprint. Once we have determined this region we will create a mask that only retains this part of the FT.

Figure 7. Original fingerprint image and its FT.

We can see from the FT in figure 7 that the center, having a much higher magnitude, contains most of the information of the image. Removing the center would result to losing not only the background but virtually the whole image. We also note that there are also two curved regions in the FT that are brighter than the other areas and most likely contain information on the fingerprint. Using these observations we now create a mask that enhances this regions hoping that it would result to an enhanced fingerprint.

Figure 8 below shows three different filtering masks I used, hoping to enhance the fingerprint. The first mask was made by simply blacking out all unwanted regions in the FT. The second mask was generated by creating two gaussians and rotating them about the center. Finally the last mask is a convolution of 2 gaussians with the first mask. Each of these masks highlights the regions I mentioned above and should produce an enhanced fingerprint. But as I found out using just one of these masks did not enhance the fingerprint enough to be considered a successful enhancement. So after numerous trials and even more errors I ended up using a combination of these three patterns, multiplying them with different scaling factors, to produce a good enough enhancement of the fingerprint (figure 9).

Figure 8. Three different filtering mask used together to
obtain an enhanced fingerprint pattern.

Figure 9 is a comparison of the original fingerprint and the enhanced one. The left image is the original fingerprint which was inverted and cropped. The right image is the resulting enhancement after the left image was filtered with the masks in figure 8. Comparing the two images we first note that in the enhanced fingerprint the background paper grain patter is not visible anymore. Moreover the text on the background have been significantly reduced. The most notable improvement though is that the fingerprint itself is much clearer. The lines are thinner and sharper, the overall fingerprint is also more uniform in color, and some lines are now distinguishable. But looking at the right edge of the fingerprint we can see some loss. A small part of the lower right edge of the fingerprint can not be seen clearly. Still this is a considerable improvement from the original.

Figure 9. Inverted original fingerprint image and the enhanced fingerprint image using frequency domain filtering.

The third part of the activity involves removing vertical lines found in a composite image of stitched photographs of the moon. This task is considerably much simpler compared to the enhancing a fingerprint. For this task it is clear what structure of the image we do not want and we simply need to create a mask that removes this in the frequency domain.

Figure 10 shows the original and gray-scale image of the moon and in this two images the unwanted vertical lines are clearly seen. But from our previous activities we know that this array of vertical lines is represented by areas along the horizontal of the FT so we create a mask to filter this region out (figure 10).

Figure 10. Original moon image and below it is the gray-scale image.
To the right is the FT and to the lower right is the filtering mask used.

Using the mask in figure 10 we removed the vertical lines and the result is seen on figure 11. The left image in figure 11 shows the resulting image after the mask was applied. It is clearly seen that there are no more vertical lines present in the image. But the final image is also seen to be darker than the original and the lighting is also different. This is probably due to some losses caused by the mask but was not enough to distort the detail in the photograph. Still the objective was achieved and there was very little lost information.

Figure 11. Gray-scale images of the original moon surface and
the resulting image after the mask was applied.

The last part of this activity is aimed to enhance an image of a section of a painting. The canvass weave is clearly seen in this image and distorts the brush strokes and the actual painting. Again we use a mask to remove this canvass weave and recover a clearer image. Figure 12 shows the original and gray-scale image of the painting as well as the FT. We know that since the canvass weave is repetitive then its image in the inverse space are probably symmetric points. So from the FT we identify areas that might be part of the weave and then create a mask to remove this areas. The mask I used is also shown in figure 12. We can see that the mask removes only selected symmetric bright points in the FT.

Figure 12. Original and Gray-scale image of the painting, the Fourier transform
and the mask used to remove the canvass weave.

Figure 13 shows the resulting image after the mask was applied. It is clearly seen that most of the canvass weave has been removed and the painting has much more distinguishable detail. But when we look closely there are still some traces of the canvass weave. The enhancement may not be perfect but the canvass weaved was sufficiently removed, enough to highlight the brushstrokes and details of the painting.

Figure 13. Gray-scale image of the painting and the
resulting image after applying the filtering mask.

Finally we look at the Fourier transform of the mask with the idea that it should result to the canvass weave, but as seen in figure 14 this is not exactly the case. The FT of the mask does resemble the canvass weave but the intensity is not uniform. An elliptic figure seems to be distort the canvass weave pattern. And also along the edges there are lines that cut across the uniformity of the pattern.

Figure 14. Inverted image of the mask and the FT. The FT mimics
the canvass weave pattern with some obvious distortions.

I got the fingerprint image using google image search but I forgot where (sorry). I give my self a grade of 9 in this activity since the results were not perfect.

Activity 6 Properties of the 2D Fourier Transform

In Physics, the Fourier Transform is widely used in various applications, including signal processing and analysis, modeling, and many more. The 2D Fourier Transform, in particular, is very useful in optics; calculations in the inverse space utilizes the Fourier Transform. In this activity we further improve our familiarity with the Fast Fourier Transform functions of Scilab and also the properties of the 2D Fourier Transform.

For the first part of this activity we generate some common shapes with familiar Fourier Transforms; figure 1 shows these shapes together with their 2D Fourier Transform. Note that the FT of a circle is an airy disk, while the FT of a double slit look very much like Young's double slit experiment. As for the square, we know that the Ft of a rectangular wave is the sinc function so the Ft of a square will resemble sinc functions along the x axis and y axis. Finally we can see that the FT of two dots is very much like a sinusoid (or cosine) pattern which is expected since the FT of a cosine is 2 peaks.

Figure 1. Different shapes or patterns generated using Scilab
and their corresponding Fourier Transform

The second part of the activity is aimed to familiarize ourselves with the properties of the 2D Fourier Transform using 2D sinusoid patterns. The sine function was chosen since, as I have said, the absolute value of the FT of a sinusoid (or cosine) is known to be 2 peaks.

Figure 2 below shows the 2D Fourier Transform of sinusoid patterns with different frequencies. As expected, the FT images contain 2 peaks vertically symmetric about the center. Also as the frequency of the sinusoid is increased we observe that the peaks move away from the center. This is also the expected result since low frequencies are concentrated on the center. Finally we see that with higher frequency the FT deviates slightly from a spot. For the higher frequencies the spots of the FT is less distinct and is slightly spread, which is probably due to aliasing.

Figure 2. Sinusoid patterns of different frequencies with their Fourier Transforms.
The frequencies are 1, 4, 40, from top to bottom.

For the next part we explore the effect of combining different functions, specifically a sine with an added constant bias and 2 sine functions with different frequencies. The base sinusoid we used has a frequency of 4 (figure 2 mid row) and we see that by adding a constant bias (figure 3 top row) that the FT only differs by the appearance of a central spot. This central spot corresponds to a 0 frequency signal which is exactly the constant bias. As for the 2 sinusoids, we simply added the sinusoid with frequencies 4 and 1. And again we see that the resulting FT is simply just the addition or superposition of their FT seen on figure 2. There for in general we can say that in 2D Fourier Transform the FT of an addition of 2 functions is just the addition of their FT.

Real life signals obtained from actual sensors usually contain a constant bias. For example, an interferogram as detected by a photodiode most likely rides above a bias. In trying to recover the frequencies of such a signal by taking its Fourier Transform, you would sure obtain a high valued zero order or zero frequency term just like in the top row of figure 3. If we want to recover the signal we can just ignore the zero order. But if it is too strong that it floods out the signal it would be better to just center your signal at zero.

Figure 3. (top) A sinusoid of frequency 4 with a constant bias and
(bottom) addition of sinusoids with frequencies 1 and 4. Beside them are their FT.

In figure 4 we see the effect of rotating the sinusoid pattern. Looking at the images we see that the FT is still 2 peaks except that it is also rotated. And just like in figure 2 the location of the peaks are symmetric about the center, also the line joining the 2 peaks is perpendicular to the pattern (same with figure 2).

Figure 4. Rotated sinusoid and its Fourier Transform.

We know look at the effect of combining a sinusoid pattern along the horizontal and the vertical. The first result is a multiplication of the two patterns, which resulted in a checkered pattern. The resulting FT of this pattern have 4 peaks, still symmetric about the center, but looks to be located at the corners of a smaller square. This makes sense since the pattern actually looks like an addition of 2 perpendicular sinusoids (horizontal and vertical) rotated by 45 degrees. In other words we see a pattern with a periodicity that is angled by 45 degrees. The next set of images is just an addition of a vertical and a horizontal sinusoid patterns.

Figure 5. (top) Multiplication and (bottom) addition of a vertical
and horizontal sinusoid with their Fourier Transforms.

Finally we make a pattern by adding multiple sinusoid patterns each rotated by an arbitrary angle. The resulting pattern doesn't seem to mimic the previous pattern we have presented but still looks to have some periodicities and symmetries. Looking at the FT of this pattern we can say that it is just like an addition or superposition of the FTs of the different rotated sinusoids. This is observation is confirmed by the circular appearance of the FT. So over all we can say this final image just reaffirm what we have already demonstrated in figures 3 and 4.

Figure 6. Addition of arbitrarily rotated sinusoid patterns and its Fourier Transform.

I give my self a grade of 10 in this activity. I also want to thank master, jica, and jaya for staying with me in the lab and for reminding me to do the blog.

Activity 5 Fourier Transform Model of Image Formation

This is an activity which explores the functionality of the built in two-dimensional Fourier Transform of Scilab 4.1.2, fft2(). We will also be showing how this function, along with the convolution and correlation theroems, can be used in various image processing and analysis tasks.

The first task is to familiarize a\ourselves with the fft2()function. Here we first constructed an image of a circle and the letter A (first row). Then using the fft2() and the fftshift() functions we are able to display both of their Fourier Transforms. We know that the results are correct by examining the images. We already know that the fourier transform of a circle or a point is that of an Airy disk and that a larger circle will produce a smaller Airy disk and vice versa. Such results are seen in the images bellow (middle row). Finally we take the fourier transform of the fourier transforms (last row). We know see that the final images are inverted with respect to the original. This is also a common result when using the fourier transform functions of other programming languages. Shifting and scaling constants should be used to be able to recover the original image.

Next we the fft2() function and convolution theorem to model an imaging device with a circular aperture. Here we see that when we use an aperture to image an object, the final image is slightly blurred or deformed. Also we only multiplied the FT of the image to the circular aperture and not its FT since the aperture is already at the focus or the inverse space. We can also see that a larger aperture produces a sharper image which is consistent with our previous results; that is a larger aperture will have a smaller Airy disk FT.

The third task is that we use correlation to locate a pattern with in a certain image. Here we have an image with texts "THE RAIN IN SPAIN STAYS MAINLY IN THE PLAIN" and another image with just "A" of the same text size. We used the fft2() function and implemented the correlation theorem to locate the occurence of the letter A in the texts. After doing correlation and taking its FT again we recover the same image but with locations of "A" highlighted.

The final task is to do edge detection using convolution. Our object is an image of "VIP" and we use different patterns in edge detection. The patterns we used are 3x3 matrices of horizontal, vertical and a point. The resulting images clearly show that the method most accurately detected figures in the image that matched the patterns.

Thank you Martin for talking.

I give myself a grade of 10 for a job well done.