Defining a control experiment

In order to assess the performance of a fingerprint recognition system under maritime environment, we first need to examine its performance under normal (dry) condition. By evaluating the system under typical dry condition, we define a control experiment to which we can compare the effects of water aging to the system.

To proceed with the controlled experiment, we first register the fingerprint of each volunteer under normal conditions, to use as a template. Once we uniquely determined the template, we then collect more dry prints from each individual. Then, we plot the ROC curve of the normal vs. normal print to analyze the behavior of the fingerprint recognition system when the dataset consists of only dry fingerprint samples. The following is a pictorial representation of the data organization defined for the controlled experiment.

Once we understood the behavior of the system under normal conditions, then we can proceed with evaluating the system under maritime environment.

A New Strategy !

Last quarter, we recruited students to contribute their fingerprint samples to our dataset; however, collecting everyone's pruney fingerprint was time consuming and inconvenient since they were required to hold their fingers in a cup of warm water for 20 minutes which did not let them do much while they were waiting. Although we expected to see a lot of pruning after exposing the subject's finger to water for 20 minute, the effect was minute. Thus in order to make the procedure more effective and less inconvenient, we came up with a new way of exposing fingers to water. Instead of having everyone immersing their fingers in warm water for 20-30 minutes, we will ask them to put on a surgical glove (latex or non-latex) that has a small amount of water inside it. Based on our observations, this procedure not only provides more wrinkly fingers in a shorter amount of time (15-20 minutes), but also allows the volunteers to continue their work as they are waiting. Therefore, this brilliant experimental proceduer will replace the old one as we continue our research.

Future Work

During the course of this quarter, we have generated several different experiments acquiring the histogram and ROC for the database. In future, we will be collecting more finger prints under the water aged conditions as more images will result in more concrete results. To acquire more data, especially with water aged finger prints, we may recruit students in swim team to contribute prints of their pruney fingers to our dataset.

Result & Conclusion

Upon running the algorithm on the entire data set, including both wet and dry finger prints, we achieved our initial results. As the ROC curve and histogram plot indicate, the algorithm does really well under the factors of water aging.


ROC curve of genuine vesus imposture match scores
There is a clear distinction between the scores of genuine versus those of the impostors. The ROC curve for the result also indicates that minutiae based algorithms overcomes the factor of water aging.


Histogram plot of genuine versus imposture match scores
Our next tests were with two separate data sets, one of which is just dry prints, and the other just wet prints. The histograms for these two experiments confirms our hypothesis that water aging does not significantly impact the performance of minutiae based algorithms.


Histogram of Genuine vs. Impostor scores for both dry and wet prints
In the test trial with just wet prints, we were assuming to see some variation in the impostor and genuine scores, believing that the image quality of the wet prints is not as good as those of the dry prints. However, the histogram plot in figure 6 indicates that the different quality did not deter the algorithm from matching the minutiae. Thus our initial hypothesis is confirmed with the results we achieved.

Approaches to Fingerprint Matching

The approaches to fingerprint matching can be classified into three classes:
1. Correlation-based matching
- Superimpose images – compare pixels
2. Minutiae-based matching
- Classical Technique – Most popular
- Compare extracted minutiae


3. Ridge Feature-based matching
- Compare the structures of the ridges
- Everything else


Minutiae-based techniques locates minutiae points and then map their relative placement on the finger. However, this approach also has some drawbacks according to Salil Prabhakar et al. One being the fact that it is hard to extract the minutiae points accurately in case we have low quality fingerprint image. Also this approach does not take into account the global pattern of ridges and furrows (Salil Prabhakar et al). Although the correlation-based method is able to overcome some of the difficulties of the minutiae-based approach, it is still not an option since it requires an exact location of a registration point and are affected by image translation and rotation. Lastly, ridge-feature-based matching is superior to the other two and is based on such feature as orientation map, ridge lines and ridge geometry.

Drawbacks and Issues

So in the last post, we have tested the proposed algorithm on a small scale database and got some unexpected result. So in this post, we will justify our theories mentioned in the last post using tested results.

First, let's reiterate the problems. One of the issues is the match score for two different prints from identical fingers of the same individual was high while it should have been zero or very close to zero. This problem is plainly illustrated in the table below. Looking at row 1, column 2, we get a very high match score while we expect a really low value. Ironically, in column 3, and 4 we get really low values while we should have been getting a high value because now we are comparing the prints of two different individuals.

In the last post, we identified the problem to be the cropping and its size. Well, we were partially right! Let's look at the the cropped images for all four images:

Visually image 3 and 4 are very similar but their match score is still too high (5.0642). We still expect lower!

Note in cropped images above, the core is only visible in image 3 and 4; even there, we do not have the complete view of it.

So according to our experimental result and after careful examination of the source code, it seems that Dr. Yang is using correlation-based matching technique in which he superimposes the images and compare pixels. This technique cannot be used for our application because it is not invariant to translation and rotation. Thus, for the time being we drop this algorithm as an option and we pursue an alternative.

Result for a small scale database

This week we explored different fingerprint algorithms (e.g. libfprint, SDKs of Sensors A & B, ...) to compare our fingerprint samples and compute a match score. We finally found a free novel texture-based Automatic Fingerprint Authentication System (AFAS) proposed by Dr. JuCheng Yang from Chonbuk National University in Korea.

Brief Description of Algorithm

First, it pre-process a fingerprint image and enhances the image by Short Time Fourier Transform (STFT) analysis. Then, a set of invariant moment features, as a kind of texture features, are extracted from a Region of Interest (ROI) area based on the reference point from the enhanced fingerprint image. Each set of invariant moments contain seven invariant moments. Fingerprint verification is realized by Euclidean distance between the two corresponding features of the test fingerprint image and template fingerprint image in the database. This computation is illustrated by the equation below,

E = norm (invMoment_img1 - invMoment_img2)

where E is the Euclidean distance, norm is the built function in MATLAB (i.e. norm(x) is the Euclidean length of vector x). We take the norm of the difference between the invariant moments of two images and that gives us a matching score between two fingerprint images.

Result

The following table illustrates the matching scores of fingerprint samples of the same individual. The samples include both normal and pruney fingerprint samples.



The scores in the diagonal is the Euclidean distance of the same image; hence, the magnitude is zero. The higher the score, the lower the similarities between the two images.

Due to lack of enough samples, we ran this algorithm for a combination of prints of both thumb and index fingers.

Pictorial Visualization of Result

Let's look at some of the scores above and see if they make sense. The maximum value (i.e. 10.34529) corresponds to the matching score between image 8 and image 9.

These two are pruney fingerprint samples of the index finger of the same individual that are taken on the same day. In other word, the amount of pruning and the fingerprint patterns should be identical. Now you may ask why the scores are high in this case. There are a couple of factors that we need to note in our analysis:

1. The role of preprocessing
2. The two different fingerprint registration techniques provided by SDK

Part of the preprocessing portion of the algorithm crops 320 x 240 images to 64 x 64 ; whereas the original image size in our database is 352 x 544. So we might have to assign a different size to our cropped images in order to get better result.

Second point refers to the two different methods to save fingerprint samples which could result in slight variations in the result.

Let's look at another interesting comparison and its corresponding score:




The images above correspond to the fingerprint samples of index finger of the same person taken on the same day. The match score provided by the algorithm for these two is 6.576957274! This is a high value considering the fact that it is the same finger and the same day. We think this is due to variations such as orientation in placement of the finger on the scanner as well as two different methods for registration and recording the samples.


Result for fingers with higher pruning effects

This is the result for an individual who had the highest pruning effect among the entire volunteers. And the scores are as follows:


Next steps ?!

We are a few steps away in our implementation from plotting ROC curves. The following steps are required before we proceed to ROC analysis.

• Develop a concrete labeling system
• Identify genuine vs impostor
  
• ROC curve
  

Going to try out Libfprint

Because of the problems and lack of features in the SDK for sensor a, we will be using a different open source fingerprint system. The open source code we will be working with is called fprint, which can be downloaded from http://reactivated.net/fprint/wiki/Main_Page. Specifically, we will be using the libfprint feature which does the fingerprint processing for the code. We will be setting up the feature on a Linux platform, then hopefully we will be able to run our data to retrieve the results needed for the ROC curve.

Visual Description

So this is a basic visual description of how we are conducting our experiment with the fingerprint data set. The diagram shows the template buckets. Each bucket represents each individual participating in the experiment, thus each bucket will have a template fingerprint scan and also a series of pruned fingerprint scan.



The diagram below depicts the score buckets for our results. We will test each fingerprint scan within the same bucket with the template (the normal finger scan) and record the score and put it into the genuine bucket. Then we will test a template from one bucket with various combination of scans from other buckets (so A.template will be tested with B.template, B.pruned1, B.pruned2, ..., H.template, H.pruned1,...) and the result of this should be put into the impostor bucket.



The histogram below shows an ideal histogram for the results. The curve on the right basically represents the genuine scores, and the curve on the left is the impostor scores. This is an ideal case however, so real results may vary.

Small Scale Histogram Representation

This week, we did a small scale histogram analysis of our fingerprint samples in MATLAB. In other word, we chose an image from our fingerprint dataset and plotted its histogram which is simply a tonal distribution of the image.

Fig1. Histogram Distribution

The image above is a graphical representation of tonal variations versus the number of pixels in that particular tone. Furthermore, using histogram equalization we looked at the intensity distribution of the original versus contrast enhanced image. The result corresponding to histogram equalization is displayed in Fig2,3, and 4.


Fig2. Original Image vs. Intensity Image



Fig3. Histogram Representation of the Original Image


Fig4. Histogram Representation of the Intensity Image

Sensor B in Action

This weekend, we were able to test Sensor B (generic TIR sensor). In comparison to our MSI sensor, the images are faded and not as significant. Below are some examples of the images





















So on the left is an enrollment of a fingerprint into its database. The sensor requires four different scans of the fingerprint. Then on the right is the verification process, which can be done recursively as many times as wanted. The panel on the right side will display whether the finger scan matched the enrolled fingerprint. Here you can see a number of VERIFIED messages, a few that did not register (the sensor did not sense a finger on the platen). And the last one is a failure (where the correct finger was placed but the sensor did not verify). The region that made contact on the platen was the tip of my finger.

TIR sensors are so picky....

There are certain conditions that TIR sensors require to get good quality images. These conditions are the following:

• Good contact between the finger and the platen
• Fingerprint features are well defined
• Skin has a proper index of refraction
• There is no water, oil or other contaminant on the platen
  

Below are some generic samples that were collected (using the fingerprint sensor described in Fingerprint Enhancement Using a Multispectral Sensor)


.

Further Analysis of TIR sensor technology

The source of light is generally from LED's (light emitting diodes) that are located on the left side of the prism. This light then reflects off of a diffuse reflective coating that is located on the right side, causing the upper region of the prism (the platen) to light up. If there is no finger placed on the platen, the light will reflect off the platen due to total internal reflectance. There is an imaging system in the lower left side, which will create the image of the top horizontal platen. So when a finger is placed on the platen, the light will cross the valley’s of the fingerprint and cause TIR at these “valley” like locations. So the image that is processed will have dark features where there is contact with the ridges, and lighter features where there is none.

Sensor A vs Sensor B

To further proceed with our research, we need to do a quick comparison and discuss the differences and similarities of two sensors.

Features of Sensor A

• Utilizes Multispectral imaging
• Flexible, powerful device outputs
• Compact design
• Pixel Resolution 500 dpi
  
• State-of-the-art prevention technology
• Expansive operating range

Features of Sensor B

• Utilizes optical fingerprint scanning technology
• Superior ESD resistance
• Small form factor
• Excellent image quality
• Encrypted fingerprint data
• Latent print rejection
• Counterfeit finger rejection
• Rotation invariant
• Rugged
• Works well with dry, moist, or rough
fingerprints
• Compatible with Windows® Vista,
XP Professional, 2000 and Windows
Server 2000, 2003

Key Specifications

• Pixel resolution: 512 dpi (average x, y
over the scan area)
• Scan capture area: 14.6 mm (nom.
width at center) 18.1 mm (nom.
length)
• 8-bit grayscale (256 levels of gray)
• Reader size (approximate): 79 mm x
49 mm x 19 mm
• Compatible with USB 1.0, 1.1 and 2.0
(Full Speed) specifications
• Indoor, home and office use

EUReKA

Every year, Jacobs School of Engineering gives undergraduate students the opportunity to showcase their research at Engineering Undergraduate Research Konference & Assembly (EUReKA). Karan and I decided to participate in this annual event by making a poster about our ongoing project especially since this year EUReKA will coincide with the Corporate Affiliates Program (CAP) board meeting in the Fung Auditorium, which could be an opportunity to get job offers from companies' representives presented at the CAP meeting. So this is a great opportunity to describe our research project, its implications, and results to-date.

What Are Fingerprint Minutiae

Fingerprints are known to be unique to every individual. A Minutiae is defined as: the points of interest in a fingerprint, such as bifurcations (a ridge splitting into two) and ridge endings. Types of ridges:

• ridge endings - a ridge that ends abruptly
• ridge bifurcation - a single ridge that divides into two ridges
• short ridges, island or independent ridge - a ridge that commences, travels a short distance and then ends
• ridge enclosures - a single ridge that bifurcates and reunites shortly afterward to continue as a single ridge
• spur - a bifurcation with a short ridge branching off a longer ridge
• crossover or bridge - a short ridge that runs bbetween two parallel ridges

Relevant papers:
www.research.ibm.com/ecvg/pubs/aws-constructive.pdf

MSI vs TIR

This week we read up a little bit more on Multispectral images (MSI) and total internal reflection images (TIR). The way that these MSI scanners are programmed is that they integrate both the capabilities of MSI and TIR images when the user places his or her finger on the scanner. The multispectral images are processed to enhance the fingerprint minutiae. Both MSI and TIR images are then passed into the the SDK for the scanner to detect the minutiae in the fingerprint.

Similarities:

1. Light Source - provides the light for our sensor to illuminate the platen on top of which the finger rests.
2. Imaging System – creates the digital imaging array from the image pn the platen.

Differences:

• MSI - The orientation of the light source and imagers do not exceed any critical angle conditions.
• TIR – In normal TIR sensors, the light strikes a medium larger than the critical angle.
• MSI - Has multiple illumination wavelengths,
• TIR - Monochromatic illumination system.
• MSI - uses polarizers that are arranged orthogonally to highlight the light that penetrates the skin surface. Once the light penetrates the skin surface, it undergoes multiple scattering events, once the light exits the skin it makes its way to the image array

Full Research Description of this is found in the following papers:

1. Multispectral Fingerprint Biometrics

2. Fingerprint enhancement using a multispectral sensor

Related Terminologies

The following terminologies will be commonly used throughout the blog:

Biometric - Recent studies of methods for recognizing distinct human intrinsic physical or behavioral characteristics.
Fingerprint - Commonly used physical trait in biometric identification
Pruney Fingers - The wrinkles occur in skin after a drawn-out (15-20 min) exposure to water.
TPR - True positive rate (benefits)
FPR - False positive rate (costs)
ROC Curves - Relative Operating Characteristic curve is used as a graphical representation to illustrate a comparison of two operating characteristics (TPR & FPR) as the criterion changes. An example of this curve is illustrated below:

An Introduction to ROC Analysis

Stratum corneum (or horny layer) - The outermost layer of the epidermis, composed of large, flat, polyhedral, plate-like envelopes filled with keratin which are the dead cells that have migrated up from the stratum granulosum.

Stratum corneum

Pruning - Wrinkling of the skin as immersed in water for prolonged period. A protein, keratin, in stratum corneum which is responsible for skin hydration, absorbs water. So, as toes or fingers are exposed to water for a prolonged time, this layer of skin will develop wrinkles due to its stretchy properties of skin. However, this condition is temporary and it wears off within minutes.

Research

In order to start our experimental trial, we need to understand the process result in pruney fingers . The duration and temperature are the two parameters that make a difference in our result. So we chose the following paper to complete our understanding of the effect of water aging on fingerprints. 

Effects of water temperature on skin wrinkling

According to this study, the pruning using 40 degree Celsius water will occur in about 5 minutes. For the purpose of our study, we would like to keep the temperature constant to the room temperature. So it will be around 20 degree Celsius. Thus, the duration will increase accordingly.



A similar study shows that the average time to wrinkling in distilled water is 11.5 minutes. The following is a link to this study:

Fingertip Skin Wrinkling the Effect of Varying Tonicity

Experimental Procedure

1. Register fingerprints of volunteer participants
2. Make pruney fingers (i.e. leave the same finger in water for 15 min)
3. Take samples of a wet-wrinkled finger
   
4. Test to see if the wet-wrinkled sample matches the normal sample
   
5. Then take samples of dried- wrinkled finger
6. Test for a match
7. Create a database for indexing and labeling
8. Use ROC curves for further analyses

*Next class period, we will ask our fellow classmates who are willing to participate in our experimental procedure to register their fingerprints and sign a consent form.
* A consent form will be passed out today

Who will benefit from this?!

• Navy Officers boarding ships
• Constructor workers
  
• People diagnosed with hyperhidrosis disorder
  

Abstract

The objective of this project is to assess the effects of maritime environment (e.g. sea water, fresh water,etc) on the performance of fingerprint recognition systems. For the purpose of this project, we obtained two kinds of sensors that will perform the data acquisition using two separate methods. To conduct a series of experimental trials, we will recruit volunteers to contribute their fingerprints to the dataset. In the course of this project, we will examine the difference of pruney fingers in fresh water vs. seawater and will address the potential problems that can arise with fingers that dry out after exposure to these environments (e.g. cracks or salt crystal debris). Finally, we will use a graphical plot, receiver operating characteristic (ROC) curve, to illustrate the final result.