UNIVERSITI PUTRA MALAYSIA FACIAL FEATURE EXTRACTION BASED ON IMPROVED HARRIS CORNER DETECTION ALGORITHM ELHAAM BAGHERIAN FSKTM 2011 9
FACIAL FEATURE EXTRACTION BASED ON IMPROVED HARRIS CORNER DETECTION ALGORITHM ELHAAM BAGHERIAN MASTER OF SCIENCE UNIVERSITI PUTRA MALAYSIA 2011
FACIAL FEATURE EXTRACTION BASED ON IMPROVED HARRIS CORNER DETECTION ALGORITHM By ELHAAM BAGHERIAN Thesis Submitted to the School of Graduate Studies, Universiti Putra Malaysia, in Fulfilment of the Requirements For the Degree of Master of Science July 2011
DEDICATION To whom made me think
Abstract of thesis presented to the Senate of Universiti Putra Malaysia in fulfilment of the requirement for the degree of Master of Science FACIAL FEATURE EXTRACTION BASED ON IMPROVED HARRIS CORNER DETECTION ALGORITHM By ELHAAM BAGHERIAN July 2011 Chairman: Assoc. Prof. Rahmita Wirza O.K. Rahmat, PhD Faculty: Computer Science and Information Technology The extraction of facial feature points has become an important issue in many applications, such as face recognition, face expression recognition and face detection. Segmenting the facial features points in an image is the first important step for human face recognition, identification and verification. Problems occur in different face orientations and poses, and under varied lighting conditions, covering and facial expressions. A method of facial feature extraction and corner detection is presented in this study to unravel these problems. The proposed technique has been developed to extract the facial features from a colored image, captured by the webcam under normal lighting condition. In order to precisely extract the facial features such as eyes, mouth and nostrils, some preprocessing steps are employed once the image is captured. Some of these steps are also used during the corner detection phase. Experiments are conducted with a number of images from the frontal, near frontal, up and down views of the head and from different expressions such as happy, sad, surprised and i
neutral. This technique is evaluated on two different standard databases, BioID and George Tech. These two databases consist of 1520 images and 710 images respectively. Each of these databases includes images with different orientations and expressions, occlusions and lighting conditions. This technique is also tested using five different webcams; with different levels of resolution and quality and web camera specifications, in order to maintain the accuracy of the technique. The performance of the technique is judged by its accuracy on each of the features like nose, eyes and mouth. After validations and verifications are made which are based on the defined performance parameter, it can be observed that the proposed technique is more accurate and precise. ii
Abstrak tesis yang dikemukakan kepada Senat Universiti Putra Malaysia sebagai memenuhi keperluan untuk ijazah Master Sains Oleh ELHAAM BAGHERIAN Julai 2011 Pengerusi: Prof. Madya Rahmita Wirza O.K. Rahmat Fakulti: Sains Komputer dan Teknologi Maklumat Pengekstrakan titik ciri wajah telah menjadi isu penting dalam banyak aplikasi, seperti pengecaman wajah, pengecaman ekspresi muka dan pengesanan wajah. Mensegmenkan titik ciri wajah dalam imej adalah langkah pertama yang penting untuk pengecaman, pengenalan dan pengesahan wajah manusia. Masalah-masalah umumnya berlaku pada kepelbagaian orientasi muka dengan kedudukan di bawah keadaan pencahayaan yang berbeza juga wajah terlindung dengan kepelbagaian riak wajah. Satu kaedah pengekstrakan ciri wajah dan pengesanan sudut dipersembahkan dalam kajian ini untuk mengungkap masalah tersebut. Teknik yang dicadangkan ini dibangunkan untuk mengekstrak ciri wajah pada gambar berwarna, yang ditangkap menggunakan webcam di bawah keadaan pencahayaan yang normal. Untuk mengekstrak ciri wajah seperti mata, mulut dan hidung secara tepat, beberapa langkah Prapemprosesan dijalankan selepas foto diambil. Langkah-langkah ini juga digunakan semasa fasa pengesanan sudut. iii
Eksperimen dijalankan dengan beberapa imej dari hadapan, berhampiran pandangan hadapan, atas dan bawah kepala pada riak wajah yang berbeza seperti seronok, sedih, terkejut dan neutral. Teknik yang dicadangkan ini, dinilai berdasarkan kepada dua pangkalan data piawai yang berbeza, iaitu BioID dan GEORTECH. Kedua-dua pangkalan data ini masing-masing terdiri daripada 1520 dan 710 gambar. Setiap pangkalan data ini mempunyai gambar dengan orientasi dan riak wajah yang berbeza, wajah terlindung dan kondisi pencahayaan yang berbeza. Teknik ini juga diuji dengan lima webcam yang berbeza; dengan pelbagai ciri seperti resolusi dan kualiti untuk mengesahkan bahawa dengan webcam yang berbeza spesifikasi, ketepatan teknik dapat dipertahankan. Prestasi teknik ini dinilai daripada ketepatan setiap ciri yang ditemui seperti hidung, mata dan mulut. Setelah disahkan dan dibuktikan berdasarkan pada parameter prestasi yang telah ditetapkan, dapat dilihat bahawa teknik yang dicadangkan lebih tepat dan jitu. iv
ACKNOWLEDGEMENTS I would like to thank my supervisor, Associate Prof.Dr.Rahmita Wirza O.K. Rahmat for her valuable comments and advice through the course of this research. Her encouragement and professional review helped this thesis and other technical papers to be further improved. My further gratitude goes to Dr Nur Izura Udzir for her great help and technical advice. Also, my eternal gratitude is owed to my family who has been supportive in everything especially to my mother, Maryam, for her never ending love and support. I also want to thank all my second family members in Malaysia, including all my friends, for providing me with great friendship and experience in my academic and social life. v
I certify that a Thesis Examination Committee has met on 1 st of July 2011 to conduct the final examination of Elhaam Bagherian on her thesis entitled FACIAL FEATURE EXTRACTION BASED ON IMPROVED HARRIS CORNER DETECTION ALGORITHM in accordance with Universities and University College Act 1971 and the Constitution of the Universiti Putra Malaysia [P.U.(A) 106] 15 March 1998. The committee recommends that the student be awarded the Master of Science. Members of the Thesis Examination Committee were as follows: Mohamed Othman, PhD Professor Faculty of Computer Science and Information Technology Universiti Putra Malaysia (Chairperson) Hajah Fatimah binti Dato Ahmad, PhD Professor Faculty of Computer Science and Information Technology Universiti Putra Malaysia (Internal Examiner) Shyamala a/p C Doraisamy, PhD Faculty of Computer Science and Information Technology Universiti Putra Malaysia (Internal Examiner) Aini Hussain, PhD Professor Department of Electrial Engineering, Electronics and Systems Universiti kebangsaan Malaysia (External Examiner) NORITAH OMAR, PhD Associate Professor and Deputy Dean School Of Graduate Studies Universiti Putra Malaysia Date: 1 July 2011 vi
DECLARATION I declare that this thesis is my original work except for quotations and citations which have been duly acknowledged. I also declare that it has not been previously, and it is not concurrently, submitted for any other degree at Universiti Putra Malaysia or at any other institutions. ELHAAM BAGHERIAN Date: 1 July 2011 vii
TABLE OF CONTENTS ABSTRACT ABSTRAK ACKNOWLEDGMENTS APPROVAL DECLARATION LIST OF TABLES LIST OF FIGURES LIST OF ABBREVIATIONS CHAPTER Page i iii v vi vii x xii xv 1 INTRODUCTION 1.1 Background... 1 1.2 Problem Statement... 5 1.2.1 Size and Orientation...5 1.2.2 Lighting Conditions...6 1.2.3 Occlusions...7 1.2.4 Facial Expression...8 1.3 Research Objectives and Scope... 9 1.4 Research Contributions... 9 1.5 Thesis Organisation... 9 2 LITERATURE REVIEW 2.1 Introduction... 11 2.2 Main Approaches... 12 2.3 Face Recognition Techniques... 13 2.3.1 Neural Networks...15 2.3.2 Geometrical Feature Matching...15 2.3.3 Graph Matching...17 2.3.4 Eigenface...18 2.3.5 Fisherface...20 2.4 Facial Feature Extraction... 22 2.5 Techniques of Facial Feature Extraction... 27 2.5.1 Geometry-Based...27 2.5.2 Template-Based...28 2.5.3 Color Segmentation Technique...28 2.5.4 Appearance-Based Approach...29 2.5.5 Hybrid Approach...29 2.5.6 Real-time Approach...31 2.6 Advantages and Disadvantages of Previous Works... 51 2.7 Conclusion... 55 3 METHODOLOGY 3.1 Introduction... 56 3.2 Research Methodology... 56 viii
3.3 Pre-Processing... 62 3.4 Detecting Corners... 64 3.5 Summary... 76 4 RESULTS AND DISCUSSIONS 4.1 Introduction... 77 4.2 Experimental Evaluation... 77 4.3 Standard Databases... 80 4.3.1 BioID Database...80 4.3.2 Georgia Tech Database...81 4.3.3 Evaluation by Databases...81 4.3.4 GT Database...85 4.4 Web cameras... 88 4.4.1 Analyze the Results of Testing by Five Different Web Cameras...101 4.5 Different Lighting Conditions... 102 4.5.1 Analysis of the Results under Different Lighting Conditions 4.6 Summary... 109 5 CONCLUSION AND FUTURE WORK 5.1 Conclusion 110 5.2 Future work 112 REFERENCES... 114 BIODATA OF STUDENT... 121 LIST OF PUBLICATIONS/AWARDS... 122 ix