1 Higher Institute for Applied Sciences and Technology Communications Department Detection of Abandoned Objects in Crowded Environments June 22 2016 Submitted by : Ali Assi Supervisors : Dr. Nizar zarka En. Bassel Shanor Multimedia 4 th year Project
Contents 1. Table of Figures... 3 2. List of Abbreviations... 4 3. Key Words... 5 4. Abstract... 6 5. Introduction... 7 6. General Idea... 8 7. Algorithm of this project... 8 Region Of Interest (ROI)... 8 Auto Threshold... 8 Subtraction... 9 Convert color Type... 9 Image Segmentation... 10 Morphological Operation... 10 Object Tracking... 10 Alarm... 11 8. Flowchart... 12 9. Results... 13
1. Table of Figures Figure 1 : Subtraction... 9 Figure 2 : RGB to YCrCb Conversion... 9 Figure 3 : Image Segmentation... 10 Figure 4 : Flowchart... 12 Figure 5 : All objects... 13 Figure 6 : Abandoned Objects... 13 Figure 7 : All Objects... 14 Figure 8 : Abandoned Objects... 14
2. List of Abbreviations ROI : Region Of Interest RGB : Red Green Blue (Color Type) YCrCb : Color Type.
3. Key Words Abandoned objects. Crowded environments.
4. Abstract We present a new project to robustly and efficiently detect abandoned and removed objects in complex environments. Several improvements are implemented to the background subtraction method for shadow removal, quick lighting change adaptation. The system is capable of handling concurrent detection of multiple abandoned objects, up to 200 objects, track them, and set an alarm if any abandoned object was detected. Any object could be left for specified period of time (certain number of frames); if it exceeds the limit, the system will set an alarm. On the other hand, abandoned object could be missed for certain number of frames, if it exceeds the limit of frames, the system will turn the alarm off.
5. Introduction The monitoring and surveillance of unattended baggage has become a priority for the security, because of the increasing number of incidents where terror organizations have planted explosive devices in ordinary baggage. One of the more critical challenges faced by security personnel monitoring mass transportation terminals and other busy public facilities is the issue of real-time detection of unattended or abandoned luggage. From airports to train stations and bus depots, museums, sports stadium and other places which are considered as crowded environments. Recent studies have shown that the average human can focus on tracking the movements of up to four dynamic targets simultaneously, and can efficiently detect changes to the attended targets but not the neighboring distractors, that is why we need to depend on the technology, which can be used to assist security officers monitoring live surveillance video by directing their attention to a potential area of interest. Professor of Artificial Intelligence at the University of Leeds, David Hogg, says: "Due to increased anxieties around the threat of terrorism, the monitoring and surveillance of unattended baggage has become a top priority across the globe. By employing advanced computer technology our system will make this kind of surveillance much less prone to human error."
6. General Idea In this project, our main goal is to detect any abandoned object in crowded environments, and then if the abandoned was left for a specified time, we will set an alarm by drawing a rectangle over the detected abandoned object. 7. Algorithm of this project Region Of Interest (ROI) You don`t need to watch all the area which the camera covers, may all you need is to watch a specified region, that`s what we called the region of interest. Selecting the region of interest minimize the errors that could happened because of the movement of some objects that we don`t care about, such as train in the train station etc. Auto Threshold Since many functions work with binary pictures it`s recommended to convert the frames to binary. The auto threshold operation determines the threshold by splitting the histogram of the input image to minimize the variance for each of the pixel groups, and that`s what we called Otsu's method.
Subtraction First, we will consider the first frame of the video as a background, and to detect the new object in the video; we will subtract the background from each frame,, we will detect every new object because of the difference between the background and the current frame. Figure 1 : Subtraction Convert color Type To get a better results, it`s recommended to convert the color type from RGB to YCbCr, which offers greater robustness to changes in illumination. Figure 2 : RGB to YCrCb Conversion
Image Segmentation To define the many objects in the videos, the segmentation process will return statics of the input video, and that`s how we get the parts of object connected to each other. Having blobs or segmented objects is necessary to detect the various objects. Figure 3 : Image Segmentation Morphological Operation A series of morphological operations may be carried out to clean up the image, retaining only the most useful segments Applying Closing operation on the segmented objects will help to get clearer frames and obvious connected objects. Object Tracking The system will track the movement of the detected object, to notify the client about any new elements in the video, and maybe to track the owner of the abandoned baggage.
Alarm If any abandoned object was detected and still exist for a period of time, we will set an alarm, and if the detected abandoned object was missed for a specified time, the alarm will turn off again. In our case, the alarm is a filled rectangle drawn over the detected object.
8. Flowchart Figure 4 : Flowchart
9. Results After applying the project on video, we notice that by changing the ROI we will get a different results, it`s necessary to select the region of interest before starting the detecting object in order to minimize the errors. Any change in the frames will be considered as new object, and could be considered as abandoned object. Even after changing the color type from RGB to YCrCb, the change in luminance and the existing of shadows will affect the result. Selecting small ROI, we got the following results: Figure 5 : All objects Figure 6 : Abandoned Objects
We notice that we got 100% correct result, but with a bigger ROI, we will get some errors : Figure 7 : All Objects Figure 8 : Abandoned Objects So we got one false detection in this case.
10. Conclusion and future work Public safety is a critical issue in our world today. Through the assistance of automatic threat detection systems, security personnel may be equipped with instant and comprehensive awareness of potential crises. In this paper, we introduce a general framework to recognize the event of object abandonment in a busy scene. The proposed algorithm is characterized by its simplicity and intuitiveness. Segmentation is another challenging issue, including foreground as well as object segmentation. For better foreground segmentation, it would be worth exploring techniques of adaptive background modeling, or a mechanism for switching among pre-stored background models (backgrounds of the platform with and without the train, for example). Many improvements could be applied on this project, when an unattended object is detected, the system traces it back in time to determine and record who its most likely owner(s) may be.