A NON-INTRUSIVE METHOD FOR DRIVER DROWSINESS DETECTION USING FACIAL LANDMARKS

Driver weariness is one of the real reasons behind accidents. Recognizing the sleepiness of the driver is one of the best methods for estimating driver weariness. The motivation behind this paper is to build up a drowsiness detection system that works by monitoring the eye movement of the driver and alerting the driver by producing an alarm or vibration when the person is found drowsy. This paper shows a non-intrusive model for the fatigue detection dependent on processing video streams of an individual's face. The proposed model is not quite the same as meddlesome techniques dependent on natural methodology (Electroencephalogram, Electrooculogram and some sensors), which require gadgets explicitly. Unlike traditional image processing techniques, we use computer vision and machine learning technique to display a prototypal adaptation of a real-time system with individual feedback to monitor and identify when the driver may be sleepy directly from a web camera. The drowsiness detection model depends on face alignment and then evaluation of the Eye Aspect Ratio (EAR) which uses Histogram of oriented gradient (HOG) features combined with Support Vector Machine (SVM) classifier for blink detection. Utilizing such a system, it is conceivable to alarm the client of the threat of nodding off, so that enough actions can be made, diminishing the risk of human mistake and avoiding accidents.

3C Tecnología. Glosas de innovación aplicadas a la pyme. ISSN: 2254 -4143 Edición ABSTRACT Driver weariness is one of the real reasons behind accidents. Recognizing the sleepiness of the driver is one of the best methods for estimating driver weariness. The motivation behind this paper is to build up a drowsiness detection system that works by monitoring the eye movement of the driver and alerting the driver by producing an alarm or vibration when the person is found drowsy. This paper shows a non-intrusive model for the fatigue detection dependent on processing video streams of an individual's face. The proposed model is not quite the same as meddlesome techniques dependent on natural methodology (Electroencephalogram, Electrooculogram and some sensors), which require gadgets explicitly. Unlike traditional image processing techniques, we use computer vision and machine learning technique to display a prototypal adaptation of a real-time system with individual feedback to monitor and identify when the driver may be sleepy directly from a web camera. The drowsiness detection model depends on face alignment and then evaluation of the Eye Aspect Ratio (EAR) which uses Histogram of oriented gradient (HOG) features combined with Support Vector Machine (SVM) classifier for blink detection. Utilizing such a system, it is conceivable to alarm the client of the threat of nodding off, so that enough actions can be made, diminishing the risk of human mistake and avoiding accidents.

KEYWORDS
Non-Intrusive Method, Facial Landmarks, Support Vector Machine, Eye Aspect Ratio (EAR).

INTRODUCTION
Drowsiness is an automatic human physical movement. Since tiredness can be specifically identified with the human fixation and liveliness, drowsiness identification has been connected in fields like in human behaviour investigation, fatigue detection, readiness level estimation and so on. When the driver experiences drowsiness state, he starts losing the control and he might be suddenly deviated from the road and may hit an obstacle or another car. Driver weariness is one of the critical factors in many auto collisions. Safe driving is a The techniques to measure drowsiness are classified into two types.
Intrusive techniques require the use of some explicit gadgets which always require some measuring gadgets to be in contact with human body which causes some sort of disturbance to the person so these are often referred to as contact type method (Electroencephalogram, Electrooculogram), whereas in Nonintrusive techniques there is no need of such devices which require contact and it requires high-quality cameras for capturing images or monitoring the video stream, it is also known as contactless method.
In this paper, we propose a system for drowsiness detection which uses computer vision techniques which uses a machine learning library. Object recognition, tracking is a standout amongst the most created research spaces in the field of image processing and computer vision. It is used in several useful applications in all the available domains like video-stream analysis, industrial automation, internet of things, robotics, security and surveillance, and in many more applications. The similar idea is used to detect the human body to identify eye, mouth, arms and recognizing its shape and movement, making a diagnosis and deriving human behavior. There are many algorithms for blink detection which are trained on a wild dataset.
In the proposed system, we monitor the driver continuously and intimate the driver about the drowsiness when the specified conditions are met. We use facial landmarks technique which identifies the face region. The algorithm behind the facial landmarks is the Histogram of oriented gradients combined with support vector machine classifier. The detector we use is a pre-trained model trained on HELEN dataset. A threshold value of eye aspect ratio will be kept. The person is monitored continuously, and the eye aspect ratio is evaluated each time and compared with the threshold value to decide whether the state of the eye is open or not. It will be implemented using OpenCV in Python. Yan et al. (2016) proposed a model for real-time drowsiness detection depending on PERCLOS evaluation and grayscale image processing to identify whether the driver is sleepy or not. Their algorithm was comprised of three steps. First, it identifies the position of the face of the driver and converts them into grey-scale images, and then the position of eyes is localized using a small template. Secondly, the data obtained in the first step is used

RELATED WORKS
to establish a drowsiness model and at last, depending on the individual drowsiness model the developed system monitors the driver's state without interruption. Once the driver is found to have sleepiness state, an alarm is produced to alert the driver. Leo and Sankar (2015) used Haar based cascade classifier for identification of eyes and the Histogram oriented gradients with Support vector machines for the detection of blink.
When the eye blinks were detected the PERCLOS value for the eyes was calculated. If the computed value is beyond six seconds, it classifies the person as drowsy. First, the driver's face is monitored through a camera and then Viola and Jones's method was used to identifying the face of the driver. Once the face is identified the Haar features are extracted and then the blink detection algorithm is used to determine the blink in the video stream and then the PERCLOS is evaluated.
According to Amin and Rahmati (2011), the developed system acquires the images in a sequence and given as input in the first step. The system follows 4 steps: Detection of the face, extraction of facial components, tracking of facial components and detection of drowsiness.
In that system, the background subtraction method is used to detect the face region in the image and this step is repeated until the face region is obtained accurately. The facial components like eyes, eyebrows and mouth region are extracted. The method used in the second step is Horizontal projection and template matching. The results obtained in the third step is matched with the reference template for tracking. Using the reference template, the person can be identified as feeling sleepy or not.

METHODOLOGY
This paper presents an algorithm to precisely estimate the position of eyes using facial landmarks in a computationally efficient way. Our proposed model has 4 phases.
• Monitoring for the face in a video stream.
• Evaluating the eye aspect ratio (EAR).
Unlike some traditional image processing techniques for extracting eye regions or to detect blinks, which typically involves the following processes: (1) Face Detection (2) (5) Evaluate some parameters related to detect the state of an eye(open or close). Using these methods, the output can be achieved but the result may vary when there is a head movement, or any facial expression variation is noticed.
Our goal is to detect the eyes using a shape prediction technique. The machine learning library 'dlib' has the facial landmark detector which has been trained based on HELEN dataset. The algorithm behind this detector is the combination of Histogram of oriented gradients (HOG) and Support Vector Machines classifier.

A. Histogram of oriented gradients
The functions we use from dlib library has been trained based on the HOG algorithm.
HOG is a feature descriptor which is a portrayal of an image that simplifies the image by removing unnecessary information and extracting useful information The feature vector generated by the algorithms is given into an image classification algorithm like Support Vector Machine (SVM) to generate accurate outcomes.
In the HOG feature descriptor, the directions of gradients distribution are used as features.
Gradients values are mapped to 0-255. Pixels with large negative change will be black, pixels with large positive change will be white, and pixels with little or no change will be gray.

B. Facial Landmarks
Dlib is a landmark's facial detector with pretrained models. It has two types of models.
• 2 points for the right eye.
• 1 point on the nose.
Whereas the 68-point detector localizes regions along the eyes, eyebrows, nose, mouth and jawline. In terms of speed, the 5-point detector is 8-10% faster than the 68-point detector and it also consumes 10 times less memory. So, the 5-point detector is fast and the most appropriate use of 5-point detector is the face alignment.
But for our proposed work since we need to compute eye aspect ratio, if we use 5-point detector we cannot evaluate the eye aspect ratio as there were no vertical points for eye 3C Tecnología. Glosas de innovación aplicadas a la pyme. ISSN: 2254 -4143 Edición Especial Special Issue Noviembre 2021 region so we are using the 68 point detector where we have six points per eye and the eye aspect ratio can be computed easily to produces the accurate results.

C. Computing Eye Aspect Ratio
The Eye Aspect Ratio is an estimate of the eye-opening state. If the computed Eye Aspect Ratio value is lower than a threshold value, we could tell that the person's eyes are closed.
The indexes of the eyes are taken into consideration to compute the eye aspect ratio value. The numerator of the EAR equation calculates the distance between the vertical eye landmarks while the denominator calculates the distance between horizontal eye landmarks.
When the eye is detected open the EAR value is almost constant and when the eye is closed (blink) the EAR suddenly drops to zero.
We can use this simple equation to avoid image processing techniques and the blinking detection simply depends on the ratio of vertical and horizontal eye landmark distances.
We use SciPy package to compute the Euclidean distance between eye facial landmarks We also include play sound library so that a suitable alert can be produced when the EAR value is below a threshold value.

RESULTS
The face of the driver was monitored continuously, and the landmarks were applied when the face was found. Even if there is a tilt in the head movement the detector was able to identify the eye regions if the face was in the scope of the camera. Once the eye regions are obtained the eye aspect ratio is evaluated. The threshold we kept is 0.28 as it would be more approximate value for almost 70% of the people. And when the eye aspect ratio computed was found to be below the threshold value then an alarm was produced. And if the EAR value is above the threshold value then the EAR would be continuously monitored until we turn off the device. We observed that the CNN based detector works well for non-frontal faces at odd angles where HOG based detector struggles.
Unfortunately, CNN based detector is computationally heavy and is not suitable for realtime video now. We use HOG detector as it is highly efficient and there is no need of detecting any faces at odd angles. When the face is detected the facial landmarks will be applied and eye regions will be extracted and to make it visible, we applied draw Contour method. The indexes of the eye are sent to the eye aspect ratio method and computed for each eye and prints the average value. The output shows the EAR value obtained.  When the eye aspect ratio computed is below the threshold value i.e. 0.28 and it lasts for 40 consecutive frames then drowsiness will be detected, and this process will be repeated continuously.

CONCLUSIONS
In this paper, a non-intrusive method for drowsiness detection was implemented using detector from dlib library and OpenCV in Python. The predictor we used is a 68-landmark predictor. The person will be monitored continuously and when detected the landmarks will be applied and EAR value is evaluated and then compared that with the threshold value. The proposed approach uses Eye Aspect Ratio and Eye Closure Ratio with adaptive thresholding to detect driver's drowsiness in real-time. This is useful in situations when the drivers are used to arduous workload and drive unceasingly for long distances. The system, if integrated, can reduce the number of fatalities and harms that happen regularly due to these drowsy states of the drivers.
The developed system works well under the daylight conditions and in any resolutions. If the device should be used under dark conditions as well, then we can implement the system using night vision cameras. This makes the system suitable to work in any conditions.