计算机视觉 基于opencv和dlib库的疲劳检测系统 使用opencv和dlib库实现人脸关键点的标定 对检测到疲劳时，发出预警 疲劳驾驶预警

基于opencv和dlib库的疲劳预警检测系统

构建点实现关键点
主要实现策略：

1、开发软件界面，获取人脸视频
2、使用opencv和dlib库实现人脸关键点的标定
3、定义并提取疲劳人脸特征
4、对检测到疲劳时，发出预警

基于OpenCV和dlib库的疲劳检测系下面是代码示例，仅供参考。，包括人脸检测、关键点定位、特征提取和疲劳判断等。代码示例，展示了如何实现这个系统。

主要代码示例：

1. 开发软件界面，获取人脸视频

使用Python的tkinter库来创建GUI界面，并使用OpenCV捕获视频流。

import tkinter as tk
from tkinter import filedialog, messagebox
import cv2
import dlib
import numpy as np
from PIL import Image, ImageTk

# 初始化dlib的人脸检测器和关键点预测器
detector = dlib.get_frontal_face_detector()
predictor = dlib.shape_predictor("shape_predictor_68_face_landmarks.dat")

class FatigueDetectionApp:
    def __init__(self, root):
        self.root = root
        self.root.title("Fatigue Detection System")
        
        # 创建左侧显示区域
        self.video_frame = tk.Frame(root)
        self.video_frame.pack(side=tk.LEFT, padx=10, pady=10)
        self.video_label = tk.Label(self.video_frame)
        self.video_label.pack()

        # 创建右侧设置区域
        self.settings_frame = tk.Frame(root)
        self.settings_frame.pack(side=tk.RIGHT, padx=10, pady=10)

        # 视频源选择
        self.video_source_var = tk.StringVar(value="摄像头ID")
        tk.Label(self.settings_frame, text="视频源").pack(anchor=tk.W)
        tk.OptionMenu(self.settings_frame, self.video_source_var, "摄像头ID", "视频文件").pack(anchor=tk.W)
        tk.Button(self.settings_frame, text="开始检测", command=self.start_detection).pack(anchor=tk.W)
        tk.Button(self.settings_frame, text="打开视频文件", command=self.open_video_file).pack(anchor=tk.W)
        tk.Button(self.settings_frame, text="暂停", command=self.pause_detection).pack(anchor=tk.W)

        # 疲劳检测设置
        tk.Label(self.settings_frame, text="疲劳检测").pack(anchor=tk.W)
        self.blink_detection_var = tk.BooleanVar(value=True)
        tk.Checkbutton(self.settings_frame, text="眨眼检测", variable=self.blink_detection_var).pack(anchor=tk.W)
        self.eye_closure_var = tk.BooleanVar(value=True)
        tk.Checkbutton(self.settings_frame, text="闭眼检测", variable=self.eye_closure_var).pack(anchor=tk.W)
        self.head_pose_var = tk.BooleanVar(value=True)
        tk.Checkbutton(self.settings_frame, text="点头检测", variable=self.head_pose_var).pack(anchor=tk.W)

        # 输出状态
        tk.Label(self.settings_frame, text="状态输出").pack(anchor=tk.W)
        self.status_text = tk.Text(self.settings_frame, height=10, width=30)
        self.status_text.pack(anchor=tk.W)

        # 其他变量
        self.cap = None
        self.running = False

    def start_detection(self):
        if not self.running:
            if self.video_source_var.get() == "摄像头ID":
                self.cap = cv2.VideoCapture(0)
            else:
                self.cap = cv2.VideoCapture(self.video_file_path)
            self.running = True
            self.update_frame()

    def open_video_file(self):
        self.video_file_path = filedialog.askopenfilename(filetypes=[("Video files", "*.mp4;*.avi")])
        if self.video_file_path:
            self.video_source_var.set("视频文件")

    def pause_detection(self):
        self.running = False

    def update_frame(self):
        ret, frame = self.cap.read()
        if ret:
            frame = self.detect_fatigue(frame)
            image = cv2.cvtColor(frame, cv2.COLOR_BGR2RGB)
            image = Image.fromarray(image)
            image = ImageTk.PhotoImage(image)
            self.video_label.config(image=image)
            self.video_label.image = image
            if self.running:
                self.root.after(10, self.update_frame)
        else:
            self.cap.release()
            self.running = False

    def detect_fatigue(self, frame):
        gray = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY)
        faces = detector(gray)
        for face in faces:
            landmarks = predictor(gray, face)
            points = []
            for i in range(68):
                point = (landmarks.part(i).x, landmarks.part(i).y)
                points.append(point)
                cv2.circle(frame, point, 2, (0, 255, 0), -1)

            # 计算EAR（眼睛长宽比）
            left_eye = points[42:48]
            right_eye = points[36:42]
            ear_left = self.calculate_ear(left_eye)
            ear_right = self.calculate_ear(right_eye)
            ear = (ear_left + ear_right) / 2.0

            # 计算MAR（嘴部长宽比）
            mouth = points[48:68]
            mar = self.calculate_mar(mouth)

            # 判断疲劳
            if ear < 0.2:
                self.status_text.insert(tk.END, f"{datetime.now()} 眨眼\n")
            if mar > 0.5:
                self.status_text.insert(tk.END, f"{datetime.now()} 打哈欠\n")

            # 绘制信息
            cv2.putText(frame, f"EAR: {ear:.2f}", (10, 30), cv2.FONT_HERSHEY_SIMPLEX, 0.7, (0, 0, 255), 2)
            cv2.putText(frame, f"MAR: {mar:.2f}", (10, 60), cv2.FONT_HERSHEY_SIMPLEX, 0.7, (0, 0, 255), 2)

        return frame

    def calculate_ear(self, eye_points):
        A = np.linalg.norm(np.array(eye_points[1]) - np.array(eye_points[5]))
        B = np.linalg.norm(np.array(eye_points[2]) - np.array(eye_points[4]))
        C = np.linalg.norm(np.array(eye_points[0]) - np.array(eye_points[3]))
        ear = (A + B) / (2.0 * C)
        return ear

    def calculate_mar(self, mouth_points):
        A = np.linalg.norm(np.array(mouth_points[13]) - np.array(mouth_points[19]))
        B = np.linalg.norm(np.array(mouth_points[14]) - np.array(mouth_points[18]))
        C = np.linalg.norm(np.array(mouth_points[15]) - np.array(mouth_points[17]))
        D = np.linalg.norm(np.array(mouth_points[12]) - np.array(mouth_points[16]))
        mar = (A + B + C) / (2.0 * D)
        return mar

if __name__ == "__main__":
    root = tk.Tk()
    app = FatigueDetectionApp(root)
    root.mainloop()

基于OpenCV和dlib库的人脸疲劳检测系统，主要包含以下步骤：

1. 人脸检测：使用dlib的HOG+SVM分类器或OpenCV中的Haar Cascade进行人脸检测。
2. 关键点定位：使用dlib提供的68个面部关键点检测模型（shape_predictor_68_face_landmarks.dat）来获取眼睛、嘴巴等部位的关键点。
3. 特征提取：计算眼睛长宽比（EAR）、嘴部长宽比（MAR）等特征用于判断疲劳状态。
4. 疲劳判断：根据EAR和MAR阈值判断是否眨眼、打哈欠，并触发预警。

---

✅ 一、环境准备

安装必要的库：

pip install opencv-python dlib numpy imutils

下载 dlib 的 68点人脸关键点模型，并解压到项目目录下。

---

✅ 二、完整代码实现

import cv2
import dlib
import numpy as np
from scipy.spatial import distance as dist
from imutils import face_utils
import time
import winsound  # Windows 报警音

# 初始化dlib人脸检测器和关键点预测器
detector = dlib.get_frontal_face_detector()
predictor = dlib.shape_predictor("shape_predictor_68_face_landmarks.dat")

# 眼睛关键点索引
(lStart, lEnd) = face_utils.FACIAL_LANDMARKS_IDXS["left_eye"]
(rStart, rEnd) = face_utils.FACIAL_LANDMARKS_IDXS["right_eye"]
(mStart, mEnd) = face_utils.FACIAL_LANDMARKS_IDXS["mouth"]

# 计算眼睛长宽比 EAR
def eye_aspect_ratio(eye):
    A = dist.euclidean(eye[1], eye[5])
    B = dist.euclidean(eye[2], eye[4])
    C = dist.euclidean(eye[0], eye[3])
    ear = (A + B) / (2.0 * C)
    return ear

# 计算嘴部张开度 MAR
def mouth_aspect_ratio(mouth):
    A = dist.euclidean(mouth[2], mouth[10])  # 上下唇垂直距离
    B = dist.euclidean(mouth[4], mouth[8])
    C = dist.euclidean(mouth[0], mouth[6])  # 水平距离
    mar = (A + B) / (2.0 * C)
    return mar

# 警报函数（Windows）
def alert():
    frequency = 2500  # Set Frequency To 2500 Hertz
    duration = 1000  # Set Duration To 1000 ms == 1 second
    winsound.Beep(frequency, duration)

# 主程序逻辑
def main():
    cap = cv2.VideoCapture(0)

    EYE_AR_THRESH = 0.23   # EAR 阈值
    MOUTH_AR_THRESH = 0.7  # MAR 阈值
    EYE_AR_CONSEC_FRAMES = 30  # 连续闭眼帧数
    YAWN_CONSEC_FRAMES = 20    # 打哈欠持续帧数

    COUNTER_EYE = 0
    COUNTER_MOUTH = 0
    ALARM_ON = False

    while True:
        ret, frame = cap.read()
        if not ret:
            break
        gray = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY)

        rects = detector(gray, 0)

        for rect in rects:
            shape = predictor(gray, rect)
            shape = face_utils.shape_to_np(shape)

            leftEye = shape[lStart:lEnd]
            rightEye = shape[rStart:rEnd]
            mouth = shape[mStart:mEnd]

            leftEAR = eye_aspect_ratio(leftEye)
            rightEAR = eye_aspect_ratio(rightEye)
            ear = (leftEAR + rightEAR) / 2.0

            mar = mouth_aspect_ratio(mouth)

            leftEyeHull = cv2.convexHull(leftEye)
            rightEyeHull = cv2.convexHull(rightEye)
            mouthHull = cv2.convexHull(mouth)

            cv2.drawContours(frame, [leftEyeHull], -1, (0, 255, 0), 1)
            cv2.drawContours(frame, [rightEyeHull], -1, (0, 255, 0), 1)
            cv2.drawContours(frame, [mouthHull], -1, (0, 255, 0), 1)

            # 判断闭眼
            if ear < EYE_AR_THRESH:
                COUNTER_EYE += 1
                if COUNTER_EYE >= EYE_AR_CONSEC_FRAMES:
                    if not ALARM_ON:
                        ALARM_ON = True
                        alert()
                    cv2.putText(frame, "DROWSINESS ALERT!", (10, 30),
                                cv2.FONT_HERSHEY_SIMPLEX, 0.7, (0, 0, 255), 2)
            else:
                COUNTER_EYE = 0
                ALARM_ON = False

            # 判断打哈欠
            if mar > MOUTH_AR_THRESH:
                COUNTER_MOUTH += 1
                if COUNTER_MOUTH >= YAWN_CONSEC_FRAMES:
                    cv2.putText(frame, "YAWNING", (10, 60),
                                cv2.FONT_HERSHEY_SIMPLEX, 0.7, (0, 0, 255), 2)
            else:
                COUNTER_MOUTH = 0

            # 显示指标
            cv2.putText(frame, f"EAR: {ear:.2f}", (300, 30),
                        cv2.FONT_HERSHEY_SIMPLEX, 0.7, (255, 255, 255), 2)
            cv2.putText(frame, f"MAR: {mar:.2f}", (300, 60),
                        cv2.FONT_HERSHEY_SIMPLEX, 0.7, (255, 255, 255), 2)

        cv2.imshow("Fatigue Detection", frame)

        key = cv2.waitKey(1) & 0xFF
        if key == ord("q"):
            break

    cap.release()
    cv2.destroyAllWindows()

if __name__ == "__main__":
    main()

---

✅ 三、各模块详解

1️⃣ 人脸检测

使用 dlib.get_frontal_face_detector() 检测图像中的人脸区域。

rects = detector(gray, 0)

2️⃣ 关键点定位

使用预训练模型 shape_predictor_68_face_landmarks.dat 提取68个面部关键点。

shape = predictor(gray, rect)
shape = face_utils.shape_to_np(shape)

3️⃣ 特征提取

• 眼睛长宽比 EAR（Eye Aspect Ratio）

  def eye_aspect_ratio(eye):
      A = dist.euclidean(eye[1], eye[5])
      B = dist.euclidean(eye[2], eye[4])
      C = dist.euclidean(eye[0], eye[3])
      ear = (A + B) / (2.0 * C)
      return ear
  

• 嘴部长宽比 MAR（Mouth Aspect Ratio）

  def mouth_aspect_ratio(mouth):
      A = dist.euclidean(mouth[2], mouth[10])
      B = dist.euclidean(mouth[4], mouth[8])
      C = dist.euclidean(mouth[0], mouth[6])
      mar = (A + B) / (2.0 * C)
      return mar
  

4️⃣ 疲劳判断与预警

• 如果EAR小于设定阈值且持续一定帧数，则判定为“闭眼”；
• 如果MAR大于设定阈值且持续一定帧数，则判定为“打哈欠”；
• 触发蜂鸣报警。

---

✅ 四、可选增强功能

功能	描述
头部姿态估计	使用solvePnP计算头部角度，判断点头
实时日志记录	将疲劳事件写入日志文件
GUI界面	使用Tkinter或PyQt构建图形化界面
多人检测	支持多人同时检测
视频保存	将检测过程录制为视频

---

✅ 五、运行效果

• 实时显示摄像头画面
• 标注眼睛和嘴巴轮廓
• 显示EAR、MAR数值
• 当检测到闭眼或打哈欠时弹出警告框并发出声音提示

---

亦可加入深度学习模型（如CNN）提升精度，或结合红外摄像头进行夜间检测，