tensorflow开发流程——表情分析

前期准备

在利用tensorflow做表情分析时，需要很多处理模块进行辅助，由于对这些模块并不熟悉，因此，本文中会针对每个模块进行测试。

CK+数据提取与label标识

本文采用CK+作为数据库，网上可以下载到（自行google）。该数据库是一个基于视频帧的表情库。目前，我们第一版本的表情分析，利用图片作为输入，对这个表情库进行提取，每个人的每种表情提取一张图片并表上标签，python代码如下：

import os
list_tuple=[]
list_filepath=[]
list_label=[]
for root, dir, files in os.walk('/home/beast/Code/emotiondata/cohn-kanade/'):
files_num = len(files)
if files_num > 0:
file_fullpath=os.path.join(root,files[files_num/2])
label = int(file_fullpath.split('_')[1])
list_tuple.append([file_fullpath, label])
list_filepath.append(file_fullpath)
list_label.append(label)
print list_filepath

cohn-kanade即为下载的CK+数据库解压后的存放位置。

opencv读取数据

• opencv读取图像数据与显示：

import cv2
im = cv2.imread(list_filepath[0])
cv2.namedWindow('emotion')
cv2.imshow('emotion',im)
cv2.waitKey(-1)

显示结果如下：

• opencv数据转换： 
  opencv读取数据后，数据的排布格式如下：

import cv2
im = cv2.imread(list_filepath[0])
print im.size
print im.shape
print im

显示结果如下： 

opencv读取的图像的格式.png 
图中是rgb的图像，490行640列，每个像素的rgb通道是连续排列的。

• 利用opencv的数据是可以直接初始化tensorflow中的tensor的，但是能不能直接使用，这是后话，测试代码如下：

import tensorflow as tf

flags = tf.app.flags
FLAGS = flags.FLAGS
flags.DEFINE_string('summaries_dir', './tf_logs', 'Summaries directory')

t1=tf.constant(im)
t2=tf.Variable(im)
print t1
print t2

with tf.Session() as sess:
    sess.run(tf.initialize_all_variables())
    merged = tf.merge_all_summaries()
    train_writer = tf.train.SummaryWriter(FLAGS.summaries_dir + '/',                                      sess.graph)

显示结果如下： 

• 利用numpy将opencv提取出来的数据，针对某一个分量（如R），单独提取出来，代码如下：

import numpy
import cv2

im_r = im[:,:,0].astype(numpy.float32)/255

print im_r.shape
print im_r

numpy的数据格式，也是可以直接用来初始化tensorflow变量的。

python中glob的使用

可以用来提取文件：

import glob

for i in glob.glob('/home/beast/Code/emotiondata/cohn-kanade/S010/001/*.png'):
    print i

python中zip的使用

zip用来合并两个list,实例代码如下：

python dlib的使用

import sys
import os
import dlib
import glob
from skimage import io

predictor_path = '/home/beast/Code/model/shape_predictor_68_face_landmarks.dat'
faces_folder_path = '/home/beast/Code/Pic/haijun'

detector = dlib.get_frontal_face_detector()
predictor = dlib.shape_predictor(predictor_path)
win = dlib.image_window()

for f in glob.glob(os.path.join(faces_folder_path, "*.png")):
    print("Processing file: {}".format(f))
    img = io.imread(f)

    win.clear_overlay()
    win.set_image(img)

    # Ask the detector to find the bounding boxes of each face. The 1 in the
    # second argument indicates that we should upsample the image 1 time. This
    # will make everything bigger and allow us to detect more faces.
    dets = detector(img, 1)
    print("Number of faces detected: {}".format(len(dets)))
    for k, d in enumerate(dets):
        print("Detection {}: Left: {} Top: {} Right: {} Bottom: {}".format(
            k, d.left(), d.top(), d.right(), d.bottom()))
        # Get the landmarks/parts for the face in box d.
        shape = predictor(img, d)
        print("Part 0: {}, Part 1: {} ...".format(shape.part(0),
                                                  shape.part(1)))
        # Draw the face landmarks on the screen.
        win.add_overlay(shape)

    win.add_overlay(dets)
    dlib.hit_enter_to_continue()

opencv和dlib结合使用

经验证在python中，opencv和dlib的数据结构是可以通用的，本例中，利用opencv开启摄像头采集数据，利用dlib进行人脸的检测以及人脸关键点的检测。该示例的目的是能够在视频中检测出人脸并作为机器学习的输入，python代码如下：

• 单张图片的dlib检测代码如下：

import sys
import os
import dlib
import glob
from skimage import io

predictor_path = '/home/beast/Code/model/shape_predictor_68_face_landmarks.dat'
faces_folder_path = '/home/beast/Code/Pic/haijun'

detector = dlib.get_frontal_face_detector()
predictor = dlib.shape_predictor(predictor_path)
win = dlib.image_window()

for f in glob.glob(os.path.join(faces_folder_path, "*.png")):
    print("Processing file: {}".format(f))
    img = io.imread(f)

    win.clear_overlay()
    win.set_image(img)

    # Ask the detector to find the bounding boxes of each face. The 1 in the
    # second argument indicates that we should upsample the image 1 time. This
    # will make everything bigger and allow us to detect more faces.
    dets = detector(img, 1)
    print("Number of faces detected: {}".format(len(dets)))
    for k, d in enumerate(dets):
        print("Detection {}: Left: {} Top: {} Right: {} Bottom: {}".format(
            k, d.left(), d.top(), d.right(), d.bottom()))
        # Get the landmarks/parts for the face in box d.
        shape = predictor(img, d)
        print("Part 0: {}, Part 1: {} ...".format(shape.part(0),
                                                  shape.part(1)))
        # Draw the face landmarks on the screen.
        win.add_overlay(shape)

    win.add_overlay(dets)
    dlib.hit_enter_to_continue()

显示结果如下： 

dlib对图片的人脸关键点定位.png

• 基于视频的人脸关键点检测代码如下：

import numpy as np
import cv2
import cv2.cv as cv
from video import create_capture
from common import clock, draw_str
import dlib

predictor_path = '/home/beast/Code/model/shape_predictor_68_face_landmarks.dat'
detector = dlib.get_frontal_face_detector()
predictor = dlib.shape_predictor(predictor_path)
win = dlib.image_window()

cam = create_capture(0, fallback='synth:bg=../cpp/lena.jpg:noise=0.05')
while True:
    ret, img = cam.read()

    dets = detector(img, 1)
    print("Number of faces detected: {}".format(len(dets)))
    win.clear_overlay()
    for k, d in enumerate(dets):
        print("Detection {}: Left: {} Top: {} Right: {} Bottom: {}".format(
            k, d.left(), d.top(), d.right(), d.bottom()))
        # Get the landmarks/parts for the face in box d.
        shape = predictor(img, d)
        print("Part 0: {}, Part 1: {} ...".format(shape.part(0),
                                                  shape.part(1)))
        # Draw the face landmarks on the screen.
        win.add_overlay(shape)
    win.set_image(img)
    win.add_overlay(dets)
    dlib.hit_enter_to_continue()

    if 0xFF & cv2.waitKey(5) == 27:
        break