FaceWarehouse: a 3D Facial Expression Database for Visual Computing
Introduction
FaceWarehouse is a database of 3D facial expressions for visual computing applications. Using a Kinect RGBD camera, we captured 150 individuals aged 7-80 from various ethnic backgrounds. For each person, we captured the RGBD data of her different expressions, including the neutral expression and 19 other expressions such as mouth-opening, smile, kiss, etc. For every RGBD raw data record, a set of facial feature points on the color image such as eye corners, mouth contour and the nose tip are automatically localized, and manually adjusted if better accuracy is required. We then deform a template facial mesh to fit the depth data as closely as possible while matching the feature points on the color image to their corresponding points on the mesh. From these fitted face meshes, we construct a set of individual-specific expression blendshapes for each person.
NEW: Thanks to the MICA team, the FLAME fittings of FaceWarehouse is available now. Please follow the same application process described below.
Obtaining the data
We make all RGBD scans,
feature points and the fitted meshes available for academic research purposes. However,
as human face is very personal, we only send the data to approved researchers.
To obtain a copy, please send an email to kunzhou at acm doc org stating
(1) your name, title, affiliation (if you are a student, please ask your advisor to contact us)
(2) your intended use of the data
(3) a statement saying that you accept the following terms of licensing:
The rights to copy, distribute, and use the data (including the RGBD images
and 3D models) you are being given access to are under the control of Kun Zhou,
head of the Graphics and Parallel Systems Lab, Zhejiang University. You are
hereby given permission to copy this data in electronic or hardcopy form for
your own scientific use and to distribute it for scientific use to colleagues
within your research group. Inclusion of rendered images or video made from
this data in a scholarly publication (printed or electronic) is also permitted.
In this case, credit must be given to the publication: FaceWarehouse:
a 3D Facial Expression Database for Visual Computing.
However, the data may not be included in the electronic version of a
publication, nor placed on the Internet. These restrictions apply to any
representations (other than images or video) derived from the data, including
but not limited to simplifications, remeshing, and
the fitting of smooth surfaces. The making of physical replicas this data is prohibited, and the data may not be distributed to students
in connection with a class. For any other use, including distribution outside
your research group, written permission is required from Kun Zhou. Any
commercial use of the data is prohibited. Commercial use includes but is not
limited to sale of the data, derivatives, replicas, images, or video, inclusion
in a product for sale, or inclusion in advertisements (printed or electronic),
on commercially-oriented web sites, or in trade shows.
File formats
We store the data for different persons into into different directories. In each directory "Tester_xx", we store the RGBD data, the labelled 74 landmarks, the reconstructed meshes, and user-specific blendshapes for this person.
- Tester_1: folder name with person 1
- TrainingPose
- pose_0.pose - pose_20.pose: raw
data from the Kinect RGBD camera
- pose_0.obj - pose_20.obj:
reconstructed face meshes
- pose_0.land - pose_20.land: labelled 2D landmarks for the first frame of raw
data
- Blendshape
- shape.bs: expression blendshape binary file
- shape_0.obj - shape_46.obj:
expression blendshapes
- Tester_2: Folder name with person 2...
- Tester_n: Folder name with person n...
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Reference
Cao Chen, Yanlin Weng, Shun Zhou, Yiying Tong, Kun Zhou: "FaceWarehouse: a 3D Facial Expression Database for Visual Computing", IEEE Transactions on Visualization and Computer Graphics, 20(3): 413-425, 2014, [PDF] [Video]
Related Work
Cao Chen, Hongzhi Wu, Yanlin Weng, Tianjia Shao, Kun Zhou: "Real-time Facial Animation with Image-based Dynamic Avatars", ACM Transactions on Graphics (SIGGRAPH), 2016. [PDF] [Video]
Cao Chen, Derek Bradley, Kun Zhou, Thabo Beeler: "Real-Time High-Fidelity Facial Performance Captures", ACM Transactions on Graphics (SIGGRAPH), 2015. [PDF]
Cao Chen, Qiming Hou, Kun Zhou: "Displaced Dynamic Expression Regression for Real-time Facial Tracking and Animation", ACM Transactions on Graphics (SIGGRAPH), 2014. [PDF] [Video]
Cao Chen, Yanlin
Weng, Steve Lin, Kun Zhou: "3D Shape Regression
for Real-time Facial Animation", ACM
Transactions on Graphics (SIGGRAPH), 32(4): 41, 2013, [PDF] [Video]
Yanlin Weng, Cao Chen, Qiming Hou, Kun Zhou: "Real-time Facial Animation on Mobile Devices", Graphical Models, 76(3): 172-179, 2014. [PDF] [Video]