Cast2Face: Assigning Character Names onto Faces in Movie with Actor-Character Correspondence

Guangyu Gao, Mengdi Xu, Jialie Shen, Huadong Ma, Shuicheng Yan

Research output: Contribution to journalArticlepeer-review


Automatically identifying characters in movies has attracted researchers' interest and led to several significant and interesting applications. However, due to the vast variation in character appearance as well as the weakness and ambiguity of available annotation, it is still a challenging problem. In this paper, we investigate this problem with the supervision of actor-character name correspondence provided by the movie cast. Our proposed framework, namely, Cast2Face, is featured by: 1) we restrict the assigned names within the set of character names in the cast; 2) for each character, by using the corresponding actor and movie name as keywords, we retrieve from the Google image search and get a group of face images to form the gallery set; 3) the probe face tracks in the movie are then identified as one of the actors by a robust kernel multitask joint sparse representation and classification method; and 4) the conditional random field model with consideration of the constraints between face tracks is introduced to enhance the final labeling. Finally, the assigned actor name of a face track is then mapped to the character name based on the cast again. Besides face naming, we further apply the proposed method to spotlight the summarization of a particular actor in his/her movies. We conduct extensive experiments and empirical evaluations on several feature-length movies to demonstrate the satisfying performance of our method.

Original languageEnglish
Pages (from-to)2299-2312
Number of pages14
JournalIEEE Transactions on Circuits and Systems for Video Technology
Issue number12
Publication statusPublished - 1 Dec 2016
Externally publishedYes


Dive into the research topics of 'Cast2Face: Assigning Character Names onto Faces in Movie with Actor-Character Correspondence'. Together they form a unique fingerprint.

Cite this