Human Consistency Evaluation of Static Video Summaries

Kannappa, Sivapriyaa, Liu, Yonghuai and Tiddeman, Bernie (2018) Human Consistency Evaluation of Static Video Summaries. Multimedia Tools and Applications. pp. 1-26. ISSN 1380-7501 DOI

[img] Text
MTAP18_sik2.pdf - Accepted Version
Restricted to Repository staff only until 20 October 2019.
Available under License Creative Commons Attribution Non-commercial No Derivatives.

Download (4MB) | Request a copy


Automatic video summarization aims to provide brief representa- tion of videos. Its evaluation is quite challenging, usually relying on comparison with user summaries. This study views it in a diferent perspective in terms of verifying the consistency of user summaries, as the outcome of video sum- marization is usually judged based on them. We focus on human consistency evaluation of static video summaries in which the user summaries are evalu- ated among themselves using the consistency modelling method we proposed recently. The purpose of such consistency evaluation is to check whether the users agree among themselves. The evaluation is performed on diferent pub- licly available datasets. Another contribution lies in the creation of static video summaries from the available video skims of the SumMe datatset. The results show that the level of agreement varies signigcantly between the users for the selection of key frames, which denotes the hidden challenge in automatic video summary evaluation. Moreover, the maximum agreement level of the users for a certain dataset, may indicate the best performance that the automatic video summarization techniques can achieve using that dataset.

Item Type: Article
Uncontrolled Keywords: Video Summarization, Keyframe Extraction, Performance Evaluation, Consistency modelling, User Consistency
Subjects: Q Science > QA Mathematics > QA75 Electronic computers. Computer science
Divisions: Computing and Information Systems
Date Deposited: 30 Oct 2018 13:30

Archive staff only

Item control page Item control page