Behera, Ardhendu, Keidel, Alex and Debnath, Bappaditya (2018) Context-driven Multi-stream LSTM (M-LSTM) for Recognizing Fine-Grained Activity of Drivers. Lecture Notes in Computer Sciences (LNCS) - Pattern Recognition. ISSN 0302-9743 (In Press)

Preview	Text 0050.pdf - Supplemental Material Available under License Creative Commons Attribution Non-commercial No Derivatives. Download (3MB) | Preview
Preview	Text Supplementary_figures_plots.pdf Available under License Creative Commons Attribution Non-commercial No Derivatives. Download (1MB) | Preview
	Video Distracted-Drivers_dataset[1]_video_demo.mp4 - Supplemental Material Available under License Creative Commons Attribution Non-commercial No Derivatives. Download (17MB)

Abstract

Automatic recognition of in-vehicle activities has significant impact on the next generation intelligent vehicles. In this paper, we present a novel Multi-stream Long Short-Term Memory (M-LSTM) network for recognizing driver activities. We bring together ideas from recent works on LSTMs, transfer learning for object detection and body pose by exploring the use of deep convolutional neural networks (CNN). Recent work has also shown that representations such as hand-object interactions are important cues in characterizing human activities. The proposed M-LSTM integrates these ideas under one framework, where two streams focus on appearance information with two different levels of abstractions. The other two streams analyze the contextual information involving configuration of body parts and body-object interactions. The proposed contextual descriptor is built to be semantically rich and meaningful, and even when coupled with appearance features it is turned out to be highly discriminating. We validate this on two challenging datasets consisting driver activities.

Item Type:	Article
Additional Information:	Conference proceedings from 40th German Conference on Pattern Recognition (GCPR) 9th to 12th October 2018 Stuttgart Germany
Uncontrolled Keywords:	Multi-stream Long Short-Term Memory (M-LSTM), Deep Learning, Transfer Learning, Autonomous Vehicles, In-vehicle Activity Monitoring, Body pose, Modelling body-objects interactions
Subjects:	Q Science > QA Mathematics > QA75 Electronic computers. Computer science
Divisions:	Computing and Information Systems
Related URLs:	Organisation
Date Deposited:	25 Oct 2018 13:10
URI:	http://repository.edgehill.ac.uk/id/eprint/10782

Archive staff only

Item control page