Please use this identifier to cite or link to this item: http://hdl.handle.net/1893/26943
Appears in Collections:Computing Science and Mathematics Conference Papers and Proceedings
Peer Review Status: Refereed
Author(s): Gogate, Mandar
Adeel, Ahsan
Hussain, Amir
Contact Email: ahu@cs.stir.ac.uk
Title: A novel brain-inspired compression-based optimised multimodal fusion for emotion recognition
Citation: Gogate M, Adeel A & Hussain A (2017) A novel brain-inspired compression-based optimised multimodal fusion for emotion recognition In: 2017 IEEE Symposium Series on Computational Intelligence (SSCI). 2017 IEEE Symposium Series on Computational Intelligence (SSCI), Piscataway, NJ, USA. Piscataway, NJ, USA: IEEE. http://ieeexplore.ieee.org/document/8285377/; https://doi.org/10.1109/SSCI.2017.8285377.
Towards visually-driven speech enhancement for cognitively-inspired multi-modal hearing-aid devices
EP/M026981/1
Issue Date: 2017
Conference Name: 2017 IEEE Symposium Series on Computational Intelligence (SSCI)
Conference Location: Piscataway, NJ, USA
Abstract: The curse of dimensionality is a well-established phenomenon. However, the properties of high dimensional data are often poorly understood and overlooked during the process of data modelling and analysis. Similarly, how to optimally fuse different modalities is still a big research question. In this paper, we addressed these challenges by proposing a novel two level brain-inspired compression based optimised multimodal fusion framework for emotion recognition. In the first level, the framework extracts the compressed and optimised multimodal features by applying a deep convolutional neural network (CNN) based compression on each modality (i.e. audio, text, and visuals). The second level simply concatenates the extracted optimised and compressed features for classification. The performance of the proposed approach with two different compression levels (i.e. 78% and 98%) is compared with late fusion (class level- 1 dimension, class probabilities level-4 dimension) and early fusion (feature level-72000 dimension). The simulation results and critical analysis have demonstrated up to 10% and 5% performance improvement as compared to the state-of-the-art support vector machine (SVM) and long-short-term memory (LSTM) based multimodal emotion recognition systems respectively. We hypothesise that there exist an optimal level of compression at which optimised multimodal features could be extracted from each modality, which could lead to a significant performance improvement.
Status: VoR - Version of Record
Rights: The publisher does not allow this work to be made publicly available in this Repository. Please use the Request a Copy feature at the foot of the Repository record to request a copy directly from the author. You can only request a copy if you wish to use this work for your own research or private study.
URL: http://ieeexplore.ieee.org/document/8285377/

Files in This Item:
File Description SizeFormat 
08285377.pdfFulltext - Published Version616.89 kBAdobe PDFUnder Embargo until 2999-12-01    Request a copy

Note: If any of the files in this item are currently embargoed, you can request a copy directly from the author by clicking the padlock icon above. However, this facility is dependent on the depositor still being contactable at their original email address.



This item is protected by original copyright



Items in the Repository are protected by copyright, with all rights reserved, unless otherwise indicated.

If you believe that any material held in STORRE infringes copyright, please contact library@stir.ac.uk providing details and we will remove the Work from public display in STORRE and investigate your claim.