|Appears in Collections:||Computing Science and Mathematics Conference Papers and Proceedings|
|Peer Review Status:||Refereed|
|Title:||A novel brain-inspired compression-based optimised multimodal fusion for emotion recognition|
|Citation:||Gogate M, Adeel A & Hussain A (2017) A novel brain-inspired compression-based optimised multimodal fusion for emotion recognition In: 2017 IEEE Symposium Series on Computational Intelligence (SSCI). 2017 IEEE Symposium Series on Computational Intelligence (SSCI), Piscataway, NJ, USA. Piscataway, NJ, USA: IEEE. http://ieeexplore.ieee.org/document/8285377/; https://doi.org/10.1109/SSCI.2017.8285377.|
Towards visually-driven speech enhancement for cognitively-inspired multi-modal hearing-aid devices
|Conference Name:||2017 IEEE Symposium Series on Computational Intelligence (SSCI)|
|Conference Location:||Piscataway, NJ, USA|
|Abstract:||The curse of dimensionality is a well-established phenomenon. However, the properties of high dimensional data are often poorly understood and overlooked during the process of data modelling and analysis. Similarly, how to optimally fuse different modalities is still a big research question. In this paper, we addressed these challenges by proposing a novel two level brain-inspired compression based optimised multimodal fusion framework for emotion recognition. In the first level, the framework extracts the compressed and optimised multimodal features by applying a deep convolutional neural network (CNN) based compression on each modality (i.e. audio, text, and visuals). The second level simply concatenates the extracted optimised and compressed features for classification. The performance of the proposed approach with two different compression levels (i.e. 78% and 98%) is compared with late fusion (class level- 1 dimension, class probabilities level-4 dimension) and early fusion (feature level-72000 dimension). The simulation results and critical analysis have demonstrated up to 10% and 5% performance improvement as compared to the state-of-the-art support vector machine (SVM) and long-short-term memory (LSTM) based multimodal emotion recognition systems respectively. We hypothesise that there exist an optimal level of compression at which optimised multimodal features could be extracted from each modality, which could lead to a significant performance improvement.|
|Status:||VoR - Version of Record|
|Rights:||The publisher does not allow this work to be made publicly available in this Repository. Please use the Request a Copy feature at the foot of the Repository record to request a copy directly from the author. You can only request a copy if you wish to use this work for your own research or private study.|
|08285377.pdf||Fulltext - Published Version||616.89 kB||Adobe PDF||Under Embargo until 2999-12-01 Request a copy|
Note: If any of the files in this item are currently embargoed, you can request a copy directly from the author by clicking the padlock icon above. However, this facility is dependent on the depositor still being contactable at their original email address.
This item is protected by original copyright
Items in the Repository are protected by copyright, with all rights reserved, unless otherwise indicated.
If you believe that any material held in STORRE infringes copyright, please contact firstname.lastname@example.org providing details and we will remove the Work from public display in STORRE and investigate your claim.