Novel symbolic and machine-learning approaches for text-based and multimodal sentiment analysis

Abstract

Emotions and sentiments play a crucial role in our everyday lives. They aid decision-making, learning, communication, and situation awareness in human-centric environments. Over the past two decades, researchers in artificial intelligence have been attempting to endow machines with cognitive capabilities to recognize, infer, interpret and express emotions and sentiments. All such efforts can be attributed to affective computing, an interdisciplinary field spanning computer science, psychology, social sciences and cognitive science. Sentiment analysis and emotion recognition has also become a new trend in social media, avidly helping users understand opinions being expressed on different platforms in the web. In this thesis, we focus on developing novel methods for text-based sentiment analysis. As an application of the developed methods, we employ them to improve multimodal polarity detection and emotion recognition. Specifically, we develop innovative text and visual-based sentiment-analysis engines and use them to improve the performance of multimodal sentiment analysis. We begin by discussing challenges involved in both text-based and multimodal sentiment analysis. Next, we present a number of novel techniques to address these challenges. In particular, in the context of concept-based sentiment analysis, a paradigm gaining increasing interest recently, it is important to identify concepts in text; accordingly, we design a syntaxbased concept-extraction engine. We then exploit the extracted concepts to develop conceptbased affective vector space which we term, EmoSenticSpace. We then use this for deep learning-based sentiment analysis, in combination with our novel linguistic pattern-based affective reasoning method termed sentiment flow. Finally, we integrate all our text-based techniques and combine them with a novel deep learning-based visual feature extractor for multimodal sentiment analysis and emotion recognition. Comparative experimental results using a range of benchmark datasets have demonstrated the effectiveness of the proposed approach.