Code-Mixed Sentiment Analysis

Feb 22, 2025
Follow
Photo taken from internet

Sentiment analysis on code-mixed text is essential for understanding user opinions, particularly in regions where people frequently switch between languages in their conversations. Traditional sentiment analysis models face difficulty in handling such data due to the complexity introduced by the mixture of languages and the lack of large labeled datasets. This project aimed to develop a robust model for detecting and classifying sentiment (positive, negative, neutral) in code-mixed social media text. A dataset was collected from social media platforms like Facebook and Twitter, and preprocessing steps included tokenization, language identification, and normalization of code-mixed text. The hybrid model combined traditional machine learning algorithms like Support Vector Machines (SVM) with deep learning techniques like Bidirectional Long Short-Term Memory (BiLSTM). The project also leveraged multilingual embeddings to capture the semantics of the words in different languages. One of the key challenges was handling context-sensitive elements like sarcasm and slang. The results showed improved accuracy by fine-tuning transformer models like BERT and multilingual BERT, significantly enhancing the performance of sentiment classification on code-mixed data.

Social media NLP Sentiment Analysis Code-Mixing cmsa Tamil-English Kannada-English Malayalam-English
Supriya Chanda
Supriya Chanda
Research Scholar (2018-2024)

Supriya Chanda (pronounced as Supriyo), completed his Ph.D in the Department of Computer Science and Engineering, IIT (BHU), Varanasi. He did his research under the guidance of Dr. Sukomal Pal at the Information retrieval lab.

Publications

Sentiment analysis of code-mixed Dravidian languages leveraging pretrained model and word-level language tag
The exponential growth of social media data in the era of Web 2.0 has necessitated advanced techniques for sentiment analysis. While sentiment analysis in monolingual datasets has received significant attention that in code-mixed datasets still need to be studied more. Code-mixed data often contain a mixture of monolingual content (might be in transliterated form), single-script but multilingual content, and multi-script multilingual content. This paper explores the issue from three important angles. What will be the best strategy to deal with the data for sentiment detection? Whether to train the classifier with the whole of the dataset or only with the pure code-mixed subset from the dataset? How much important is the language identification (LID) for the task? If LID is to be done, how, and when will it be used to yield the best performance? We explore the questions in the light of three datasets of Tamil–English, Kannada–English, and Malayalam–English YouTube social media comments. Our solution incorporated mBERT and an optional LID module. We report our results using a set of metrics like precision, recall, score, and accuracy. The solutions provide considerable performance gain and some interesting insights for sentiment analysis from code-mixed data.
Supriya Chanda, Anshika Mishra, Sukomal Pal
Cite Project DOI