Back to Projects

Decoding the Role of Emojis in Bangla Sarcasm and Offensiveness

A Multitask, Emoji-Intent-Aware Approach with a New Sarcastic-Offense Class

RESEARCH / NLPBuilt during: NLP Researcher & Machine Learning Engineer
Tags -
AI/MLNLPPyTorchPythonTransformersBangla NLPLIMEInterpretabilityDeep LearningResearch
Source

View the source code, research documentation, and benchmarks:

View Source Code

Project Overview

In Bangla online comments, emojis often reverse or intensify the literal meaning of text. Prior work treats sarcasm and offensiveness as separate tasks, ignores emojis, and cannot represent comments that are both sarcastic and offensive (sarcastic offense). Furthermore, no prior work quantifies how much emojis contribute compared with words.

This project introduces a multitask, emoji-intent-aware approach evaluated on a newly constructed Bangla corpus of 3,000 social media comments featuring rich emojis and Bangla-English code-mixing, establishing a new Sarcastic-Offense class.

Interpretability is analyzed via LIME applied to the model's actual emoji-expanded input to quantify token-level attribution. Controlled ablation experiments validate the emoji contribution using fold-level paired t-tests, sample-level one-sided McNemar tests on shared hold-out sets, bootstrap 95% confidence intervals, and label-permutation tests.

Key Features

  • Novel Sarcastic-Offense Composite Class — Introduces a new category capturing comments that are simultaneously sarcastic and offensive, bridging a critical gap in emotion and toxicity analysis.
  • Curated 3,000-Comment Dataset — A newly annotated Bangla corpus of 3,000 social media comments with emojis and Bangla-English code-mixing.
  • Emoji vs. Word Attribution via LIME — LIME explainability applied to the emoji-expanded input to compare mean absolute attribution of emoji and word tokens across all test samples.
  • Controlled No-Emoji Ablation Study — Trained on identical folds, seed, and hyperparameters to empirically isolate and prove the contribution of emoji semantics.
  • Rigorous Statistical Significance Testing — Fold-level paired t-tests, sample-level one-sided McNemar tests, bootstrap 95% confidence intervals, and label-permutation tests against chance.

Technical Highlights

  • Multitask Emoji-Intent Architecture — Jointly learns sarcasm, offensiveness, and the composite sarcastic-offense class using transformer backbones.
  • Token-Level Interpretability with LIME — Quantifies how individual emoji tokens alter decision boundaries and sentiment polarity compared to word tokens.
  • Hypothesis Testing & Permutation Validation — Validates model performance gains with paired t-tests and permutation testing, proving findings are not artifacts of seed variation.

Challenges & Solutions

Challenge: Research Gap Addressed (Emoji-Driven Semantic Inversion): In Bangla online comments, emojis often reverse or intensify the literal meaning of text, yet prior work treated sarcasm and offensiveness separately and ignored emojis.
Solution: Constructed a multitask, emoji-intent-aware framework that introduces the new 'Sarcastic Offense' class for comments that are simultaneously sarcastic and offensive.
Result: Effectively captures nuanced, toxic sarcasm that traditional text-only classifiers fail to detect.
Challenge: Quantifying Emoji vs. Word Contribution: No prior work quantified how much emojis contribute to predictions compared with textual words.
Solution: Applied LIME to the model's actual emoji-expanded input for both sarcasm and offensiveness, calculating mean absolute attribution across all emoji-bearing test samples.
Result: Empirically proved and quantified the substantial decision weight exerted by emoji tokens over word tokens.
Challenge: Measurement Rigor & Overfitting to Random Folds: Ensuring that emoji-driven performance gains are statistically significant rather than random chance.
Solution: Trained a controlled no-emoji ablation on identical folds, seed, and hyperparameters, evaluated via fold-level paired t-tests, McNemar tests, bootstrap 95% CIs, and permutation tests.
Result: Statistically verified (p < 0.05) that emoji inclusion directly improves classification accuracy.

Tech Stack Breakdown

frontend

Matplotlib, Seaborn (Attribution & Performance Plots)

backend

Python, PyTorch, Hugging Face Transformers, LIME

database

Pandas, NumPy, Scikit-learn (Corpus & Benchmarks)

tools

Git, Jupyter Notebook, LaTeX, Google Colab Pro