MULTIMODAL DEEP LEARNING APPROACH FOR HUMOR DETECTION IN BENGALI

Abstract

Humor and sarcasm detection is essential for intelligent agents to revolutionize customer service, healthcare, and other industries by enabling truly natural and empathetic in teractions. However, detecting sarcasm and humor in Bengali videos, one of the most spoken languages in the world, is significantly challenged by the language’s intricate linguistic nuances and cultural specificity. As per our knowledge, this work introduces the first publicly available multimodal dataset for Bengali humor detection, consisting of 600 annotated video clips categorized into humor, sarcasm, and normal. Context-Aware Multi-Modal Fusion Framework (CAMFusion) is proposed, integrating visual and textual features. Visual features are extracted from a Time-Distributed MobileNetV2 with 0.985 million parameters, enhanced with reverse fusion mechanism and squeeze and excitation blocks. In contrast, textual features are processed through Bi-LSTM and Bi-GRU networks with group-wise enhancement. A new Bengali Text Extraction Algorithm ensures robust text retrieval from complex video frames. CAMFusion achieves an accuracy of 90% through dynamic attention-based fusion with 13.88 million parameters, significantly out performing state-of-the-art baselines, including MobileNetV3 with XLM-R (81.10%) and TimeSformer-L with DeBERTaV3 (84.45%). This work establishes a new benchmark for Bengali humor detection, addressing a critical gap in low-resource language research and advancing multimodal computational linguistics for culturally sensitive AI applications.