Market sentiment analysis using multimodal transformers: Integrating earnings calls, social media, and technical indicators

This research introduces a novel multimodal transformer architecture for comprehensive market sentiment analysis that integrates traditionally disparate data sources: earnings call transcripts, social media content, and technical market indicators. Our approach addresses the limitations of unimodal sentiment analysis systems by leveraging the complementary nature of textual, social, and quantitative data streams. We developed a specialized transformer-based model with modality-specific encoders and cross-modal attention mechanisms, achieving superior performance compared to unimodal baselines. Experimental results on a diverse financial dataset demonstrate significant improvements in sentiment classification accuracy (87.2% vs. 78.4% baseline) and directional market prediction (74.3% vs. 63.8% baseline). The model shows particular strength in detecting subtle sentiment shifts during market volatility periods, with a 9.3% performance improvement over traditional methods. Our findings illustrate that integrating multiple information channels enables more nuanced market sentiment understanding, potentially offering valuable insights for financial decision-making and risk management.