Project: Regime-Aware Multimodal Forecasting: Integrating LLM Semantic Context with Financial Time Series
Description
Background & Motivation
Financial markets operate through distinct regimes, which are persistent states characterised by qualitatively different dynamics in volatility, correlation structure, liquidity, and the transmission of information to prices. A low-volatility bull market, a rate-hiking cycle, and a systemic liquidity crisis are not merely different points on a continuous distribution: they reflect fundamentally different operating modes of the market mechanism. Forecasting models that treat financial time series as a single stationary process are misspecified at the level of the data-generating process.
Classical regime-switching models, such as Hamilton's Markov-switching model and hidden Markov models, provide principled frameworks but are limited in expressiveness and cannot easily integrate heterogeneous information. In practice, experienced market participants switch their mental models when they perceive a regime change, primarily by interpreting qualitative signals such as central bank rhetoric, fiscal policy announcements, and geopolitical developments. This qualitative, contextual reasoning is precisely the domain where large language models have demonstrated remarkable capability.
Existing LLM-based financial forecasting approaches treat the language model either as a direct predictor or as a feature extractor, without explicitly modelling regime dynamics or coupling the LLM's contextual understanding to a dedicated numerical time series model. This project addresses that gap: it develops a regime-aware multimodal architecture where an LLM acts as a semantic regime identifier, and a structured time series model handles quantitative pattern recognition, with the two coupled through a gating mechanism.
Research Objectives
The following objectives represent the broader research landscape for this project. The precise thesis scope will be defined collaboratively with the supervisor based on the student's background and interests. Students are not expected to address all objectives.
1. Review the literature on regime-switching models, multimodal fusion architectures, and LLM-based financial reasoning.
2. Design and implement a two-component architecture coupling an LLM regime classifier with a numerical time series encoder.
3. Explore approaches for regime-conditioned fusion, starting with simple gating mechanisms and extending as appropriate.
4. Evaluate the model across a range of market conditions, assessing forecasting accuracy and regime classification quality.
5. Analyse which information sources the model relies on in different regimes, providing interpretable insights.
Proposed Approach
Stage 1: LLM-based regime identification
Prompt or fine-tune an instruction-tuned LLM to classify market regime from a context window containing recent macroeconomic releases, central bank communications, and summary price statistics. Regime categories are defined through a combination of quantitative clustering and economic intuition.
Stage 2: Regime-conditioned multimodal fusion
Encode numerical time series patches and textual context in parallel. A gating network, conditioned on the LLM regime estimate, produces a weighted combination of the two representation streams for the forecasting head. Explore how semantic routing differs from standard end-to-end learned routing.
Stage 3: Evaluation and interpretability
Evaluate on a multi-asset dataset covering at least one major structural transition. Analyse gating weight distributions across regimes and the marginal contribution of the text modality in each state. Compare against unimodal baselines and static fusion methods.
Possible Thesis Directions
- LLM-based market regime classification as a standalone study, comparing against statistical baselines.
-A focused fusion experiment on a single asset class, such as equities, with two or three regime categories.
-Interpretability analysis: which text signals are most informative for regime identification, and does this align with practitioner intuition?
-Extending the architecture to multiple asset classes and studying how cross-asset regime dynamics interact.
-Comparing different gating mechanisms, such as hard switching, soft routing, and mixture-of-experts, under the same regime signal.
The exact direction will be determined jointly. Students interested in extending toward publication may explore the full multi-asset evaluation and interpretability analysis.
Students with strong interests, relevant hands-on experience, and motivation for a publishable thesis are encouraged. For more details, please contact the supervisors (s.deng@tue.nl or y.deng2@tue.nl).
References
Nie, Y., et al. ``A Time Series is Worth 64 Words: Long-Term Forecasting with Transformers (PatchTST).'' ICLR 2023.
Liu, Y., et al. ``iTransformer: Inverted Transformers Are Effective for Time Series Forecasting.'' ICLR 2024.
Jin, M., et al. ``Time-LLM: Time Series Forecasting by Reprogramming Large Language Models.'' ICLR 2024.
Zhang, W., et al. ``FinAgent: A Multimodal Foundation Agent for Financial Trading: Tool-Augmented, Diversified, and Generalist.''.
Lee S., et al. ``FinSTaR: Towards Financial Reasoning with Time Series Reasoning Models.'' arXiv 2026.
Lee, G., et al. ``TimeCAP: Learning to Contextualize, Augment, and Predict Time Series Events with Large Language Model Agents.'' AAAI 2025.
Details
- Supervisor
-
Amy Deng
- Secondary supervisor
-
Yuanyuan Deng
- Interested?
- Get in contact