Advanced Time Series Analysis: From probabilistic to foundational models

Advanced Time Series Analysis: From Probabilistic to Foundational Models

Time series analysis studies data that change as a function of time, such as stock market prices, weather patterns, or household electricity consumption.  This seminar covers advanced techniques for analyzing time series, starting with probabilistic methods and progressing to state-of-the-art deep learning approaches, including neural architectures and foundation models. We will also explore connections between time series and other modalities, such as text and images/videos, to offer a comprehensive view of the field. The aim is for students to critically assess existing methods, understand their strengths and limitations, and identify potential directions for future research.

The course begins with three introductory lectures to establish foundational concepts, followed by three main blocks on time series analysis, covering

• Block 1: Explicit Modeling of Time (Traditional Approaches)

• Block 2: Implicit Modeling of Time (Data-Driven Approaches)

• Block 3: Foundation Models and Task-Agnostic Approaches

Each block includes two paper sessions and a panel discussion, as detailed below. Student evaluation is based on their presentations, active participation in discussions, and a final report summarizing the seminar topics, offering critical analysis, identifying limitations, and suggesting potential research directions.

 

Day	Block	Content
16/04/2025	Presentation	Short session on seminar organization
23/04/2025	Lecture 1	Introduction to Time Series Analysis
30/04/2025	Lecture 2	Probabilistic Foundations for Time Series I
07/05/2025	Lecture 3	Probabilistic Foundations for Time Series II
14/05/2025	Block 1 - Explicit Modeling of Time	2 Student Presentation + Q&A
21/05/2025	Block 1 - Explicit Modeling of Time	2 Student Presentations + Q&A
28/05/2025	Block 1 - Explicit Modeling of Time	Round Table
04/06/2025	Block 2 - Implicit Modeling of Time	2 Student Presentations + Q&A
11/06/2025	Block 2 - Implicit Modeling of Time	2 Student Presentations + Q&A
18/06/2025	Block 2 - Implicit Modeling of Time	Round Table
25/06/2025	Block 3 - Foundation Models and Task-Agnostic Approaches	2 Student Presentations + Q&A
02/07/2025	Block 3 - Foundation Models and Task-Agnostic Approaches	2 Student Presentations + Q&A
09/07/2025	Block 3 - Foundation Models and Task-Agnostic Approaches	Round Table

 

Date and time:  Weekly.  Wednesday 12:30-14:00

Location:  SR 4 (U16) in E 2.5

 

Paper Presentations - Final Assignment and Dates

1. Block 1: Explicit Modeling of Time 

   • Session 1 [14/05/2025]
     • Modeling Interleaved Hidden Processes  [S​​​​​​​​​tudent:​​ ​​7062722]
     • FlowHMM: Flow-based continuous hidden Markov Models [Student: 7074398]
   • ​​​​​​​Session 2 [21/05/2025]
     • ​​​​​​​WaveNet: A Generative Model for Raw Audio [Student: 7056420]
     • DeepAR: Probabilistic forecasting with autoregressive recurrent networks [Student: 7069106]

2. Block 2: Implicit Modeling of Time

   • ​​​​​​​Session 1 [04/06/2025]
     • ​​​​​​​BRITS: Bidirectional Recurrent Imputation for Time Series [Student: 7071404]
     • Attention Is All You Need [Student: 7047803]
   • ​​​​​​​Session 2 [11/06/2025]
     • ​​​​​​​A Time Series is Worth 64 Words: Long-term Forecasting with Transformers [Student: 7022859]
     • iTransformer: Inverted Transformers Are Effective for Time Series Forecasting [Student: 7076474]

3. ​​​​​​​​​​​​​​Block 3: Foundation Models and Task-Agnostic Approaches

   • ​​​​​​​Session 1 [25/06/2025]
     • BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding [Student: 7075927]
     • Unified Training of Universal Time Series Forecasting Transformers (MOIRAI) [Student: 7025878]
   • ​​​​​​​Session 2 [02/07/2025]
     • Visual Autoregressive Modeling: Scalable Image Generation via Next-Scale Prediction [Student: 7071951]
     • Align Your Latents: High-Resolution Video Synthesis with Latent Diffusion Models  [Student: 7072812]

 

Deliverables and Grading Scheme 

• Paper Presentation (15-20 minutes) (40%)

  • Submission (requirement): Students should submit a pdf file with the slides the day they are presenting.
  • Context 
    • Positioning of the paper within the state of the art and identification of gaps the paper addresses. 
    • Clear articulation of What/Why/How. 
    • Connection to the seminar blocks, e.g. Explicit, Implicit, Task Agnostic modeling of time. 
  • Content 
    • Clear explanation of the paper's core intuition and methodology. 
    • Rationale behind experiments and significance of results. 
    • Advantages and disadvantages of the approach. 
    • Final slide/section presenting the take-home messages. 
  • Q&A 
    • Responding questions from TAs and audience.

• Discussion Session (20%)

  • Pre-Submission Requirements
    • Each participant must submit one discussion question per block.
    • Submission deadline: 2 days before the session (Monday) via CMS.
  • Participation Expectations
    • For Presenters:
      • Quality of pre-submitted questions.
      • Active engagement with questions. 
      • Facilitating broader discussion. 
    • For Listeners:
      • Quality of pre-submitted questions. 
      • Active participation in discussions. 

• Final Report (6-8 pages, excluding references) (40%)

  • Template: https://www.overleaf.com/read/jmrjkxzrxpjn#2eff89
  • Critical Analysis
    • Comprehensive summary of the seminar from the perspective of time series modeling approaches (implicit, explicit, and foundation models), including the key points from the roundtable sessions. 
    • Comprehensive analysis regarding limitations, advantages, and open challenges in the field. 
  • Evaluation Criteria
    • Content
      • Depth of the analysis.
      • Demonstration of understanding across all three seminar blocks.
      • Synthesis of seminar content with broader research context.
    • Delivery 
      • Quality of writing and argumentation.