Skip to content

Time Series Forecasting using Deep Learning

Forecasting the future is an extremely valuable superpower. The forecasting game has been dominated by statisticians who are real experts in time series problems. As the amount of data increases, many of the statistical methods are not squeezing enough out of the massive datasets. Consequently, time series forecasting using deep learning emerges and became a fast-growing field. It is trendy, not only in LinkedIn debates but also in academic papers. We plotted the number of related publications per year using the keyword "deep learning forecasting" on dimensions.ai) 2.

deep-learning-forecasting-dimension-ai

This chart is obtained on 2022-08-06, from Digital Science’s Dimensions platform, available at https://app.dimensions.ai

On the other hand, deep learning methods are not yet winning all the games of forecasting. Time series forecasting is a complicated problem with a great variety of data generating processes (DGP). Some argue that we don't need deep learning to forecast since well tuned statistical models and trees are already performing well and are faster and more interpretable than deep neural networks34. Ensembles of statistical models performing great, even outperforming many deep learning models on the M3 data1.

However, deep learning models are picking up speed. In the M5 competition, deep learning "have shown forecasting potential, motivating further research in this direction"5. As the complexity and size of time series data are growing and more and more deep learning forecasting models are being developed, forecasting with deep learning is on the path to be an important alternative to statistical forecasting methods.

In Coding Tips, we provide coding tips to help some readers set up the development environment. In Deep Learning Fundamentals, we introduce the fundamentals of deep neural networks and their practices. For completeness, we also provide code and derivations for the models. With these two parts, we introduce time series data and statistical forecasting models in Time Series Forecasting Fundamentals, where we discuss methods to analyze time series data, several universal data generating processes of time series data, and some statistical forecasting methods. Finally, we fulfill our promise in the title in Time Series Forecasting with Deep Learning.

Blueprint

The following is my first version of the blueprint.

  • Coding Tips
    • Environment, VSCode, Git, ...
    • Python Project Tips
  • Time Series Fundamentals
    • Data
      • Time Series Analysis
      • Data Augmentation
      • DTW and DBA
      • Time Delayed Embedding for Deep Learning
    • Some Statistical Models
      • AR and VAR
      • State Space Models
      • Naive Forecasters
    • Data Generating Process (DGP)
      • General discussions
      • Dynamical systems, e.g., the diffusion process
      • Creating synthetic data (GluonTS, eerily)
    • Metrics
      • List of forecasting metrics, their properties and demos.
    • Hierarchical Forecasting
      • Hierarchical time series data
      • Reconciliation
      • End2End hierarchical forecasting methods
    • Useful Datasets
  • Deep Learning Fundamentals
    • Energy-based Models
      • Diffusion models
    • Neural ODE
      • Neural ODE Basics
    • Generative
      • AR
      • AE
      • VAE
      • Flow
      • MADE
      • MAF
    • Constrastive
      • Deep Infomax
      • Contrastive Predictive Coding
    • Adversarial
      • GAN
      • f-GAN
      • Info-GAN
    • Transformers
      • Vanilla transformers
  • Deep Learning Models for Forecasting
    • RNN, e.g., LSTM
    • DeepAR, DeepVAR
    • Diffusion Models, e.g., TimeGrad
    • Transformer
      • TFT
      • DLinear
      • NLinear
    • Spatial-temporal Models, e.g., GNN
    • Conformal Prediction
    • Graph Neural Networks
    • Spiking Neural Networks

  1. Nixtla. statsforecast/experiments/m3 at main · Nixtla/statsforecast. In: GitHub [Internet]. [cited 12 Dec 2022]. Available: https://github.com/Nixtla/statsforecast/tree/main/experiments/m3 ↩

  2. Daniel W. Hook, Simon J. Porter, and Christian Herzog. Dimensions: building context for search and evaluation. Frontiers in Research Metrics and Analytics, 3():23, 2018. https://www.frontiersin.org/articles/10.3389/frma.2018.00023/pdf. URL: https://app.dimensions.ai/details/publication/pub.1106289502, doi:10.3389/frma.2018.00023. ↩

  3. Shereen Elsayed, Daniela Thyssens, Ahmed Rashed, Hadi Samer Jomaa, and Lars Schmidt-Thieme. Do we really need deep learning models for time series forecasting? 6 January 2021. URL: http://arxiv.org/abs/2101.02118, arXiv:2101.02118, doi:10.48550/ARXIV.2101.02118. ↩

  4. Léo Grinsztajn, Edouard Oyallon, and Gaël Varoquaux. Why do tree-based models still outperform deep learning on tabular data? 18 July 2022. URL: http://arxiv.org/abs/2207.08815, arXiv:2207.08815, doi:10.48550/ARXIV.2207.08815. ↩

  5. Spyros Makridakis, Evangelos Spiliotis, and Vassilios Assimakopoulos. M5 accuracy competition: results, findings, and conclusions. International journal of forecasting, 38(4):1346–1364, 1 October 2022. URL: https://www.sciencedirect.com/science/article/pii/S0169207021001874, doi:10.1016/j.ijforecast.2021.11.013. ↩

Contributors: LM