Time Series Forecasting using Deep Learning¶
Forecasting the future is an extremely valuable superpower. The forecasting game has been dominated by statisticians who are real experts in time series problems. As the amount of data increases, many of the statistical methods are not squeezing enough out of the massive datasets. Consequently, time series forecasting using deep learning emerges and became a fastgrowing field. It is trendy, not only in LinkedIn debates but also in academic papers. We plotted the number of related publications per year using the keyword "deep learning forecasting" on dimensions.ai) ^{2}.
On the other hand, deep learning methods are not yet winning all the games of forecasting. Time series forecasting is a complicated problem with a great variety of data generating processes (DGP). Some argue that we don't need deep learning to forecast since well tuned statistical models and trees are already performing well and are faster and more interpretable than deep neural networks^{3}^{4}. Ensembles of statistical models performing great, even outperforming many deep learning models on the M3 data^{1}.
However, deep learning models are picking up speed. In the M5 competition, deep learning "have shown forecasting potential, motivating further research in this direction"^{5}. As the complexity and size of time series data are growing and more and more deep learning forecasting models are being developed, forecasting with deep learning is on the path to be an important alternative to statistical forecasting methods.
In Coding Tips, we provide coding tips to help some readers set up the development environment. In Deep Learning Fundamentals, we introduce the fundamentals of deep neural networks and their practices. For completeness, we also provide code and derivations for the models. With these two parts, we introduce time series data and statistical forecasting models in Time Series Forecasting Fundamentals, where we discuss methods to analyze time series data, several universal data generating processes of time series data, and some statistical forecasting methods. Finally, we fulfill our promise in the title in Time Series Forecasting with Deep Learning.
Blueprint¶
The following is my first version of the blueprint.
 Engineering Tips
 Environment, VSCode, Git, ...
 Python Project Tips
 Fundamentals of Time Series Forecasting
 Time Series Data and Terminologies
 Transformation of Time Series
 Twoway Fixed Effects
 Time Delayed Embedding
 Data Generating Process (DGP)
 DGP: Langevin Equation
 Kindergarten Models for Time Series Forecasting
 Statistical Models
 Statistical Model: AR
 Statistical Model: VAR
 Synthetic Datasets
 Synthetic Time Series
 Creating Synthetic Dataset
 Data Augmentation
 Forecasting
 Time Series Forecasting Tasks
 Naive Forecasts
 Evaluation and Metrics
 Time Series Forecasting Evaluation
 Time Series Forecasting Metrics
 CRPS
 Hierarchical Time Series
 Hierarchical Time Series Data
 Hierarchical Time Series Reconciliation
 Some Useful Datasets
 Trees
 Treebased Models
 Random Forest
 Gradient Boosted Trees
 Forecasting with Trees
 Fundamentals of Deep Learning
 Deep Learning Introduction
 Learning from Data
 Neural Networks
 Recurrent Neural Networks
 Convolutional Neural Networks
 Transformers
 Dynamical Systems
 Why Dynamical Systems
 Neural ODE
 Energybased Models
 Diffusion Models
 Generative Models
 Autoregressive Model
 AutoEncoder
 Variational AutoEncoder
 Flow
 Generative Adversarial Network (GAN)
 Time Series Forecasting with Deep Learning
 A Few Datasets
 Forecasting with MLP
 Forecasting with RNN
 Forecasting with Transformers
 TFT
 DLinear
 NLinear
 Forecasting with CNN
 Forecasting with VAE
 Forecasting with Flow
 Forecasting with GAN
 Forecasting with Neural ODE
 Forecasting with Diffusion Models
 Extras Topics, Supplementary Concepts, and Code
 DTW and DBA
 fGAN
 InfoGAN
 Spatialtemporal Models, e.g., GNN
 Conformal Prediction
 Graph Neural Networks
 Spiking Neural Networks
 Deep Infomax
 Contrastive Predictive Coding
 MADE
 MAF
 ...

Nixtla. statsforecast/experiments/m3 at main · Nixtla/statsforecast. In: GitHub [Internet]. [cited 12 Dec 2022]. Available: https://github.com/Nixtla/statsforecast/tree/main/experiments/m3 ↩

Hook DW, Porter SJ, Herzog C. Dimensions: Building context for search and evaluation. Frontiers in Research Metrics and Analytics 2018; 3: 23. ↩

Elsayed S, Thyssens D, Rashed A, Jomaa HS, SchmidtThieme L. Do we really need deep learning models for time series forecasting? 2021. doi:10.48550/ARXIV.2101.02118. ↩

Grinsztajn L, Oyallon E, Varoquaux G. Why do treebased models still outperform deep learning on tabular data? 2022. doi:10.48550/ARXIV.2207.08815. ↩

Makridakis S, Spiliotis E, Assimakopoulos V. M5 accuracy competition: Results, findings, and conclusions. International journal of forecasting 2022; 38: 1346–1364. ↩