Time Series Forecasting using Deep Learning¶
Forecasting the future is an extremely valuable superpower. The forecasting game has been dominated by statisticians who are real experts in time series problems. As the amount of data increases, many of the statistical methods are not squeezing enough out of the massive datasets. Consequently, time series forecasting using deep learning emerges and became a fastgrowing field. It is trendy, not only in LinkedIn debates but also in academic papers. We plotted the number of related publications per year using the keyword "deep learning forecasting" on dimensions.ai) ^{2}.
On the other hand, deep learning methods are not yet winning all the games of forecasting. Time series forecasting is a complicated problem with a great variety of data generating processes (DGP). Some argue that we don't need deep learning to forecast since well tuned statistical models and trees are already performing well and are faster and more interpretable than deep neural networks^{3}^{4}. Ensembles of statistical models performing great, even outperforming many deep learning models on the M3 data^{1}.
However, deep learning models are picking up speed. In the M5 competition, deep learning "have shown forecasting potential, motivating further research in this direction"^{5}. As the complexity and size of time series data are growing and more and more deep learning forecasting models are being developed, forecasting with deep learning is on the path to be an important alternative to statistical forecasting methods.
In Coding Tips, we provide coding tips to help some readers set up the development environment. In Deep Learning Fundamentals, we introduce the fundamentals of deep neural networks and their practices. For completeness, we also provide code and derivations for the models. With these two parts, we introduce time series data and statistical forecasting models in Time Series Forecasting Fundamentals, where we discuss methods to analyze time series data, several universal data generating processes of time series data, and some statistical forecasting methods. Finally, we fulfill our promise in the title in Time Series Forecasting with Deep Learning.
Blueprint¶
The following is my first version of the blueprint.
 Coding Tips
 Environment, VSCode, Git, ...
 Python Project Tips
 Time Series Fundamentals
 Data
 Time Series Analysis
 Data Augmentation
 DTW and DBA
 Time Delayed Embedding for Deep Learning
 Some Statistical Models
 AR and VAR
 State Space Models
 Naive Forecasters
 Data Generating Process (DGP)
 General discussions
 Dynamical systems, e.g., the diffusion process
 Creating synthetic data (GluonTS, eerily)
 Metrics
 List of forecasting metrics, their properties and demos.
 Hierarchical Forecasting
 Hierarchical time series data
 Reconciliation
 End2End hierarchical forecasting methods
 Useful Datasets
 Data
 Deep Learning Fundamentals
 Energybased Models
 Diffusion models
 Neural ODE
 Neural ODE Basics
 Generative
 AR
 AE
 VAE
 Flow
 MADE
 MAF
 Constrastive
 Deep Infomax
 Contrastive Predictive Coding
 Adversarial
 GAN
 fGAN
 InfoGAN
 Transformers
 Vanilla transformers
 Energybased Models
 Deep Learning Models for Forecasting
 RNN, e.g., LSTM
 DeepAR, DeepVAR
 Diffusion Models, e.g., TimeGrad
 Transformer
 TFT
 DLinear
 NLinear
 Spatialtemporal Models, e.g., GNN
 Conformal Prediction
 Graph Neural Networks
 Spiking Neural Networks

Nixtla. statsforecast/experiments/m3 at main · Nixtla/statsforecast. In: GitHub [Internet]. [cited 12 Dec 2022]. Available: https://github.com/Nixtla/statsforecast/tree/main/experiments/m3 ↩

Daniel W. Hook, Simon J. Porter, and Christian Herzog. Dimensions: building context for search and evaluation. Frontiers in Research Metrics and Analytics, 3():23, 2018. https://www.frontiersin.org/articles/10.3389/frma.2018.00023/pdf. URL: https://app.dimensions.ai/details/publication/pub.1106289502, doi:10.3389/frma.2018.00023. ↩

Shereen Elsayed, Daniela Thyssens, Ahmed Rashed, Hadi Samer Jomaa, and Lars SchmidtThieme. Do we really need deep learning models for time series forecasting? 6 January 2021. URL: http://arxiv.org/abs/2101.02118, arXiv:2101.02118, doi:10.48550/ARXIV.2101.02118. ↩

Léo Grinsztajn, Edouard Oyallon, and Gaël Varoquaux. Why do treebased models still outperform deep learning on tabular data? 18 July 2022. URL: http://arxiv.org/abs/2207.08815, arXiv:2207.08815, doi:10.48550/ARXIV.2207.08815. ↩

Spyros Makridakis, Evangelos Spiliotis, and Vassilios Assimakopoulos. M5 accuracy competition: results, findings, and conclusions. International journal of forecasting, 38(4):1346–1364, 1 October 2022. URL: https://www.sciencedirect.com/science/article/pii/S0169207021001874, doi:10.1016/j.ijforecast.2021.11.013. ↩