Autoregressive Model¶
In one minute
An autoregressive model describes each variable conditioned on the ones before it. The page also sorts out the different notations the literature uses for those preceding variables.

An autoregressive (AR) model is autoregressive,
Notations and Conventions
In AR models, we have to mention the preceding nodes (\(\{x_{<t}\}\)) of a specific node (\(x_{t}\)). For \(t=5\), the relations between \(\{x_{<5}\}\) and \(x_5\) are shown in the following illustration.

There are different notations for such relations.
- In Uria et al., the authors use \(p(x_{o_d}\mid \mathbf x_{o_{<d}})\) 1.
- In Liu et al. and Papamakarios et al., the authors use \(p(x_{t}\mid \mathbf x_{1:t-1})\) 64.
- In Germain et al., the authors use \(p(x_t\mid \mathbf x_{<t})\) 5.
In the current review, we expanded the vector notation \(\mathbf x_{<t}\) into a set notation as it is not necessarily a vector.
-
Uria B, Côté M-A, Gregor K, Murray I, Larochelle H. Neural Autoregressive Distribution Estimation. arXiv [cs.LG]. 2016. Available: http://arxiv.org/abs/1605.02226 ↩
-
Triebe O, Laptev N, Rajagopal R. AR-Net: A simple Auto-Regressive Neural Network for time-series. arXiv [cs.LG]. 2019. Available: http://arxiv.org/abs/1911.12436 ↩
-
Ho G. George Ho. In: Eigenfoo [Internet]. 9 Mar 2019 [cited 19 Sep 2021]. Available: https://www.eigenfoo.xyz/deep-autoregressive-models/ ↩
-
Papamakarios G, Pavlakou T, Murray I. Masked Autoregressive Flow for Density Estimation. arXiv [stat.ML]. 2017. Available: http://arxiv.org/abs/1705.07057 ↩
-
Germain M, Gregor K, Murray I, Larochelle H. MADE: Masked autoencoder for distribution estimation. 32nd International Conference on Machine Learning, ICML 2015. 2015;2: 881–889. Available: http://arxiv.org/abs/1502.03509 ↩
-
Liu X, Zhang F, Hou Z, Wang Z, Mian L, Zhang J, et al. Self-supervised Learning: Generative or Contrastive. arXiv [cs.LG]. 2020. Available: http://arxiv.org/abs/2006.08218 ↩
-
Lippe P. Tutorial 12: Autoregressive Image Modeling — UvA DL Notebooks v1.1 documentation. In: UvA Deep Learning Tutorials [Internet]. [cited 20 Sep 2021]. Available: https://uvadlc-notebooks.readthedocs.io/en/latest/tutorial_notebooks/tutorial12/Autoregressive_Image_Modeling.html ↩
-
rogen-george. rogen-george/Deep-Autoregressive-Model. In: GitHub [Internet]. [cited 20 Sep 2021]. Available: https://github.com/rogen-george/Deep-Autoregressive-Model ↩