Statistical Models of Time Series¶
Though statistical models are not our focus, it is always beneficial to understand how those famous statistical models work. To best understand how the models work, we will build some data generating process using these models and explore their behavior.
In the following paragraphs, we list some of the most applied statistical models. For a comprehensive review of statistical models, please refer to Petropoulos et al., 2022 and Hyndman et al., 202134.
ARIMA is one of the most famous forecasting models1. We will not discuss the details of the model. However, for reference, we sketch the relations between the different components of the ARIMA model in the following chart.
flowchart TD AR --"interdependencies"--> VAR MA --"add autoregressive"--> ARMA AR --"add moving average"--> ARMA ARMA --"difference between values"--> ARIMA ARMA --"interdependencies"--> VARMA VAR --"moving average"--> VARMA ARIMA --"interdependencies"--> VARIMA VAR --"difference and moving average"--> VARIMA VARMA --"difference"--> VARIMA
A Naive Forecast
In time series forecasting, one of the naive forecasts we can use it the previous observation, i.e.,
where we use \(\hat s\) to denote the forecasts and \(s\) for the observations.
where \(\hat s\) is the forecast and \(s\) is the observation. Expanding this form, we observe the exponential decaying effect of history in the long past4.
State Space Models¶
State space models (SSM) are amazing models due to their simplicity. SSM applies Markov chains but is not limited to the Markovian assumptions5.
Bishop CM. Pattern Recognition and Machine Learning. Springer; 2006. Available: https://play.google.com/store/books/details?id=qWPwnQEACAAJ ↩