Skip to content

Time Series Analysis

Time series analysis is not our focus here. However, it is beneficial to grasp some basic ideas of time series.

Stationarity

Time series data is stationary if the distribution of the observables do not change126.

A strict stationary series guarantees the same distribution for a segment {xi+1,,xx+k} and a time-shifted segment xi+1+Δ,,xx+k+Δ} for integer Δ1.

A less strict form (WSS) concerns only the mean and autocorrelation13, i.e.,

(1)E[xi+1]=E[xi+Δ](2)Cov[xi+1,xi+k]=Cov[xi+1+Δ,xx+k+Δ]

In deep learning, a lot of models require the training data to be I.I.D.47. The I.I.D. requirement in time series is stationarity.

A stationary time series is clean and pure. However, real-world data is not necessarily stationary, e.g., macroeconomic series data are non-stationary6.

Serial Dependence

Autocorrelation measures the serial dependency of a time series5. By definition, the autocorrelation is the autocovariance normalized by the variance,

ρ=Cov[xt,xt+δ]Var[xt].

One naive expectation is that the autocorrelation diminishes if δ3.

Terminology

Terminologies for time series data may be different in different fields8. For example, we may encounter the term "panel data" in econometrics, which is the same as "multivariate time series" in "data science".

Panel Data

Panel data is multivariate time series data,

ytyit.
time variable y1 variable y2 variable y3
t1 y11 y21 y31
t2 y12 y22 y32
t3 y13 y23 y33
t4 y14 y24 y34
t5 y15 y25 y35
  1. doi:10.1162/99608f92.dec7d780

  1. Contributors to Wikimedia projects. Stationary process. In: Wikipedia [Internet]. 18 Sep 2022 [cited 13 Nov 2022]. Available: https://en.wikipedia.org/wiki/Stationary_process 

  2. 6.4.4.2. Stationarity. In: Engineering Statistics Handbook [Internet]. NIST; [cited 13 Nov 2022]. Available: https://www.itl.nist.gov/div898/handbook/pmc/section4/pmc442.htm#:~:text=Stationarity%20can%20be%20defined%20in,no%20periodic%20fluctuations%20(seasonality). 

  3. Shalizi C. 36-402, Undergraduate Advanced Data Analysis (2012). In: Undergraduate Advanced Data Analysis [Internet]. 2012 [cited 13 Nov 2022]. Available: https://www.stat.cmu.edu/~cshalizi/uADA/12/ 

  4. Schölkopf B, Locatello F, Bauer S, Ke NR, Kalchbrenner N, Goyal A, et al. Toward Causal Representation Learning. Proc IEEE. 2021;109: 612–634. doi:10.1109/JPROC.2021.3058954 

  5. Contributors to Wikimedia projects. Autocorrelation. In: Wikipedia [Internet]. 10 Nov 2022 [cited 13 Nov 2022]. Available: https://en.wikipedia.org/wiki/Autocorrelation 

  6. Das P. Econometrics in Theory and Practice. Springer Nature Singapore; doi:10.1007/978-981-32-9019-8 

  7. Dawid P, Tewari A. On learnability under general stochastic processes. Harvard Data Science Review. 2022; 

  8. Hyndman R. Rob J Hyndman - Terminology matters. In: Rob J Hyndman [Internet]. 26 Jun 2020 [cited 9 Nov 2023]. Available: https://robjhyndman.com/hyndsight/terminology-matters/#same-concept-different-terminology 


Contributors: LM