# Time Series Forecasting with Neural ODE

> [!NOTE]
>
> **In one minute**
>
> A neural ODE, built with torchdyn and with a small neural network as its field, forecasts the pendulum series from 100 past steps. It suits this dataset because the pendulum is itself generated by a differential equation; the page reports one-step and three-step results against the naive forecast.
>
> **Read first:** [Neural ODE](<https://dl.leima.is/dynamical-systems/neural-ode/index.md>) · [Pendulum Dataset](<https://dl.leima.is/time-series-deep-learning/timeseries.dataset.pendulum/index.md>)

> [!NOTE]
>
> **Jupyter Notebook Available**
>
> We have a [📓 notebook](<https://dl.leima.is/notebooks/neuralode_timeseries/index.md>) for this section which includes all the code used in this section.

> [!NOTE]
>
> **Introduction to Neural ODE**
>
> We explain the theories of neuralode in [this section](<https://dl.leima.is/dynamical-systems/neural-ode/index.md>). Please read it first if you are not familiar with neural ode.

In the section [Neural ODE](<https://dl.leima.is/dynamical-systems/neural-ode/index.md>), we have introduced the concept of neural ODE. In this section, we will show how to use neural ODE to do time series forecasting.

## A Neural ODE Model

We built a single hidden layer neural network as the field,

```
graph TD
    input["Input (100)"]
    input_layer["Hidden Layer (100)"]
    output_layer["Output Layer (100)"]
    hidden_layer["Hidden Layer (256)"]
    output["Output (1)"]

    input --> input_layer
    input_layer --> hidden_layer
    hidden_layer --> output_layer
    output_layer --> output
```

The model is built using the package called [torchdyn](<https://github.com/DiffEqML/torchdyn>) [^/time-series-deep-learning/timeseries.neural-ode/#bib-Poli_TorchDyn_Implicit_Models].

> [!NOTE]
>
> **Packages**
>
> Apart from the torchdyn package we used here, there is another package called [torchdiffeq](<https://github.com/rtqichen/torchdiffeq>) [^/time-series-deep-learning/timeseries.neural-ode/#bib-Chen_torchdiffeq_2021] which is developed by the authors of neural ode.

## Single Step Forecasts

We trained the model using a history length of 100 and only forecast one step (with a gap of 10 between the input and target). The result is shown below.

![Results](<https://dl.leima.is/time-series-deep-learning/assets/timeseries.neural-ode/neuralode_univariate_results.png>)

Neural ODE is a good forecaster for our pendulum dataset since the pendulum is simply generated by a differential equation. The metrics are also computed and listed below.

| Metric | Neural ODE | Naive |
| --- | --- | --- |
| Mean Absolute Error | **0.003052** | 0.092666 |
| Mean Squared Error | **0.000009** | 0.010553 |
| Symmetric Mean Absolute Percentage Error | **0.021231** | 0.376550 |

> [!NOTE]
>
> **Training**
>
> The training loss is shown below.
>
> ![Training](<https://dl.leima.is/time-series-deep-learning/assets/timeseries.neural-ode/neuralode_univariate_single_step_training.png>)

## Multi-Step Forecasts

We perform a similar experiment but forecast 3 steps.

![NODE results](<https://dl.leima.is/time-series-deep-learning/assets/timeseries.neural-ode/neuralode_univariate_m_step_results.png>)

We plot out some samples and shade the predictions using orange color. The plot below shows that the forecasts are mostly on the right trend.

![NODE samples](<https://dl.leima.is/time-series-deep-learning/assets/timeseries.neural-ode/neuralode_univariate_m_step_samples.png>)

| Metric | Neural ODE | Naive |
| --- | --- | --- |
| Mean Absolute Error | **0.038421** | 0.109485 |
| Mean Squared Error | **0.001478** | 0.014723 |
| Symmetric Mean Absolute Percentage Error | **0.153392** | 0.423563 |

---

**Related:** [Dynamical Systems](<https://dl.leima.is/dynamical-systems/index.md>) · [Forecasting with RNN](<https://dl.leima.is/time-series-deep-learning/timeseries.rnn/index.md>)

[^/time-series-deep-learning/timeseries.neural-ode/#bib-Poli_TorchDyn_Implicit_Models]: Poli M., Massaroli S., Yamashita A. et al.. TorchDyn: Implicit Models and Neural Numerical Methods in PyTorch. [https://github.com/DiffEqML/torchdyn](<https://github.com/DiffEqML/torchdyn>)

[^/time-series-deep-learning/timeseries.neural-ode/#bib-Chen_torchdiffeq_2021]: Chen R.T.Q.. torchdiffeq. 2021. [https://github.com/rtqichen/torchdiffeq](<https://github.com/rtqichen/torchdiffeq>)
