Skip to content

Time Series Forecasting with Neural ODE

Jupyter Notebook Available

We have a 📓 notebook for this section which includes all the code used in this section.

Introduction to Neural ODE

We explain the theories of neuralode in this section. Please read it first if you are not familiar with neural ode.

In the section Neural ODE, we have introduced the concept of neural ODE. In this section, we will show how to use neural ODE to do time series forecasting.

A Neural ODE Model

We built a single hidden layer neural network as the field,

graph TD
    input["Input (100)"]
    input_layer["Hidden Layer (100)"]
    output_layer["Output Layer (100)"]
    hidden_layer["Hidden Layer (256)"]
    output["Output (1)"]

    input --> input_layer
    input_layer --> hidden_layer
    hidden_layer --> output_layer
    output_layer --> output

The model is built using the package called torchdyn 1.


Apart from the torchdyn package we used here, there is another package called torchdiffeq 2 which is developed by the authors of neural ode.

Single Step Forecasts

We trained the model using a history length of 100 and only forecast one step (with a gap of 3 between the input and target). The result is shown below.


Neural ODE is a good forecaster for our pendulum dataset since the pendulum is simply generated by a differential equation. The metrics are also computed and listed below.

Metric Neural ODE Naive
Mean Absolute Error 0.003052 0.092666
Mean Squared Error 0.000009 0.010553
Symmetric Mean Absolute Percentage Error 0.021231 0.376550

The training loss is shown below.


Multi-Step Forecasts

We perform a similar experiment but forecast 3 steps.

NODE results

We plot out some samples and shade the predictions using orange color. The plot below shows that the forecasts are mostly on the right trend.

NODE samples

Metric Neural ODE Naive
Mean Absolute Error 0.038421 0.109485
Mean Squared Error 0.001478 0.014723
Symmetric Mean Absolute Percentage Error 0.153392 0.423563

  1. Poli M, Massaroli S, Yamashita A, Asama H, Park J, Ermon S. TorchDyn: Implicit Models and Neural Numerical Methods in PyTorch 

  2. Chen RTQ. torchdiffeq 

Contributors: LM