Time Series Analysis

name: stochasticmodeling
class: center,middle, .toc[[✧](../index.html)]
.title[Stochastic Modeling]

---
class: center, .toc[[✧](../index.html)]
#Ocean Turbulence

An idealized model of oceanic turbulence by Jeffrey Early.
Shading is speed with particle trajectories shown in color.

---
class: center, .toc[[✧](../index.html)]

#Modeling Trajectories

A simple stochastic model for trajectories in oceanic turbulence is the *Matérn process* or damped Fractional Brownian motion.

This three-parameter model lets us match (i) the kinetic energy, (ii) the dispersion, and (iii) the degree of small-scale roughness.

---
class: left, .toc[[✧](../index.html)]
#The Matérn Process

A relatively little-known random process called the Matérn process .cite[(Matérn, 1960)] has a spectrum given by
`\[S_{zz}(\omega) = \frac{A^2}{(\omega^2+\lambda^2)^\alpha}\]`
where `$A$` sets the energy level, `$\alpha$` sets the high-frequency slope, and `$\lambda$` determines a low-frequency transition to a constant spectral value.

--
Recently, we investigated this process in detail, and showed that it is an excellent model for trajectories in ocean turbulence.  We speculate it will prove suitable for many other processes as well.

We showed the Matérn process is a generalization of *fractional Brownian motion* .cite[(Mandelbrot and Van Ness, 1968)] to include a damping term, `$\lambda$`, that is the essential ingredient for modeling dispersion.

.cite[Lilly, Sykulski, Early, and Olhede (2017). Fractional Brownian motion, the Matérn process, and stochastic modeling of turbulent dispersion. <a href="./papers/lilly17-npg.pdf">[3.2 Mb pdf] </a>]
---
class: left, .toc[[✧](../index.html)]

## An Hierarchy of Stochastic Models

The following stochastic differential equations (SDEs) should more formally be written as the stochastic *integral* equations. Note that `$z(t)=u(t)+iv(t)$` represents a complex-valued velocity.

Brownian motion:

`\[\frac{dz}{d t}  = A\,\frac{d w}{d t} \quad\longrightarrow\quad S_{zz}(\omega) = \frac{A^2}{\omega^2}\]`

Fractional Brownian motion (fBm):

`\[\frac{d^\alpha z}{d t^\alpha}  = A\,\frac{d w}{d t} \quad\longrightarrow\quad S_{zz}(\omega) = \frac{A^2}{\omega^{2\alpha}}\]`

Damped Fractional Brownian motion a.k.a. the Matérn process:

`\[\left[\frac{d}{d t} + \lambda \right]^\alpha z = A\,\frac{d w}{d t} \quad\longrightarrow\quad S_{zz}(\omega) = \frac{A^2}{(\omega^2+\lambda^2)^\alpha}\]`

---
class: left, .toc[[✧](../index.html)]

## Another Hierarchy of Stochastic Models

Damped Brownian motion a.k.a. Ornstein-Uhlenbeck (OU) process:
`\[\frac{dz}{d t} + \lambda z = A\,\frac{d w}{d t} \label{maternsde}\quad\longrightarrow\quad S_{zz}(\omega) = \frac{A^2}{\omega^2+\lambda^2}\]`

Damped Brownian motion + spin a.k.a. complex OU (cOU) process:
`\[\frac{dz}{d t} - i\Omega z+ \lambda z = A\,\frac{d w}{d t} \quad\longrightarrow\quad S_{zz}(\omega) = \frac{A^2}{(\omega-\Omega)^2+\lambda^2}\]`

Damped fBm + spin a.k.a. Matérn + spin a.k.a. fractional cOU:
`\[\left[\frac{d}{d t} - i\Omega+ \lambda \right]^\alpha z = A\,\frac{d w}{d t} \quad\longrightarrow\quad S_{zz}(\omega) = \frac{A^2}{[(\omega-\Omega)^2+\lambda^2]^\alpha}\]`

Matérn + spin generalizes cOU to fractional orders, thus unifying OU+cOU+fBm into a single larger family.
---
class: center, .toc[[✧](../index.html)]
#Slope and Roughness

The `$\alpha$` parameter sets the slope in fractional Brownian motion, or at high frequencies in the Matérn process, as `$S(\omega)\sim \omega^{-2\alpha}$`.

It also sets the degree of small-scale roughness in the trajectories.
In the above, we go from slopes of `$\omega^{-3}$` at left to `$\omega^{-1}$` at right.

Steeper spectral slope = more smooth.

Spectral slope can be formally linked to a mathematical measure of roughness called the *fractal dimension*, though in my opinion, fractal dimension seems to have little practical value.
---
class: center, .toc[[✧](../index.html)]
#Slope and Self-Similarity

The `$\alpha$` parameter also sets *aspect ratio* of statistical self-similarity.

---
class: left, .toc[[✧](../index.html)]
##Diffusivity is the Spectrum's Value at Zero

The velocity autocovariance and spectrum are defined as

`\[R_{zz} (\tau)\equiv\left\langle z(t+\tau)\,z^*(t)\right\rangle = \frac{1}{2\pi}\int_{-\infty}^\infty e^{i \omega \tau} S_{zz}(\omega) \, d\omega\]`

The corresponding diffusivity is given by
`\[\kappa \equiv \lim_{t\longrightarrow\infty}
 \frac{1}{4} \frac{d}{d t} \left\langle x^2(t) + y^2(t)\right\rangle
 =\frac{1}{4} \int_{-\infty}^\infty R_{zz}(\tau)\, d \tau =\frac{1}{4}S_{zz}(0)\]`

with the last equality following from `$\int_{-\infty}^\infty e^{-i \omega \tau} R_{zz}(\tau) \, d\tau=  S_{zz}(\omega)$`.

Matérn:  `$ \,\,\,\,S_{zz}(\omega) = \frac{A^2}{(\omega^2+\lambda^2)^\alpha} \quad\longrightarrow\quad \kappa =  \frac{1}{4}\frac{A^2}{\lambda^{2\alpha}}$`

fBm: `$ \quad\quad S_{zz}(\omega)= \frac{A^2}{\omega^{2\alpha}} \quad\quad\,\,\longrightarrow\quad \kappa =  \infty$`

Infinity means the diffusivity increases without bound.  Thus the Matérn, unlike fBm, can model diffusive processes.

---
class: left
##Another Take on Diffusivity

Most simply, the absolute diffusivity can be understood as a fundamental second-order statistical property of the Lagrangian velocity.

The diffusivity is the twin of the variance.

`\[\sigma^2  =\frac{1}{2\pi} \int_{-\infty}^\infty S_{zz}(\tau)\, d \tau =R_{zz}(0)\]`
`\[\kappa  =\frac{1}{4} \int_{-\infty}^\infty R_{zz}(\tau)\, d \tau =\frac{1}{4}S_{zz}(0)\]`

Thus diffusivity is a basic second-order property of any time series.

The zero frequency value of the spectrum of a process describes how the temporal integral of the process spreads out over time.

You don't have to think that diffusion is a good physical model in order to usefully quantify a process using `$\kappa$` as one of the parameters.

---
class: left
##The Damping Time Scale

The `$\lambda$` parameter sets the frequency, or timescale `$2\pi/\lambda$`, at which the Matérn process transitions from behaving like fractional Brownian motion to behaving like white noise:

`\[S_{zz}(\omega) = \frac{A^2}{(\omega^2+\lambda^2)^\alpha}.\]`

`\[S_{zz}(\omega) \approx \frac{A^2}{\omega^{2\alpha}},\quad\quad\quad \omega\gg \lambda\]`
`\[S_{zz}(\omega) \approx \frac{A^2}{\lambda^{2\alpha}},\quad\quad\quad \omega\ll \lambda\]`

If you zoom in to the velocity process, you see fBm.
If you zoom out, you see white noise (and thus finite diffusivity).

Note that `$2\pi/\lambda$` is not the same as the Lagrangian integral timescale.

---
class: left, .toc[[✧](../index.html)]
#Summary of Parameters

Returning the Matérn SDE and spectrum

`\[\left[\frac{d}{d t} + \lambda \right]^\alpha z = A\,\frac{d w}{d t} \quad\longrightarrow\quad S_{zz}(\omega) = \frac{A^2}{(\omega^2+\lambda^2)^\alpha}\]`

we find the following interpretation of parameters:

1.  `$A$` sets the process amplitude.
2.  `$\alpha$`, the order of the SDE, sets the spectral slope, fractal dimension, and self-similarity aspect ratio at small scales.
3.  `$\lambda$`, which appears as a damping in the SDE, sets the low-frequency transition to a white spectrum, and also controls the diffusivity together with `$A$`.

Keep in mind that the SDE is written only for notational convenience; formally, one should write the corresponding stochastic integral equation, as discussed in Lilly et al. (2017).

---
class: center, .toc[[✧](../index.html)]
#Comparison of Models

Top left: numerical simulations.  Top right: Matérn process.
Bottom left: white noise.  Bottom right: red noise (Brownian).

---
class: left, .toc[[✧](../index.html)]
#Argument for Generality

We conjecture that the Matérn may be of general usefulness:

“Systems are often characterized by a pressure to grow—represented by a forcing—together with some drag or resistance on that growth, represented by a damping. After a sufficiently long time, the forcing and the damping equilibrate and one reaches a bounded state. This leads to the speculation that many time series that are well described as fBm over relatively short timescales may be better matched by the Matérn process over longer timescales. More generally, the Matérn process adds a third parameter (damping) to the two parameters (amplitude together with spectral slope or the Hurst parameter) of fBm, thus permitting a wider range of spectral forms to be accommodated. It is therefore reasonable to think that the Matérn process could be of broad interest in many areas in which fBm has already proven itself useful.”

.cite[Lilly, Sykulski, Early, and Olhede (2017)]

---
class: center, .toc[[✧](../index.html)]
## The Global Surface Drifter Dataset

Now we will apply the Matérn process to the global drifter dataset.

---
class: center, .toc[[✧](../index.html)]
## Spectra of Drifter Velocities

We see a low-frequency maximium and tidal maxima—all vertical lines—and a maximum along a curving line, the inertial frequency.

---
class: center, .toc[[✧](../index.html)]
##A Conceptual Model for the Spectrum

\begin{equation}
S(\omega) = \overset{\mathrm{background}}{\overbrace{\frac{A_b^2}{\left[\omega^2+\lambda_b^2\right]^\alpha}}}
+\overset{\mathrm{inertial\,\, oscillations}}{\overbrace{\frac{A_o^2}{\left(\omega-f_o\right)^2+\lambda_o^2}}}+
\overset{\mathrm{semidiurnal\,\,tide}}{\overbrace{\frac{A_s^2}{\left(\omega-f_s\right)^2+\lambda_s^2}}}
\end{equation}

This is just several versions of the Matérn spectrum added together.

---
class: left, .toc[[✧](../index.html)]

## Details of the Stochastic Modeling

We have developed a method for stochastic modeling of trajectories in ocean turbulence, and inferring parameters from large datasets.

This is an example of *parametric spectral analysis*:  rather than trying to estimate the spectrum directly, one begins with a *conceptual model*, controlled by some parameters, and then finds the choice of parameters which provide the best match to the data.

The full method consists of several ingredients:

*  Create an appropriate stochastic model for particle trajectories.
.cite[Sykulski, Olhede, Lilly, and Danioux (2015)]
* Employ the  Matérn  process as a building block.  .cite[Lilly, Sykulski, Early, and Olhede (2017)]
* Parameter estimation is best done in the frequency domain.
.cite[Whittle (1953)]
* Parameter estimation must be adjusted to handle *complex-valued* time series. .cite[Sykulski, Olhede, Lilly, and Early (2017)]
* Parameter estimation must be corrected for subtle bias due to small sample size effects.  .cite[Sykulski, Olhede, Guillaumin, Lilly, and Early (2019)]

---
class: center, .toc[[✧](../index.html)]
### Spectra in Latitude Bands
<img style="width:75%;margin-top:-0.7em;margin-bottom:-0.8em" src="../../figures/drifterspectralinestransparency.png">

Averages in `$\pm 5 ^\circ $` bins.  Frequency is nondimensionalized as `$\omega/f_o$`.
Inertial peak broadens equatorward.

---
class: center, .toc[[✧](../index.html)]
### Comparison with Spectral Fit

In general, spectral model compares very well with estimated spectra.  Inertial and semidiurnal peaks are well captured.

---
class: center, .toc[[✧](../index.html)]
### Fit to Background vs. Cyclonic Side

“Background” fit to anticyclonic (inertial side), matches well the spectrum on the cyclonic (anti-inertial) side, apart from tides.

---
class: center, .toc[[✧](../index.html)]
##Application to the Global Drifter Dataset

The nondimensionalized damping parameter from a parametric fit to the Matérn, only on the anti-inertial side.

Shorter ‘memory’ in more energetic regions.
We are not aware of any theory for this.
---
class: center, .toc[[✧](../index.html)]
##Application to the Global Drifter Dataset

The first global map of the slope parameter `$\alpha$`, with `$|\omega|^{-2\alpha}$`.
Slopes vary from `$|\omega|^{-1}$` to `$|\omega|^{-3}$`, with `$|\omega|^{-2}$` over major currents.

Does not fit the conventional wisdom of only `$|\omega|^{-2n}$` slopes,
as in e.g. .cite[Berloff and McWilliams (2002b)]

---
class: left, .toc[[✧](../index.html)]
#Summary

The  Matérn process is the simplest random process that can simultaneously reproduce (i) the velocity variance (ii) the diffusivity and (iii) the spectral slope or degree of small-scale roughness.

This is a promising stochastic model for Lagrangian trajectories, where it is useful for parameteric spectral modeling, as a model for synthetic trajectories, as a null hypothesis in eddy studies.

The  Matérn process is conjectured to be of broad applicability.

--
The paper describing the  Matérn process is intended to serve as an accessible introduction to stochastic processes in general.

Code can be found in [{jLab}](http://www.jmlilly.net/jmlsoft.html).