Flow

For a probability density $p(x)$ and a transformation of coordinate $x=g(z)$ or $z=f(x)$, the density can be expressed using the coordinate transformations, i.e.,

$$\begin{array}{}\text{(1)}& p(x)& =\tilde{p}(f(x))|\det \mathrm{D}g(f(x)){|}^{-1}\text{(2)}& & =\tilde{p}(f(x))|\det \mathrm{D}f(x)|\end{array}$$

where the Jacobian is

$$\mathrm{D}g(z)\to \frac{\partial }{\partial z}g.$$

The operation $g_{\ast }\circ \tilde{p}(z)$ is the push forward of $\tilde{p}(z)$. The operation $g_{\ast }$ will pushforward simple distribution $\tilde{p}(z)$ to a more complex distribution $p(x)$.

• The generative direction: sample $z$ from distribution $\tilde{p}(z)$, apply transformation $g(z)$;
• The normalizing direction: "simplify" $p(x)$ to some simple distribution $\tilde{p}(z)$.

The key to the flow model is the chaining of the transformations

$$\det \mathrm{D}f(x)={\mathrm{\Pi }}_{i=1}^N\det \mathrm{D}{f}_i(x_i)$$

where

$$\begin{array}{}\text{(3)}& x_i& =g_i\circ \cdots \circ g_1(z)\text{(4)}& & =f_{i+1}\circ \cdots \circ f_N(x).\end{array}$$

---

1. Liu X, Zhang F, Hou Z, Wang Z, Mian L, Zhang J, et al. Self-supervised Learning: Generative or Contrastive. arXiv [cs.LG]. 2020. Available: http://arxiv.org/abs/2006.08218 ↩

---

Contributors: LM