趣味の研究

趣味の数学を公開しています。初めての方はaboutをご覧ください。

Mixture distribution(e.g. GMM) and wavelet transform

Can we represent the arbitrary function by mixture distribution?

I thought this problem by using the analogy of wavelet transform.

Definition.

Let $p(x)$ be a probability density function in $\mathrm{R^n}$ .

$p_{a,b}(x)=\Pi_k|a_k|^{-1}p(\frac{x-b}{a})$

for distribution function $p(x)$

$p(x)$ indicates $p(x_1,x_2,\cdot\cdot\cdot,x_n)$ .

In the following, we omitte the arguments of function in the same way.

We confirm $p_{a,b}$ is probability density function easily.

Proposition

Let $p(x)\in L^2(\mathrm{R^n})$ be probability density function.

and $C^2$ class function such that

$C_p=\int\Pi_k|\omega_k| |\hat{p}(\omega)|^2d^n\omega$ < $+\infty$

We can expand any probability density function $f(x)\in L^2(\mathrm{R^n})$ ,

$f(x)=\int_{\mathrm{R^n}}d^na\int_{\mathrm{R^n}}d^nbF(a,b)p_{a,b}(x)$ (1)

We represent the expansion coefficients as below.

$F(a,b)=-\frac{\Pi_k|a_k|}{C_p}\int[\Pi_k\frac{d^2}{dt_k^2}f(t)] p_{a,b}(t)d^nt$ (2)

We ommitte the integral range in the case the integral range is $\mathrm{R^n}$ .

Where, $\hat{p}(\omega)$ is Fourier transform of $p(x)$ .

We have Gaussian mixture model if $p(x)$ is normal distribution p.d.f.

Proof.

We can prove in the same way as continuous wavelet transform.([1]Theorem4.4)

The right integral

$q(x)=\int d^na\int d^nbF(a,b)p_{a,b}(x)$ in (1) can be rewritten as the sum of convolution.

$q(x)=\int d^na\int d^nbF(a,b)\Pi_k |a_k|^{-1} p(\frac{x-b}{a})=\int d^na\Pi_k |a_k|^{-1} F(a,\cdot\cdot)\star p_a(x)$

$\star$ indicates convolution, the "." indicates the variable over which the convolution is caliculated.

We define $p_a(x)=p(\frac{x}{a})$ .

In the same way, we have

$F(a,b)=-\frac{1}{C_p}[\Pi_k\frac{d^2}{dt_k^2}f]\star \tilde{p_a}(b)$

where, $\tilde{p_a}(x)=p_a(-x)$ .

Combining the results,

$q(x)=-\frac{1}{C_p}\int d^na \Pi_k|a_k|^{-1}[\Pi_k\frac{d^2}{dt_k^2}f]\star\tilde{p_a}\star p_a(x)$ .

Using Fourier transform formula, we have

$\hat{p_a}(\omega)=\Pi_k a_k\hat{p}(a\omega)$

and

$\hat{\tilde{p_a}}(\omega)=\hat{p_a}(-\omega)=\overline{\hat{p_a}(\omega)}$ .

$\overline{p}$ indicates the complex conjugate of $p$ and we use $p_a$ is a real function.

Fourier transform of $q(x)$ is

$\hat{q}(\omega)=\frac{\hat{f}(\omega)}{C_p}\int d^na \Pi_k|a_k|^{-1}a_k^2\omega_k^2 |\hat{p}(a\omega)|^2$ .

The change of variable $a'_k=\omega_k a_k$ and using $|\hat{p}(\omega)|^2=|\hat{p}(-\omega)|^2$ , we have

$\hat{q}(\omega)=\hat{f}(\omega)$ . (3)

By inverse Fourier transform to (3), we prove the Proposition.

Reference.

[1] St´ephane Mallat., "A Wavelet Tour of Signal Processing"