### Background

Kullback Leibler:

$$

KL(p||q) = \sum\nolimits_{k=1}^K{p_k}\ln\frac{p_k}{q_k} \geq 0

$$

is used to calculate the distance between two distributions where:

$$

\sum\nolimits_{k=1}^K{q_k}=1

$$

$$

\sum\nolimits_{k=1}^K{p_k} = 1

$$

$$

{p_k} \geq 0

$$

$$

{q_k} \geq 0

$$

### Proof

$$

{p_k}·\ln\frac{p_k}{q_k} = -{p_k}·\ln\frac{q_k}{p_k}

$$

According to:

$$

-\ln x \geq {1 - x}

$$

we can get the following equation:

$$

-p_k\ln\frac{q_k}{p_k} \geq p_k(1 - \frac{q_k}{p_k})

$$

And go ahead:

$$

\sum\nolimits_{k=1}^K{p_k}\ln\frac{p_k}{q_k} \geq \sum\nolimits_{k=1}^K{p_k}-\sum\nolimits_{k=1}^K{q_k}\frac{p_k}{p_k} = 0

$$