### Background

Kullback Leibler:
$$KL(p||q) = \sum\nolimits_{k=1}^K{p_k}\ln\frac{p_k}{q_k} \geq 0$$
is used to calculate the distance between two distributions where:

$$\sum\nolimits_{k=1}^K{q_k}=1$$

$$\sum\nolimits_{k=1}^K{p_k} = 1$$

$${p_k} \geq 0$$

$${q_k} \geq 0$$

### Proof

$${p_k}·\ln\frac{p_k}{q_k} = -{p_k}·\ln\frac{q_k}{p_k}$$

According to:

$$-\ln x \geq {1 - x}$$

we can get the following equation:

$$-p_k\ln\frac{q_k}{p_k} \geq p_k(1 - \frac{q_k}{p_k})$$

$$\sum\nolimits_{k=1}^K{p_k}\ln\frac{p_k}{q_k} \geq \sum\nolimits_{k=1}^K{p_k}-\sum\nolimits_{k=1}^K{q_k}\frac{p_k}{p_k} = 0$$