## quinta-feira, 3 de abril de 2014

### Sigmoid Function, Derivative and Limits

We have two objectives. First, we need derivate the sigmoid function:

$S(x) = \frac {1}{1 + e^{-px}}$

using very simple techniques.

Second, we must to find the more right and more left function limits:

$\lim_{x \to +\infty }{S(x)} = 1$ e $\lim_{x \to -\infty }{S(x)} = 0$

Derivative sigmoid

We are looking for

$\frac{\mathrm{d} S(x)}{\mathrm{d} x} = S'(x)$.

To deriving it, we'll apply the quocient rule, which is remembered here:

$\left ( \frac{f}{g} \right )' = \frac{f'g - g'f}{g^2}$

Said that, taking:

$f(x) = 1 \Rightarrow f'(x) = 0$
and
$g(x) = 1 + e^{-px} \Rightarrow g'(x) = -pe^{-px}$

Then,

$S'(x) = \left ( \frac{f}{g} \right )' = \frac{0g - g'1}{g^2} = \frac{pe^{-px}}{(1 + e^{-px})^2}$

Objective reached. But if you try Wolfram Alpha to do it, you'll take a different answer:

Ok, no problem. Let's show that both answers are equivalent.

$= \frac{pe^{-px}}{(1 + e^{-px})^2}$

$= \frac{(e^{2px})pe^{-px}}{(e^{2px})(1 + 2e^{-px} + e^{-2px})}$

$= \frac{pe^{px}}{e^{2px} + 2e^{px} + 1}$

And finally

$= \frac{pe^{px}}{(e^{px} + 1)^2}$

Limits in $+\infty$ and $-\infty$

The two limits are easy to do:

$lim_{x \to +\infty }{\frac {1}{1 + e^{-px}}} = \frac {1}{1 + \frac{1}{\infty}} = \frac {1}{1 + 0} = 1$

$lim_{x \to -\infty }{\frac {1}{1 + e^{-px}}} = \frac {1}{1 + e^{p\infty}} = \frac {1}{1 + \infty} = 0$