Function $f$ is defined as follows: \[f(x) = \begin{cases} x + 3 &\quad \text{if } x < -4, \\ x^2-4 &\quad \text{if } x \ge -4. \end{cases} \]For how many values of $x$ is $f(f(x)) = 5$?