📄️ General Notes
All models are wrong, but some are useful.
📄️ Neural Networks
You cannot really use a linear function to capture 'loops' like in kNN. You need arbitrarily expressive functions. You want it to be parametric (doesn't grow with dataset) and is "efficiently auto-differentiable".
📄️ Activation Functions
| Function | Formula | Derivative | Range | Where used |