Sparse coding

An ordinary least square regression does not apply regularization on the parameters we try to estimate. Often we regularize the optimization problem. This practice is know as shrinkage in statistics. The classic regularizer is the squared \(l_2\) norm of the parameters to estimate. This results in the familiar ridge regression problem.

Lasso stands for "Least Absolute Shrinkage and Selection Operator." It replaces the 2-norm in ridge regression with a 1-norm. Lasso is often used in sparse coding because it provides a sparse solution in the sense that many parameters are zero for large enough regularization factor.

References:
http://pages.cs.wisc.edu/~jerryzhu/cs731/regression.pdf

DFS in action

Solution to level 13 robotMaze in Untrusted: http://alex.nisnevich.com/untrusted/