Deep Learning notes:Techniques improving backpropogation

Deep Learning notes:Techniques improving backpropogation
1. a better choice of cost function
⇒ the cross-entropy cost function
2. "regularization" methods
    L1 
    L2 regularization, 
    dropout, 
    artificial expansion of the training data
3.  better method for initializing the weights in the network
4.  set of heuristics to help choose good hyper-parameters for the network