Jack Dermody

Gradient

Extending Bright Wire: Custom Activation Function

More specifically, the gradient from the activation function will not vanish (as with RELU) as the layers get deeper.