Neural Network Activation Functions Demo
Visualizing the effect of different activation functions on a simple neural network
Hidden Layer 1 Size:
Hidden Layer 2 Size:
Resize Network
Simplified View
Activation Function:
None
Step
Sigmoid
Tanh
ReLU
Target Function:
None
Linear (y = x)
Quadratic (y = x²)
Sine (y = sin(x))
Cosine (y = cos(x))
Absolute Value (y = |x|)
Composite Trig (y = sin(x/2) + cos(x²))
Gaussian (y = e^(-(x²)/2))
Rational (y = x/(1+x²))
Step with Noise (y = step(x) + 0.1*sin(5x))
Square Wave (y = (2/π)*asin(sin(x)))
Frequency Modulation (y = sin(x + 0.5*sin(2x)))
Decaying Sine (y = sin(x)*e^(-0.1x²))
Bumpy Function (y = sin(x)*(1-cos(x)))
Input Value:
Train Step
Start Training
Learning Rate:
Batch Size:
L2 Regularization
Error:
N/A
Steps:
0