Activation Functions & Optimization
1. What is the primary role of activation functions in neural networks?
2. Which activation function outputs values between 0 and 1?
3. What does SGD stand for?
4. What is the learning rate in optimization algorithms?
5. Which optimizer adjusts the learning rate individually for each parameter?
6. What problem is commonly associated with Sigmoid and Tanh activation functions?
7. Which activation function outputs zero for negative inputs?
8. Which activation function is zero-centred?
9. Which optimizer uses momentum and adaptive learning rates?
10. Which activation function helps reduce the vanishing gradient problem?