Are bias weights needed in output layer?
Are bias weights needed in output layer? I am implementing a neural network. it is composed with LSTM and 1 dense output layer. I use bias weights for the output layer. but, I don't know it is really needed. LSTM has bias weights also. What is the main reason to use bias weights in output layer. Additionally, I'm using the 'sampled_softmax_loss' function. So, the output layer is just projection layer. There is no activation function such as ReLU. I tested both of models those are with bias and without bias, but the result was almost same. So, I want to know the bias is really effective at the final projection layer in general. Thanks in advance. Possible duplicate of Role of Bias in Neural Networks – Willy satrio nugroho Jun 26 at 7:12 How else do you plan on going from the last layer to the outpu...