Search between a Gas Station

Neural network immediately overfitting I have a FFNN with 2 hidden layers for a regression task that overfits almost immediately (epoch 2-5, depending on # hidden units). (ReLU, Adam, MSE, same # hidden units per layer, tf.keras) 32 neurons: 128 neurons: I will be tuning the number of hidden units, but to limit the search space I would like to know what the upper and lower bounds should be. Afaik it is better to have a too large network and try to regularize via L2-reg or dropout than to lower the network's capacity -- because a larger network will have more local minima, but the actual loss value will be better. Is there any point in trying to regularize (via e.g. dropout) a network that overfits from the get-go? If so I suppose I could increase both bounds. If not I would lower them. model = Sequential() model.add(Dense(n_neurons, 'relu')) model.add(Dense(n_neurons, 'relu')) model.add(Dense(1, 'linear')) model.compile('adam', 'mse') ...

Neural Network - ValueError: Cannot feed value of shape I'm new in Python and Tensorflow . For the beginning I watched the MNIST tutorial and understood it so far. But now I have to create a new Neural Network with numerical input_datas. I got a dataset which delivers an input_data and v_data. If I run input_data.shape -> (1000,25,4) If I run v_data.shape -> (1000,2) What I tried to do is to split the data for (Training + Validation) and Testing. Training + Validation = 90% of train_data (90% of the input.pkl) Testing data = the remaining 10% And then I devided the 90% of the input_data in training and validation (70% training, 30% validation) The network should correctly predict based on v_data, but I still get an error. See the code and the error below. import os os.environ['TF_CPP_MIN_LOG_LEVEL'] = '2' # Imports import tensorflow as tf import pickle as pkl import numpy as np # load data with open('input.pkl', 'rb') as f: input_data = pkl...

Are bias weights needed in output layer? I am implementing a neural network. it is composed with LSTM and 1 dense output layer. I use bias weights for the output layer. but, I don't know it is really needed. LSTM has bias weights also. What is the main reason to use bias weights in output layer. Additionally, I'm using the 'sampled_softmax_loss' function. So, the output layer is just projection layer. There is no activation function such as ReLU. I tested both of models those are with bias and without bias, but the result was almost same. So, I want to know the bias is really effective at the final projection layer in general. Thanks in advance. Possible duplicate of Role of Bias in Neural Networks – Willy satrio nugroho Jun 26 at 7:12 How else do you plan on going from the last layer to the outpu...

Why do neural networks work so well? I understand all the computational steps of training a neural network with gradient descent using forwardprop and backprop, but I'm trying to wrap my head around why they work so much better than logistic regression. For now all I can think of is: A) the neural network can learn it's own parameters B) there are many more weights than simple logistic regression thus allowing for more complex hypotheses Can someone explain why a neural network works so well in general? I am a relative beginner. "work so well" is a subjective judgment. – Don Reba Jul 26 '16 at 16:53 that's true, what I meant is why do they work better than logistic regression? – Danny Liu Jul 26 '16 at 16:54 ...

Updated: reshape each row data into a (x, 1) array Recently I was reading neural network and deep learning by Michael Nielsen (link) and wanted to test the neural network on loan default data. However after quite a few tries I still did not manage to transform my csv format data into the required matrix format by the script. The csv file contains 769 variables and 1 boolean default entry. looks like this: . v1 v2 v3 ... v770; 1. 1 2 3 ... 0; 2. 2 1 2 ... 1; ... This is how I do my import: import numpy as np tr_input = [np.reshape(genfromtxt('training.csv', delimiter=','), (769,10000))] tr_res = np.reshape(genfromtxt('training2.csv', delimiter=','),(1, 10000)) tr_test = [np.reshape(genfromtxt('testing.csv', delimiter=','), (769,2000))] tr_test2 = np.reshape(genfromtxt('testing2.csv', delimiter=','), (1, 2000)) test_data = list(zip(tr_test, tr_test2)) training_data = list(zip(tr_input, tr_res)) However it returns Traceb...