Search between a Gas Station

Neural network immediately overfitting I have a FFNN with 2 hidden layers for a regression task that overfits almost immediately (epoch 2-5, depending on # hidden units). (ReLU, Adam, MSE, same # hidden units per layer, tf.keras) 32 neurons: 128 neurons: I will be tuning the number of hidden units, but to limit the search space I would like to know what the upper and lower bounds should be. Afaik it is better to have a too large network and try to regularize via L2-reg or dropout than to lower the network's capacity -- because a larger network will have more local minima, but the actual loss value will be better. Is there any point in trying to regularize (via e.g. dropout) a network that overfits from the get-go? If so I suppose I could increase both bounds. If not I would lower them. model = Sequential() model.add(Dense(n_neurons, 'relu')) model.add(Dense(n_neurons, 'relu')) model.add(Dense(1, 'linear')) model.compile('adam', 'mse') ...

Neural Network - ValueError: Cannot feed value of shape I'm new in Python and Tensorflow . For the beginning I watched the MNIST tutorial and understood it so far. But now I have to create a new Neural Network with numerical input_datas. I got a dataset which delivers an input_data and v_data. If I run input_data.shape -> (1000,25,4) If I run v_data.shape -> (1000,2) What I tried to do is to split the data for (Training + Validation) and Testing. Training + Validation = 90% of train_data (90% of the input.pkl) Testing data = the remaining 10% And then I devided the 90% of the input_data in training and validation (70% training, 30% validation) The network should correctly predict based on v_data, but I still get an error. See the code and the error below. import os os.environ['TF_CPP_MIN_LOG_LEVEL'] = '2' # Imports import tensorflow as tf import pickle as pkl import numpy as np # load data with open('input.pkl', 'rb') as f: input_data = pkl...

Why use constant i in `tf.while_loop` as `loop_vars`? The while_loop is like this: i = tf.constant(0) c = lambda i: tf.less(i, 10) b = lambda i: tf.add(i, 1) r = tf.while_loop(c, b, [i]) i is used as a incremental variable. So, i is changeable. Why we define i as a constant? Why not i = tf.Variable(0, tf.int32) i i i i = tf.Variable(0, tf.int32) 2 Answers 2 i is used as a incremental variable. So, i is changeable. i i Not quite true — if it was, i could not be a constant. i tf.add(i, 1) does not change i , it takes tensor i and creates a new tensor by adding 1 to it. (In practice, tensorflow is likely to reuse the same memory allocation for the resulting tensor, but that is an optimization irrelevant to the logic of tf.where ). tf.add(i, 1) i i tf.where You may be confused because the same name i is used in lambdas, but all those tensors are different tensors corresponding to the output of...

How to multiply k 2x2 Matrices by k 2x2 matrices in tensorflow? I have 2 3D tensors in tensorflow, where the two tensors have the shape Kx2x2. The tensors represent a set of 2x2 matrices. Is there a way to multiply the 2x2 matrices in the first tensor with the corresponding matrix in the second so that I get a Kx2x2 tensor in the end? 3 Answers 3 You can use tf.matmul . tf.matmul c = tf.matmul(a, b) No, in the question I asked about matrix multiplication. The documentation of tf.multiply states: "Returns x * y element-wise." – Todor Kostov Jun 30 at 22:57 Ok. Maybe a bit more detail on the question should help. let a,b and c are Kx2x2 tensors. Now what I need is: c[i,:,:] = tf.matmul(a[i,:,:]...

How to deploy distributed Tensorflow Slim example on multiple nodes (CPUs only) I have implemented and deployed a CNN training example on my cluster (multiple hosts/nodes)following the tutorial distributed TensorFlow tutorial. Now I want to run a Tensorflow Slim example in a cluster which contains a few hosts/nodes (CPUs only). In the example code of distributed TensorFlow tutorial, I can use --ps_hosts, --worker_host, --job_name to specify a cluster and a particular job type (ps or worker). --ps_hosts, --worker_host, --job_name However, in the train_image_classifier.py I did not find arguments via which I can specify the cluster and job name. Here is the tutorial for deploy TF slim:TF Slim Deploy . I was wondering if the current TF slim library support deploying a training job on multiple nodes. If yes, how to launch a distributed TF slim job on a cluster? It would be good if you could provide some example code/scripts, just like the code example in distributed TensorFlow tutorial. ...

how to read batches in one hdf5 data file for training? I have a hdf5 training dataset with size (21760, 1, 33, 33) . 21760 is the whole number of training samples. I want to use the mini-batch training data with the size 128 to train the network. (21760, 1, 33, 33) 21760 128 I want to ask: How to feed 128 mini-batch training data from the whole dataset with tensorflow each time? 128 3 Answers 3 You can read the hdf5 dataset into a numpy array, and feed slices of the numpy array to the TensorFlow model. Pseudo code like the following would work : import numpy, h5py f = h5py.File('somefile.h5','r') data = f.get('path/to/my/dataset') data_as_array = numpy.array(data) for i in range(0, 21760, 128): sess.run(train_op, feed_dict={input:data_as_array[i:i+128, :, :, :]}) Thank you. But when the number of training iterations i is large, e.g. 100000, ...