Python/Keras/Neural Networks - Basics to make it run -

i'm beginner working keras make predictions. understand concept , theory behind it. i'm having hard time make run. @ stage i'm not worried efficiency of it, want run , see output can evolve later.

i have dummy pandas dataframe i'm using predictor_train (x):

                 value        1lag        2lag        3lag        4lag... date                                                                      2005-04-01  231.721933  165.195418  170.418903  225.892387  206.282539    2005-05-01  163.259812  231.721933  165.195418  170.418903  225.892387    2005-06-01  211.649963  163.259812  231.721933  165.195418  170.418903    2005-07-01  249.054951  211.649963  163.259812  231.721933  165.195418    2005-08-01  168.657539  249.054951  211.649963  163.259812  231.721933    2005-09-01  179.623448  168.657539  249.054951  211.649963  163.259812    2005-10-01  228.437842  179.623448  168.657539  249.054951  211.649963    2005-11-01  165.805266  228.437842  179.623448  168.657539  249.054951 ... [129 rows x 96 columns] 

i have other dataframe i'm using target_train (y):

date 2005-04-01   -0.01136 2005-05-01    0.04713 2005-06-01    0.00329 2005-07-01   -0.00985 2005-08-01    0.05635 2005-09-01   -0.03766 2005-10-01    0.01848 2005-11-01   -0.01387 [129 rows x 1 column] 

i'm using following code:

from keras.models import sequential keras.layers.core import dense, activation   model=sequential()  model.add(dense(output_dim=64, input_dim=100, init="glorot_uniform")) model.add(activation("tanh")) model.add(dense(output_dim=10, init="glorot_uniform")) model.add(activation("linear")) model.compile(loss="mean_squared_error", optimizer="rmsprop") model.fit(predictor_train, target_train, nb_epoch=5, batch_size=32,show_accuracy=true) prediction=model.predict(predictor_train)  print prediction 

and i'm getting following error:

file "/users/file.py", line 1271, in var_neural_reg1 model.fit(predictor_train, target_train, nb_epoch=5, batch_size=32,show_accuracy=true) file "/library/python/2.7/site-packages/keras/models.py", line 581, in fit shuffle=shuffle, metrics=metrics) file "/library/python/2.7/site-packages/keras/models.py", line 230, in _fit ins_batch = slice_x(ins, batch_ids) file "/library/python/2.7/site-packages/keras/models.py", line 65, in slice_x return [x[start] x in x] file "/library/python/2.7/site-packages/pandas/core/frame.py", line 1908, in __getitem__ return self._getitem_array(key) file "/library/python/2.7/site-packages/pandas/core/frame.py", line 1953, in _getitem_array return self.take(indexer, axis=1, convert=true) file "/library/python/2.7/site-packages/pandas/core/generic.py", line 1370, in take convert=true, verify=true) file "/library/python/2.7/site-packages/pandas/core/internals.py", line 3508, in take indexer = maybe_convert_indices(indexer, n) file "/library/python/2.7/site-packages/pandas/core/indexing.py", line 1721, in maybe_convert_indices raise indexerror("indices out-of-bounds")  indexerror: indices out-of-bounds 

any insights on how make mammoth move?

so, don't want set input_dim 129. that's number of samples. keras doesn't care if data has 1 row or 1 million rows; that's not dimensionality of data. if data numpy array, input shape (1, 2, 3) = (2, 3). number of rows (1) doesn't make difference.

but, that's not issue here. error you're receiving doesn't have specified dimensionality. if case, you'd receiving different error (valueerror , not indexerror) traces backend (theano or tensorflow), if there in fact shape mismatch between data , specified in model. example:

valueerror: ('shapes (x,x) , (x,x) not aligned: x (dim 1) != x (dim 0)', (x, x), (x, x)) 

looking @ error message, problem how you're preprocessing data; i.e., problem pandas , not keras. you're using keras' slice_x function segment data, when keras talks pandas , tells want data dataframe[x:y], pandas responding indices you've specified aren't valid given dataframe.

after that's sorted, though, might run different errors once program gets keras functions you're calling in model.fit. specifically, you'll see shape mismatch between data , model expecting. how columns being encoded before being fed keras model? keras expecting numpy array x , y in model.fit. also, kind of output expecting such output_dim = 10?

i recommend this keras example shows sequence sequence prediction lstm model, in case learning addition, since want; i.e., it's concrete example shows model running , evolving @ every epoch.

-------------------------------------------------- iteration 1 train on 45000 samples, validate on 5000 samples epoch 1/1 45000/45000 [==============================] - 30s - loss: 1.6745 - acc:0.3927- val_loss: 1.5373 - val_acc: 0.4320 q 58333+69862 t 128195 ☒ 13335  --- . . . --- q 83911+65    t 83976  ☒ 11115  --- 

the model starts learning what's around epoch 7:

iteration 7 train on 45000 samples, validate on 5000 samples epoch 1/1 45000/45000 [==============================] - 31s - loss: 0.9129 - acc: 0.6549 - val_loss: 0.9117 - val_acc: 0.6510 q 80+3375     t 3455   ☒ 3420   --- . . . --- q 6+50853     t 50859  ☑ 50859    <-------------------------- --- 

you can add following print line decode function in charactertable class following in order see what's happening. in case, source addition questions inputted in reverse, improves performance in type of model introducing short term dependencies between inputs , targets (sutskever, vinyals, , le 2014)

    def decode(self, x, calc_argmax=true):         if calc_argmax:             x = x.argmax(axis=-1)         print('{0} --> {1}'.format(x, [self.indices_char[x] x in x]))         return ''.join(self.indices_char[x] x in x) 

for example:

iteration 3 train on 45000 samples, validate on 5000 samples epoch 1/1 45000/45000 [==============================] - 21s - loss: 1.4865 - acc: 0.4491 - val_loss: 1.4132 - val_acc: 0.4676 [ 0 11 12  2  4 11 12] --> [' ', '7', '8', '+', '0', '7', '8'] [13  9 11  0] --> ['9', '5', '7', ' '] [12  4 11  0] --> ['8', '0', '7', ' '] q 870+87  t 957  ☒ 807  --- [ 0  8 10  3  8  4 10] --> [' ', '4', '6', '-', '4', '0', '6'] [9 8 4 0] --> ['5', '4', '0', ' '] [10  4 13  0] --> ['6', '0', '9', ' '] q 604-64  t 540  ☒ 609  --- 

this encoder/decoder model can changed quite adapt other kinds of sequence sequence prediction problems.

you'd want revise code fit data. but, in case, if going feed above data model is, believe input_dim len('0123456789.- '), since that's dimensionality of data; i.e., set of characters in inputs , targets. in linked example, sequences converted one-hot encodings of individual characters.

so, you'd specify input shape in first layer of model:

model.add(lstm(hidden_dimensions, input_shape=(max_length, input_dimensionality)))