Is it possible that some weight matrix does not have the right size?
It looks like it expects an input of size 54 but receives 864 instead.
If you took the code from the MNIST tutorial and changed the input
shape, you will have to at least adjust the size of the fully-connected
layer. For better results, you may need to change the architecture of
the network (number of layers, filter size, pooling factor, ...).
On Tue, Aug 30, 2016, Tony Lee wrote:
> Hi Pascal:
> Thanks for your reply which is really help for me! And I have solved this
> problem now. But I still have another problem when I train the model.
> the original data is r,g,b image with size of (3,50,50) for each
> Firstly I reshape the X_train and X_test as you can see in my code. After
> building the model, I want to train it. Here is the code of my training
> model.
>
> print "training and result:"
>
> for i in range(1):
>
> print "iteration %d" % (i + 1)
> for start in range(0, len(X_train), batch_size):
> x_batch = X_train[start:start + batch_size]
> y_batch = y_train[start:start + batch_size]
> cost = train(x_batch, y_batch)
>
> predictions_test = predict(X_test)
> accuracy = np.mean(predictions_test == y_test)
> print "accuracy: %.5f" % accuracy
>
>
> There are a few bug when I run this code:
>
> Traceback (most recent call last):
> File "/CNN.py", line 194, in <module>
> cost = train(x_batch, y_batch)
>
> ValueError: ('matrices are not aligned', (20, 864), (54, 70))
> Apply node that caused the error: Dot22(Reshape{2}.0,
> <TensorType(float64, matrix)>)
> Toposort index: 49
> Inputs types: [TensorType(float64, matrix), TensorType(float64, matrix)]
> Inputs shapes: [(20, 864), (54, 70)]
> Inputs strides: [(6912, 8), (560, 8)]
> Inputs values: ['not shown', 'not shown']
> Outputs clients: [[Elemwise{maximum,no_inplace}(Dot22.0,
> TensorConstant{(1, 1) of 0.0}), Elemwise{Composite{(i0 * EQ(i1, i2) *
> i3)}}[(0, 0)](Dot22Scalar.0, Elemwise{maximum,no_inplace}.0, Dot22.0,
> Elemwise{Composite{Cast{float64}(LT(i0, i1))}}[(0, 0)].0)]]
>
> HINT: Re-running with most Theano optimization disabled could give you
> a back-trace of when this node was created. This can be done with by
> setting the Theano flag 'optimizer=fast_compile'. If that does not
> work, Theano optimizations can be disabled with 'optimizer=None'.
> HINT: Use the Theano flag 'exception_verbosity=high' for a debugprint
> and storage map footprint of this apply node.
>
>
> Is it because of my the wrong process when I train the model? Could
> you please tell me how to train the model? Which kind of data format I
> should have on this "cost = train(x_batch, y_batch)" part? I am
> looking forward to your reply. Thank you!
>
>
> Best,
>
> Tony
>
>
>
> On Tue, Aug 30, 2016 at 7:17 PM, Pascal Lamblin <[email protected]>
> wrote:
>
> > For some reason, some of the 4D arrays in your graph seem to be arrays
> > of type int64, which throws off the logic trying to ensure gradients
> > have the same precision as their forward equivalent.
> >
> > You can try to use theano.printing.debugprint(cost, print_type=True) to
> > find out which variables are int.
> >
> > On Tue, Aug 30, 2016, [email protected] wrote:
> > > Hi, everyone:
> > > I am writing CNN code but there is an error when I compute T.grad in the
> > > Theano. Hopefully you can help me to analysis it. I don't know why there
> > is
> > > an error and how should I modify it. If you can help me, I will be
> > > appreciate of you! Here is some details of dataset. There is a lot of
> > > images which has R,G,B for each. And both width and height of each image
> > > are 50. I am waiting for your reply :)
> > > Here is my code
> > >
> > > # function definition for CNN
> > > srng = RandomStreams()
> > > def floatX(X):
> > > return np.asarray(X, dtype=theano.config.floatX)
> > >
> > > def init_weights(shape):
> > > return theano.shared(floatX(np.random.randn(*shape) * 0.01))
> > >
> > > def dropout(X, p_use=1.):
> > > if p_use < 1:
> > > p_sampled = srng.binomial(p=p_use, n=1, size=X.shape,
> > dtype=theano.config.floatX)
> > > X = X * p_sampled / p_use
> > > return X
> > >
> > > def rectify(X):
> > > return T.maximum(X, 0.)
> > >
> > > def PRelu(X,a):
> > > return T.maximum(X, 0.) + a * T.minimum(X, 0.)
> > >
> > > def softmax(X):
> > > e_x = T.exp(X - X.max(axis=1).dimshuffle(0, 'x'))
> > > print e_x
> > > return e_x / e_x.sum(axis=1).dimshuffle(0, 'x')
> > >
> > > def RMSprop(cost, params, lr=0.001, rho=0.9, epsilon=1e-6):
> > > grads = T.grad(cost=cost, wrt=params)
> > >
> > > updates = []
> > > for p, g in zip(params, grads):
> > > acc = theano.shared(p.get_value() * 0.)
> > > acc_new = rho * acc + (1 - rho) * g ** 2
> > > gradient_scaling = T.sqrt(acc_new + epsilon)
> > > g = g / gradient_scaling
> > > updates.append((acc, acc_new))
> > > updates.append((p, p - lr * g))
> > > return updates
> > >
> > > # model building
> > > X = T.ftensor4('x')
> > > Y = T.fmatrix('y')
> > >
> > > # parameters initialization
> > > X_train = X_train.reshape(-1, 3, 50, 50)
> > > X_test = X_test.reshape(-1, 3, 50, 50)
> > > W_conv1 = init_weights((4, 3, 5, 5))
> > > b_conv1 = np.zeros((4,))
> > > W_conv2 = init_weights((6, 4, 3, 3))
> > > b_conv2 = np.zeros((6,))
> > > W_fcn = init_weights((54, 70))
> > > b_fcn = np.zeros((70,))
> > > W_fcn2 = init_weights((70, 43))
> > > b_fcn2 = np.zeros((43,))
> > >
> > > # convolution and pooling
> > > maxpool_shape = (2, 2)
> > > p_drop_input = 0.8
> > > conv_layer1 = rectify(conv2d(X_train, W_conv1, border_mode='full'))
> > > subsampling_layer1 = pool_2d(conv_layer1, maxpool_shape,
> > ignore_border=True)
> > > out_layer1 = subsampling_layer1
> > > out_layer1 = dropout(subsampling_layer1, p_drop_input)
> > >
> > > p_drop_hidden = 0.6
> > > conv_layer2 = rectify(conv2d(out_layer1, W_conv2, border_mode='valid'))
> > > subsampling_layer2 = pool_2d(conv_layer2, maxpool_shape,
> > ignore_border=True)
> > > out_layer2 = dropout(subsampling_layer2, p_drop_hidden)
> > > conv_out = T.flatten(out_layer2, outdim = 2)
> > >
> > > # fully connected NN
> > > hidden = rectify(T.dot(conv_out, W_fcn))
> > > hidden = dropout(hidden, p_drop_hidden)
> > >
> > > py_x = softmax(T.dot(hidden, W_fcn2))
> > > y_x = T.argmax(py_x, axis=1)
> > >
> > > # compute cost and update
> > > cost = T.mean(T.nnet.categorical_crossentropy(py_x , Y))
> > > params = [W_conv1, W_conv2, W_fcn, W_fcn2]
> > > print cost
> > > print params
> > > updates = RMSprop(cost, params, lr=0.001)
> > >
> > > And the error is that:
> > >
> > > Traceback (most recent call last):
> > > File "/PycharmProjects/CNN/CNN.py", line 180, in <module>
> > > updates = RMSprop(cost, params, lr=0.001)
> > > File "/PycharmProjects/CNN/CNN.py", line 116, in RMSprop
> > > grads = T.grad(cost=cost, wrt=params)
> > > File "/Library/Python/2.7/site-packages/theano/gradient.py", line
> > 561, in grad
> > > grad_dict, wrt, cost_name)
> > > File "/Library/Python/2.7/site-packages/theano/gradient.py", line
> > 1324, in _populate_grad_dict
> > > rval = [access_grad_cache(elem) for elem in wrt]
> > > File "/Library/Python/2.7/site-packages/theano/gradient.py", line
> > 1279, in access_grad_cache
> > > term = access_term_cache(node)[idx]
> > > File "/Library/Python/2.7/site-packages/theano/gradient.py", line
> > 1113, in access_term_cache
> > > input_grads = node.op.grad(inputs, new_output_grads)
> > > File
> > > "/Library/Python/2.7/site-packages/theano/tensor/nnet/abstract_conv.py",
> > line 828, in grad
> > > d_bottom = bottom.type.filter_variable(d_bottom)
> > > File "/Library/Python/2.7/site-packages/theano/tensor/type.py", line
> > 233, in filter_variable
> > > self=self))
> > > TypeError: Cannot convert Type TensorType(float64, 4D) (of Variable
> > AbstractConv2d_gradInputs{border_mode='full', subsample=(1, 1),
> > filter_flip=True, imshp=(None, None, None, None), kshp=(None, None, None,
> > None)}.0) into Type TensorType(int64, 4D). You can try to manually convert
> > AbstractConv2d_gradInputs{border_mode='full', subsample=(1, 1),
> > filter_flip=True, imshp=(None, None, None, None), kshp=(None, None, None,
> > None)}.0 into a TensorType(int64, 4D).
> > >
> > > Process finished with exit code 1
> > >
> > > --
> > >
> > > ---
> > > You received this message because you are subscribed to the Google
> > Groups "theano-users" group.
> > > To unsubscribe from this group and stop receiving emails from it, send
> > an email to [email protected].
> > > For more options, visit https://groups.google.com/d/optout.
> >
> >
> > --
> > Pascal
> >
> > --
> >
> > ---
> > You received this message because you are subscribed to a topic in the
> > Google Groups "theano-users" group.
> > To unsubscribe from this topic, visit https://groups.google.com/d/
> > topic/theano-users/0hUbG1vy_hU/unsubscribe.
> > To unsubscribe from this group and all its topics, send an email to
> > [email protected].
> > For more options, visit https://groups.google.com/d/optout.
> >
>
> --
>
> ---
> You received this message because you are subscribed to the Google Groups
> "theano-users" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to [email protected].
> For more options, visit https://groups.google.com/d/optout.
--
Pascal
--
---
You received this message because you are subscribed to the Google Groups
"theano-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
For more options, visit https://groups.google.com/d/optout.
