[GitHub] MoritzMaxeiner commented on issue #8836: Backward shape inconsistent with custom HybridBlock and gluon.loss
MoritzMaxeiner commented on issue #8836: Backward shape inconsistent with custom HybridBlock and gluon.loss URL: https://github.com/apache/incubator-mxnet/issues/8836#issuecomment-379465681 @reminisce Sure, I just didn't close for the reasons mentioned [here](https://github.com/apache/incubator-mxnet/issues/8836#issuecomment-372075664) and [here](https://github.com/apache/incubator-mxnet/issues/8836#issuecomment-372076287). This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] MoritzMaxeiner commented on issue #8836: Backward shape inconsistent with custom HybridBlock and gluon.loss
MoritzMaxeiner commented on issue #8836: Backward shape inconsistent with custom HybridBlock and gluon.loss URL: https://github.com/apache/incubator-mxnet/issues/8836#issuecomment-379354690 @reminisce Ah, ok. W.r.t. reproducing: I haven't encountered the issue [in 1.0 or 1.1](https://github.com/apache/incubator-mxnet/issues/8836#issuecomment-372075664), so no I can't reproduce it in lastest code/release. Concerning the numerical result: That's hard to say precisely as it involves plently of (inherently inaccurate) floating point math (so I can't just calculate the operations another way and compare the results). If a guestimate is of use to you: The RNN trains and predicts as I would expect it to (in 1.0 and 1.1). This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] MoritzMaxeiner commented on issue #8836: Backward shape inconsistent with custom HybridBlock and gluon.loss
MoritzMaxeiner commented on issue #8836: Backward shape inconsistent with custom HybridBlock and gluon.loss URL: https://github.com/apache/incubator-mxnet/issues/8836#issuecomment-379354690 @reminisce Ah, ok. W.r.t. reproducing: I haven't encountered the issue [in 1.0 or 1.1](https://github.com/apache/incubator-mxnet/issues/8836#issuecomment-372075664), so no I can't reproduce it in lastest code/release. Concerning the numerical result: That's hard to say precisely as it involves plently of (inherently inaccurate) floating point math (so I can't just calculate the operations another way and compare the results). If a guestimate is acceptable: The RNN trains and predicts as I would expect it to (in 1.0 and 1.1). This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] MoritzMaxeiner commented on issue #8836: Backward shape inconsistent with custom HybridBlock and gluon.loss
MoritzMaxeiner commented on issue #8836: Backward shape inconsistent with custom HybridBlock and gluon.loss URL: https://github.com/apache/incubator-mxnet/issues/8836#issuecomment-379242737 @reminisce I'm unsure as to what you're asking, specifically. The script should ideally have terminated with exit code 0 and no stdout/stderr output, which is not what happened, so that was unexpected for me. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] MoritzMaxeiner commented on issue #8836: Backward shape inconsistent with custom HybridBlock and gluon.loss
MoritzMaxeiner commented on issue #8836: Backward shape inconsistent with custom HybridBlock and gluon.loss URL: https://github.com/apache/incubator-mxnet/issues/8836#issuecomment-372075664 For what it's worth, I don't have this issue with MXNet 1.0.0 nor 1.1.0, but I don't know if that's because the root cause (which is unknown to me) has been fixed, or if it just doesn't get triggered anymore, so I'm hesitant to close. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] MoritzMaxeiner commented on issue #8836: Backward shape inconsistent with custom HybridBlock and gluon.loss
MoritzMaxeiner commented on issue #8836: Backward shape inconsistent with
custom HybridBlock and gluon.loss
URL:
https://github.com/apache/incubator-mxnet/issues/8836#issuecomment-347384351
@reminisce I've removed the time unrolling (and the issue is still being
triggered), but if I remove either of the two cells, or the reshape operation,
the issue won't arise, so I don't think I can reduce it any further.
```python
import mxnet as mx
class Test(mx.gluon.HybridBlock):
def __init__(self, input_size, output_size, **kwargs):
super(Test, self).__init__(**kwargs)
self.input_size = input_size
self.output_size = output_size
self.hidden_unit_size = output_size*input_size
self.num_cells = 2
with self.name_scope():
self.cell_a = mx.gluon.rnn.GRUCell(self.hidden_unit_size,
input_size=input_size)
self.cell_b = mx.gluon.rnn.GRUCell(self.hidden_unit_size,
input_size=self.hidden_unit_size)
def hybrid_forward(self, F, inputs, states):
prev_h = states[0]
if F is mx.symbol:
prev_h = F.split(prev_h, axis=0, num_outputs=self.num_cells,
squeeze_axis=1)
cell_a_next_h, _ = self.cell_a(inputs, [prev_h[0]])
cell_b_next_h, _ = self.cell_b(prev_h[1], [prev_h[1]])
b_output = cell_b_next_h.reshape(shape=(0, self.input_size,
self.output_size))
return cell_a_next_h, b_output, []
def state_info(self, batch_size=0):
return [{'shape': (self.num_cells, batch_size,
self.hidden_unit_size), '__layout__': 'LNC'}]
def begin_state(self, batch_size=0, func=mx.ndarray.zeros, **kwargs):
states = []
for i, info in enumerate(self.state_info(batch_size)):
if info is not None:
info.update(kwargs)
else:
info = kwargs
states.append(func(name='%sh0_%d'%(self.prefix, i), **info))
return states
args_nof_examples = 1
args_nof_batches = 1
args_batch_size = 1
args_input_size = 1
args_output_size = 1
data = mx.ndarray.zeros(shape=(args_nof_examples, args_input_size))
labels = mx.ndarray.ones((args_nof_examples, args_input_size))
gen = mx.io.NDArrayIter(data, labels, args_batch_size,
last_batch_handle='discard')
with mx.cpu(0) as context:
model = Test(args_input_size, args_output_size)
model.initialize(mx.init.Xavier(), ctx = context)
model.hybridize()
loss = mx.gluon.loss.SoftmaxCrossEntropyLoss()
states = model.begin_state(args_batch_size)
for batch in gen:
with mx.autograd.record():
a, b, _ = model(batch.data[0], states)
L = loss(b, batch.label[0])
L.backward()
```
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] MoritzMaxeiner commented on issue #8836: Backward shape inconsistent with custom HybridBlock and gluon.loss
MoritzMaxeiner commented on issue #8836: Backward shape inconsistent with
custom HybridBlock and gluon.loss
URL:
https://github.com/apache/incubator-mxnet/issues/8836#issuecomment-347384351
@reminisce I've removed the time unrolling (and the issue is still
triggered), but if I remove either of the two cells, or the reshape operation,
the issue won't arise, so I don't think I can reduce it any further.
```python
import mxnet as mx
class Test(mx.gluon.HybridBlock):
def __init__(self, input_size, output_size, **kwargs):
super(Test, self).__init__(**kwargs)
self.input_size = input_size
self.output_size = output_size
self.hidden_unit_size = output_size*input_size
self.num_cells = 2
with self.name_scope():
self.cell_a = mx.gluon.rnn.GRUCell(self.hidden_unit_size,
input_size=input_size)
self.cell_b = mx.gluon.rnn.GRUCell(self.hidden_unit_size,
input_size=input_size)
def hybrid_forward(self, F, inputs, states):
prev_h = states[0]
if F is mx.symbol:
prev_h = F.split(prev_h, axis=0, num_outputs=self.num_cells,
squeeze_axis=1)
cell_a_next_h, _ = self.cell_a(inputs, [prev_h[0]])
cell_b_next_h, _ = self.cell_b(prev_h[1], [prev_h[1]])
b_output = cell_b_next_h.reshape(shape=(0, self.input_size,
self.output_size))
return cell_a_next_h, b_output, []
def state_info(self, batch_size=0):
return [{'shape': (self.num_cells, batch_size,
self.hidden_unit_size), '__layout__': 'LNC'}]
def begin_state(self, batch_size=0, func=mx.ndarray.zeros, **kwargs):
states = []
for i, info in enumerate(self.state_info(batch_size)):
if info is not None:
info.update(kwargs)
else:
info = kwargs
states.append(func(name='%sh0_%d'%(self.prefix, i), **info))
return states
args_nof_examples = 1
args_nof_batches = 1
args_batch_size = 1
args_input_size = 1
args_output_size = 1
data = mx.ndarray.zeros(shape=(args_nof_examples, args_input_size))
labels = mx.ndarray.ones((args_nof_examples, args_input_size))
gen = mx.io.NDArrayIter(data, labels, args_batch_size,
last_batch_handle='discard')
with mx.cpu(0) as context:
model = Test(args_input_size, args_output_size)
model.initialize(mx.init.Xavier(), ctx = context)
model.hybridize()
loss = mx.gluon.loss.SoftmaxCrossEntropyLoss()
states = model.begin_state(args_batch_size)
for batch in gen:
with mx.autograd.record():
a, b, _ = model(batch.data[0], states)
L = loss(b, batch.label[0])
L.backward()
```
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] MoritzMaxeiner commented on issue #8836: Backward shape inconsistent with custom HybridBlock and gluon.loss
MoritzMaxeiner commented on issue #8836: Backward shape inconsistent with
custom HybridBlock and gluon.loss
URL:
https://github.com/apache/incubator-mxnet/issues/8836#issuecomment-347384351
@reminisce I've removed the time unrolling (and the issue is still being
triggered), but if I remove either of the two cells, or the reshape operation,
the issue won't arise, so I don't think I can reduce it any further.
```python
import mxnet as mx
class Test(mx.gluon.HybridBlock):
def __init__(self, input_size, output_size, **kwargs):
super(Test, self).__init__(**kwargs)
self.input_size = input_size
self.output_size = output_size
self.hidden_unit_size = output_size*input_size
self.num_cells = 2
with self.name_scope():
self.cell_a = mx.gluon.rnn.GRUCell(self.hidden_unit_size,
input_size=input_size)
self.cell_b = mx.gluon.rnn.GRUCell(self.hidden_unit_size,
input_size=input_size)
def hybrid_forward(self, F, inputs, states):
prev_h = states[0]
if F is mx.symbol:
prev_h = F.split(prev_h, axis=0, num_outputs=self.num_cells,
squeeze_axis=1)
cell_a_next_h, _ = self.cell_a(inputs, [prev_h[0]])
cell_b_next_h, _ = self.cell_b(prev_h[1], [prev_h[1]])
b_output = cell_b_next_h.reshape(shape=(0, self.input_size,
self.output_size))
return cell_a_next_h, b_output, []
def state_info(self, batch_size=0):
return [{'shape': (self.num_cells, batch_size,
self.hidden_unit_size), '__layout__': 'LNC'}]
def begin_state(self, batch_size=0, func=mx.ndarray.zeros, **kwargs):
states = []
for i, info in enumerate(self.state_info(batch_size)):
if info is not None:
info.update(kwargs)
else:
info = kwargs
states.append(func(name='%sh0_%d'%(self.prefix, i), **info))
return states
args_nof_examples = 1
args_nof_batches = 1
args_batch_size = 1
args_input_size = 1
args_output_size = 1
data = mx.ndarray.zeros(shape=(args_nof_examples, args_input_size))
labels = mx.ndarray.ones((args_nof_examples, args_input_size))
gen = mx.io.NDArrayIter(data, labels, args_batch_size,
last_batch_handle='discard')
with mx.cpu(0) as context:
model = Test(args_input_size, args_output_size)
model.initialize(mx.init.Xavier(), ctx = context)
model.hybridize()
loss = mx.gluon.loss.SoftmaxCrossEntropyLoss()
states = model.begin_state(args_batch_size)
for batch in gen:
with mx.autograd.record():
a, b, _ = model(batch.data[0], states)
L = loss(b, batch.label[0])
L.backward()
```
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] MoritzMaxeiner commented on issue #8836: Backward shape inconsistent with custom HybridBlock and gluon.loss
MoritzMaxeiner commented on issue #8836: Backward shape inconsistent with
custom HybridBlock and gluon.loss
URL:
https://github.com/apache/incubator-mxnet/issues/8836#issuecomment-347384351
@reminisce I've removed the time unrolling, but if I remove either of the
two cells, or the reshape operation, the issue won't arise, so I don't think I
can reduce it any further.
```python
import mxnet as mx
class Test(mx.gluon.HybridBlock):
def __init__(self, input_size, output_size, **kwargs):
super(Test, self).__init__(**kwargs)
self.input_size = input_size
self.output_size = output_size
self.hidden_unit_size = output_size*input_size
self.num_cells = 2
with self.name_scope():
self.cell_a = mx.gluon.rnn.GRUCell(self.hidden_unit_size,
input_size=input_size)
self.cell_b = mx.gluon.rnn.GRUCell(self.hidden_unit_size,
input_size=input_size)
def hybrid_forward(self, F, inputs, states):
prev_h = states[0]
if F is mx.symbol:
prev_h = F.split(prev_h, axis=0, num_outputs=self.num_cells,
squeeze_axis=1)
cell_a_next_h, _ = self.cell_a(inputs, [prev_h[0]])
cell_b_next_h, _ = self.cell_b(prev_h[1], [prev_h[1]])
b_output = cell_b_next_h.reshape(shape=(0, self.input_size,
self.output_size))
return cell_a_next_h, b_output, []
def state_info(self, batch_size=0):
return [{'shape': (self.num_cells, batch_size,
self.hidden_unit_size), '__layout__': 'LNC'}]
def begin_state(self, batch_size=0, func=mx.ndarray.zeros, **kwargs):
states = []
for i, info in enumerate(self.state_info(batch_size)):
if info is not None:
info.update(kwargs)
else:
info = kwargs
states.append(func(name='%sh0_%d'%(self.prefix, i), **info))
return states
args_nof_examples = 1
args_nof_batches = 1
args_batch_size = 1
args_input_size = 1
args_output_size = 1
data = mx.ndarray.zeros(shape=(args_nof_examples, args_input_size))
labels = mx.ndarray.ones((args_nof_examples, args_input_size))
gen = mx.io.NDArrayIter(data, labels, args_batch_size,
last_batch_handle='discard')
with mx.cpu(0) as context:
model = Test(args_input_size, args_output_size)
model.initialize(mx.init.Xavier(), ctx = context)
model.hybridize()
loss = mx.gluon.loss.SoftmaxCrossEntropyLoss()
states = model.begin_state(args_batch_size)
for batch in gen:
with mx.autograd.record():
a, b, _ = model(batch.data[0], states)
L = loss(b, batch.label[0])
L.backward()
```
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] MoritzMaxeiner commented on issue #8836: Backward shape inconsistent with custom HybridBlock and gluon.loss
MoritzMaxeiner commented on issue #8836: Backward shape inconsistent with custom HybridBlock and gluon.loss URL: https://github.com/apache/incubator-mxnet/issues/8836#issuecomment-347370079 The issue is also present in 0.12.1. @reminisce I'll try to reduce it further. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] MoritzMaxeiner commented on issue #8836: Backward shape inconsistent with custom HybridBlock and gluon.loss
MoritzMaxeiner commented on issue #8836: Backward shape inconsistent with custom HybridBlock and gluon.loss URL: https://github.com/apache/incubator-mxnet/issues/8836#issuecomment-347362013 @reminisce Hm, I've tried out 1.0.0rc0 now and so far I haven't been able to reproduce the issue in that version. If there's an issue in master, I'd assume that to be a (separate) regression? This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] MoritzMaxeiner commented on issue #8836: Backward shape inconsistent with custom HybridBlock and gluon.loss
MoritzMaxeiner commented on issue #8836: Backward shape inconsistent with custom HybridBlock and gluon.loss URL: https://github.com/apache/incubator-mxnet/issues/8836#issuecomment-347349872 After further experimentation it seems that the reshape operation ```python F.reshape(a, shape=(0,1,self.feature_size, self.output_size) ``` is the issue here, as it works fine when I move it outside of the `hybrid_forward` and perform it on the NDArray result. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
