leleamol commented on issue #15268: Backward doesn't work on LSTM with
sequence_length
URL:
https://github.com/apache/incubator-mxnet/issues/15268#issuecomment-503331624
I could reproduce this issue. Here is a full callstack.
ubuntu@ip-172-31-31-181:~$ python lstm_test.py
[1.387 2.887 3.000114 4.000115 5.68 6.405 7. ]
Traceback (most recent call last):
File "lstm_test.py", line 42, in
mx.nd.waitall()
File "/home/ubuntu/incubator-mxnet/python/mxnet/ndarray/ndarray.py", line
166, in waitall
check_call(_LIB.MXNDArrayWaitAll())
File "/home/ubuntu/incubator-mxnet/python/mxnet/base.py", line 253, in
check_call
raise MXNetError(py_str(_LIB.MXGetLastError()))
mxnet.base.MXNetError: [22:04:02] src/operator/./rnn-inl.h:1006: Check
failed: in_data.size() == num_inputs (4 vs. 5) :
Stack trace:
[bt] (0)
/home/ubuntu/incubator-mxnet/python/mxnet/../../lib/libmxnet.so(dmlc::LogMessageFatal::~LogMessageFatal()+0x32)
[0x7ffa222a1082]
[bt] (1)
/home/ubuntu/incubator-mxnet/python/mxnet/../../lib/libmxnet.so(mxnet::op::RNNOp::Backward(mxnet::OpContext const&, std::vector > const&, std::vector > const&, std::vector > const&, std::vector > const&, std::vector > const&)+0x1dc) [0x7ffa26c335bc]
[bt] (2)
/home/ubuntu/incubator-mxnet/python/mxnet/../../lib/libmxnet.so(void
mxnet::op::RNNStatefulGradCompute(mxnet::OpStatePtr const&,
mxnet::OpContext const&, std::vector
> const&, std::vector >
const&, std::vector >
const&)+0x21b6) [0x7ffa26c68186]
[bt] (3)
/home/ubuntu/incubator-mxnet/python/mxnet/../../lib/libmxnet.so(mxnet::imperative::PushOperator(mxnet::OpStatePtr
const&, nnvm::Op const*, nnvm::NodeAttrs const&, mxnet::Context const&,
std::vector > const&,
std::vector > const&,
std::vector > const&,
std::vector > const&,
std::vector > const&,
std::vector > const&,
std::vector > const&,
mxnet::DispatchMode)::{lambda(mxnet::RunContext,
mxnet::engine::CallbackOnComplete)#3}::operator()(mxnet::RunContext,
mxnet::engine::CallbackOnComplete) const+0x1333) [0x7ffa24680473]
[bt] (4)
/home/ubuntu/incubator-mxnet/python/mxnet/../../lib/libmxnet.so(std::_Function_handler > const&,
std::vector > const&,
std::vector > const&,
std::vector > const&,
std::vector > const&,
std::vector > const&,
std::vector > const&,
mxnet::DispatchMode)::{lambda(mxnet::RunContext)#4}>::_M_invoke(std::_Any_data
const&, mxnet::RunContext&&)+0x1d) [0x7ffa2468173d]
[bt] (5)
/home/ubuntu/incubator-mxnet/python/mxnet/../../lib/libmxnet.so(std::_Function_handler::_M_invoke(std::_Any_data const&,
mxnet::RunContext&&, mxnet::engine::CallbackOnComplete&&)+0x1ec)
[0x7ffa24e3f3dc]
[bt] (6)
/home/ubuntu/incubator-mxnet/python/mxnet/../../lib/libmxnet.so(mxnet::engine::ThreadedEngine::ExecuteOprBlock(mxnet::RunContext,
mxnet::engine::OprBlock*)+0x945) [0x7ffa24e42c35]
[bt] (7)
/home/ubuntu/incubator-mxnet/python/mxnet/../../lib/libmxnet.so(void
mxnet::engine::ThreadedEnginePerDevice::GPUWorker<(dmlc::ConcurrentQueueType)0>(mxnet::Context,
bool,
mxnet::engine::ThreadedEnginePerDevice::ThreadWorkerBlock<(dmlc::ConcurrentQueueType)0>*,
std::shared_ptr const&)+0x11d) [0x7ffa24e5a64d]
[bt] (8)
/home/ubuntu/incubator-mxnet/python/mxnet/../../lib/libmxnet.so(std::_Function_handler),
mxnet::engine::ThreadedEnginePerDevice::PushToExecute(mxnet::engine::OprBlock*,
bool)::{lambda()#4}::operator()()
const::{lambda(std::shared_ptr)#1}>::_M_invoke(std::_Any_data
const&, std::shared_ptr&&)+0x4e) [0x7ffa24e5a8fe]
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services