leezu opened a new issue #18382:
URL: https://github.com/apache/incubator-mxnet/issues/18382
```
[2020-05-22T01:08:46.081Z] _________________________________ test_CSVIter
_________________________________
[2020-05-22T01:08:46.081Z] [gw0] linux -- Python 3.6.9 /usr/bin/python3
[2020-05-22T01:08:46.081Z]
[2020-05-22T01:08:46.081Z] def test_CSVIter():
[2020-05-22T01:08:46.081Z] def
check_CSVIter_synthetic(dtype='float32'):
[2020-05-22T01:08:46.081Z] cwd = os.getcwd()
[2020-05-22T01:08:46.081Z] data_path = os.path.join(cwd,
'data.t')
[2020-05-22T01:08:46.081Z] label_path = os.path.join(cwd,
'label.t')
[2020-05-22T01:08:46.081Z] entry_str = '1'
[2020-05-22T01:08:46.081Z] if dtype is 'int32':
[2020-05-22T01:08:46.081Z] entry_str = '200000001'
[2020-05-22T01:08:46.081Z] if dtype is 'int64':
[2020-05-22T01:08:46.081Z] entry_str = '2147483648'
[2020-05-22T01:08:46.081Z] with open(data_path, 'w') as fout:
[2020-05-22T01:08:46.081Z] for i in range(1000):
[2020-05-22T01:08:46.081Z]
fout.write(','.join([entry_str for _ in range(8*8)]) + '\n')
[2020-05-22T01:08:46.081Z] with open(label_path, 'w') as fout:
[2020-05-22T01:08:46.081Z] for i in range(1000):
[2020-05-22T01:08:46.081Z] fout.write('0\n')
[2020-05-22T01:08:46.081Z]
[2020-05-22T01:08:46.081Z] data_train =
mx.io.CSVIter(data_csv=data_path, data_shape=(8, 8),
[2020-05-22T01:08:46.081Z]
label_csv=label_path, batch_size=100, dtype=dtype)
[2020-05-22T01:08:46.081Z] expected = mx.nd.ones((100, 8, 8),
dtype=dtype) * int(entry_str)
[2020-05-22T01:08:46.081Z] for batch in iter(data_train):
[2020-05-22T01:08:46.081Z] data_batch = data_train.getdata()
[2020-05-22T01:08:46.081Z]
assert_almost_equal(data_batch.asnumpy(), expected.asnumpy())
[2020-05-22T01:08:46.081Z] assert data_batch.asnumpy().dtype
== expected.asnumpy().dtype
[2020-05-22T01:08:46.081Z]
[2020-05-22T01:08:46.081Z] for dtype in ['int32', 'int64',
'float32']:
[2020-05-22T01:08:46.081Z] > check_CSVIter_synthetic(dtype=dtype)
[2020-05-22T01:08:46.081Z]
[2020-05-22T01:08:46.081Z] tests/python/unittest/test_io.py:455:
[2020-05-22T01:08:46.081Z] _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _
[2020-05-22T01:08:46.081Z] tests/python/unittest/test_io.py:447: in
check_CSVIter_synthetic
[2020-05-22T01:08:46.081Z] label_csv=label_path, batch_size=100,
dtype=dtype)
[2020-05-22T01:08:46.081Z] python/mxnet/io/io.py:1004: in creator
[2020-05-22T01:08:46.081Z] return MXDataIter(iter_handle, **kwargs)
[2020-05-22T01:08:46.081Z] python/mxnet/io/io.py:837: in __init__
[2020-05-22T01:08:46.081Z] self.first_batch = self.next()
[2020-05-22T01:08:46.081Z] python/mxnet/io/io.py:871: in next
[2020-05-22T01:08:46.081Z] check_call(_LIB.MXDataIterNext(self.handle,
ctypes.byref(next_res)))
[2020-05-22T01:08:46.081Z] _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _
[2020-05-22T01:08:46.081Z]
[2020-05-22T01:08:46.081Z] ret = -1
[2020-05-22T01:08:46.081Z]
[2020-05-22T01:08:46.081Z] def check_call(ret):
[2020-05-22T01:08:46.081Z] """Check the return value of C API call.
[2020-05-22T01:08:46.081Z]
[2020-05-22T01:08:46.081Z] This function will raise an exception
when an error occurs.
[2020-05-22T01:08:46.081Z] Wrap every API call with this function.
[2020-05-22T01:08:46.081Z]
[2020-05-22T01:08:46.081Z] Parameters
[2020-05-22T01:08:46.081Z] ----------
[2020-05-22T01:08:46.081Z] ret : int
[2020-05-22T01:08:46.081Z] return value from API calls.
[2020-05-22T01:08:46.081Z] """
[2020-05-22T01:08:46.081Z] if ret != 0:
[2020-05-22T01:08:46.081Z] > raise get_last_ffi_error()
[2020-05-22T01:08:46.081Z] E mxnet.base.MXNetError: Traceback
(most recent call last):
[2020-05-22T01:08:46.081Z] E [bt] (4)
/work/mxnet/python/mxnet/../../build/libmxnet.so(MXDataIterNext+0x31)
[0x7f36a8936ca1]
[2020-05-22T01:08:46.081Z] E [bt] (3)
/work/mxnet/python/mxnet/../../build/libmxnet.so(mxnet::io::PrefetcherIter::Next()+0x186)
[0x7f36a8c7c896]
[2020-05-22T01:08:46.081Z] E [bt] (2)
/work/mxnet/python/mxnet/../../build/libmxnet.so(dmlc::ThreadedIter<mxnet::DataBatch>::Next(mxnet::DataBatch**)+0x2d6)
[0x7f36a8c777b6]
[2020-05-22T01:08:46.081Z] E [bt] (1)
/work/mxnet/python/mxnet/../../build/libmxnet.so(dmlc::ThreadedIter<mxnet::DataBatch>::ThrowExceptionIfSet()+0x23f)
[0x7f36a8c7746f]
[2020-05-22T01:08:46.081Z] E [bt] (0)
/work/mxnet/python/mxnet/../../build/libmxnet.so(dmlc::LogMessageFatal::~LogMessageFatal()+0x75)
[0x7f36a880c785]
[2020-05-22T01:08:46.081Z] E [bt] (9)
/lib/x86_64-linux-gnu/libpthread.so.0(+0x76db) [0x7f375d13a6db]
[2020-05-22T01:08:46.081Z] E [bt] (8)
/usr/lib/x86_64-linux-gnu/libstdc++.so.6(+0xbd6df) [0x7f374e4b86df]
[2020-05-22T01:08:46.081Z] E [bt] (7)
/work/mxnet/python/mxnet/../../build/libmxnet.so(std::thread::_State_impl<std::thread::_Invoker<std::tuple<dmlc::ThreadedIter<mxnet::DataBatch>::Init(std::function<bool
(mxnet::DataBatch**)>, std::function<void ()>)::{lambda()#1}> >
>::_M_run()+0x15) [0x7f36a8c7c6f5]
[2020-05-22T01:08:46.081Z] E [bt] (6)
/work/mxnet/python/mxnet/../../build/libmxnet.so(dmlc::ThreadedIter<mxnet::DataBatch>::Init(std::function<bool
(mxnet::DataBatch**)>, std::function<void ()>)::{lambda()#1}::operator()()
const+0x1c6) [0x7f36a8c7b906]
[2020-05-22T01:08:46.081Z] E [bt] (5)
/work/mxnet/python/mxnet/../../build/libmxnet.so(std::_Function_handler<bool
(mxnet::DataBatch**),
mxnet::io::PrefetcherIter::Init(std::vector<std::pair<std::__cxx11::basic_string<char,
std::char_traits<char>, std::allocator<char> >,
std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >
>, std::allocator<std::pair<std::__cxx11::basic_string<char,
std::char_traits<char>, std::allocator<char> >,
std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >
> > > const&)::{lambda(mxnet::DataBatch**)#1}>::_M_invoke(std::_Any_data
const&, mxnet::DataBatch**&&)+0x14) [0x7f36a8c7ac24]
[2020-05-22T01:08:46.081Z] E [bt] (4)
/work/mxnet/python/mxnet/../../build/libmxnet.so(mxnet::io::PrefetcherIter::Init(std::vector<std::pair<std::__cxx11::basic_string<char,
std::char_traits<char>, std::allocator<char> >,
std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >
>, std::allocator<std::pair<std::__cxx11::basic_string<char,
std::char_traits<char>, std::allocator<char> >,
std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >
> > > const&)::{lambda(mxnet::DataBatch**)#1}::operator()(mxnet::DataBatch**)
const+0x42) [0x7f36a8c792b2]
[2020-05-22T01:08:46.081Z] E [bt] (3)
/work/mxnet/python/mxnet/../../build/libmxnet.so(mxnet::io::BatchLoader::Next()+0x6b)
[0x7f36a8ceff1b]
[2020-05-22T01:08:46.081Z] E [bt] (2)
/work/mxnet/python/mxnet/../../build/libmxnet.so(mxnet::io::CSVIter::Next()+0x19)
[0x7f36a8ce7329]
[2020-05-22T01:08:46.081Z] E [bt] (1)
/work/mxnet/python/mxnet/../../build/libmxnet.so(mxnet::io::CSVIterTyped<int>::Next()+0x333)
[0x7f36a8cf3b53]
[2020-05-22T01:08:46.081Z] E [bt] (0)
/work/mxnet/python/mxnet/../../build/libmxnet.so(dmlc::LogMessageFatal::~LogMessageFatal()+0x75)
[0x7f36a880c785]
[2020-05-22T01:08:46.081Z] E File
"/work/mxnet/include/dmlc/threadediter.h", line 500
[2020-05-22T01:08:46.081Z] E MXNetError: [00:55:19]
/work/mxnet/src/io/iter_csv.cc:137: Check failed: label_parser_->Next(): Data
CSV's row is smaller than the number of rows in label_csv
```
http://jenkins.mxnet-ci.amazon-ml.com/blue/rest/organizations/jenkins/pipelines/mxnet-validation/pipelines/unix-cpu/branches/PR-18376/runs/8/nodes/363/steps/746/log/?start=0
cc: @zhreshold
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
