jwfromm commented on issue #9974: DataLoader with workers not compatible with ImageRecordDataset URL: https://github.com/apache/incubator-mxnet/issues/9974#issuecomment-377030447 Unfortunately, it seems like this fix doesn't always work. When attempting to use it on a an imagenet record, I got the following the error ``` Traceback (most recent call last): File "/opt/conda/lib/python3.6/multiprocessing/process.py", line 258, in _bootstrap self.run() File "/opt/conda/lib/python3.6/multiprocessing/process.py", line 93, in run self._target(*self._args, **self._kwargs) File "<ipython-input-13-e05cb056ee0e>", line 24, in worker_loop mx.gluon.data.dataloader.worker_loop_(dataset, key_queue, data_queue, batchify_fn) File "<ipython-input-9-e05cb056ee0e>", line 24, in worker_loop mx.gluon.data.dataloader.worker_loop_(dataset, key_queue, data_queue, batchify_fn) File "<ipython-input-9-e05cb056ee0e>", line 24, in worker_loop mx.gluon.data.dataloader.worker_loop_(dataset, key_queue, data_queue, batchify_fn) File "<ipython-input-9-e05cb056ee0e>", line 24, in worker_loop mx.gluon.data.dataloader.worker_loop_(dataset, key_queue, data_queue, batchify_fn) [Previous line repeated 2937 more times] File "<ipython-input-9-e05cb056ee0e>", line 23, in worker_loop dataset.reinit() File "<ipython-input-13-e05cb056ee0e>", line 11, in reinit self._record = mx.recordio.MXIndexedRecordIO(idx_file, self._filename, 'r') File "/incubator-mxnet/python/mxnet/recordio.py", line 199, in __init__ super(MXIndexedRecordIO, self).__init__(uri, flag) File "/incubator-mxnet/python/mxnet/recordio.py", line 69, in __init__ self.open() File "/incubator-mxnet/python/mxnet/recordio.py", line 205, in open self.fidx = open(self.idx_path, self.flag) File "/opt/conda/lib/python3.6/codecs.py", line 309, in __init__ IncrementalDecoder.__init__(self, errors) RecursionError: maximum recursion depth exceeded Exception ignored in: <bound method MXRecordIO.__del__ of <mxnet.recordio.MXIndexedRecordIO object at 0x7f0489973588>> Traceback (most recent call last): File "/incubator-mxnet/python/mxnet/recordio.py", line 84, in __del__ self.close() File "/incubator-mxnet/python/mxnet/recordio.py", line 218, in close self.fidx.close() AttributeError: 'NoneType' object has no attribute 'close' ```
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
