[GitHub] jwfromm commented on issue #9974: DataLoader with workers not compatible with ImageRecordDataset

2018-03-29 Thread GitBox 
jwfromm commented on issue #9974: DataLoader with workers not compatible with 
ImageRecordDataset
URL: 
https://github.com/apache/incubator-mxnet/issues/9974#issuecomment-377372332
 
 
   The bug above is due to something on the master branch, when I revert to 
V1.1, your fix works great!


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] jwfromm commented on issue #9974: DataLoader with workers not compatible with ImageRecordDataset

2018-03-28 Thread GitBox 
jwfromm commented on issue #9974: DataLoader with workers not compatible with 
ImageRecordDataset
URL: 
https://github.com/apache/incubator-mxnet/issues/9974#issuecomment-377030447
 
 
   Unfortunately, it seems like this fix doesn't always work. When attempting 
to use it on a an imagenet record, I got the following the error
   ```
   Traceback (most recent call last):
 File "/opt/conda/lib/python3.6/multiprocessing/process.py", line 258, in 
_bootstrap
   self.run()
 File "/opt/conda/lib/python3.6/multiprocessing/process.py", line 93, in run
   self._target(*self._args, **self._kwargs)
 File "", line 24, in worker_loop
   mx.gluon.data.dataloader.worker_loop_(dataset, key_queue, data_queue, 
batchify_fn)
 File "", line 24, in worker_loop
   mx.gluon.data.dataloader.worker_loop_(dataset, key_queue, data_queue, 
batchify_fn)
 File "", line 24, in worker_loop
   mx.gluon.data.dataloader.worker_loop_(dataset, key_queue, data_queue, 
batchify_fn)
 File "", line 24, in worker_loop
   mx.gluon.data.dataloader.worker_loop_(dataset, key_queue, data_queue, 
batchify_fn)
 [Previous line repeated 2937 more times]
 File "", line 23, in worker_loop
   dataset.reinit()
 File "", line 11, in reinit
   self._record = mx.recordio.MXIndexedRecordIO(idx_file, self._filename, 
'r')
 File "/incubator-mxnet/python/mxnet/recordio.py", line 199, in __init__
   super(MXIndexedRecordIO, self).__init__(uri, flag)
 File "/incubator-mxnet/python/mxnet/recordio.py", line 69, in __init__
   self.open()
 File "/incubator-mxnet/python/mxnet/recordio.py", line 205, in open
   self.fidx = open(self.idx_path, self.flag)
 File "/opt/conda/lib/python3.6/codecs.py", line 309, in __init__
   IncrementalDecoder.__init__(self, errors)
   RecursionError: maximum recursion depth exceeded
   Exception ignored in: >
   Traceback (most recent call last):
 File "/incubator-mxnet/python/mxnet/recordio.py", line 84, in __del__
   self.close()
 File "/incubator-mxnet/python/mxnet/recordio.py", line 218, in close
   self.fidx.close()
   AttributeError: 'NoneType' object has no attribute 'close'
   ```


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] jwfromm commented on issue #9974: DataLoader with workers not compatible with ImageRecordDataset

2018-03-26 Thread GitBox 
jwfromm commented on issue #9974: DataLoader with workers not compatible with 
ImageRecordDataset
URL: 
https://github.com/apache/incubator-mxnet/issues/9974#issuecomment-376351149
 
 
   If this is an acceptable fix, it'd be great to get it in the master branch 
since I'm sure other people will start hitting this error soon.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services