ashkanaev commented on issue #11421: Can't iterate custom dataset by dataloader with 'num_workers > 1' param on Linux URL: https://github.com/apache/incubator-mxnet/issues/11421#issuecomment-400984864 I have achieved some result after I turn off multithreading OpenMP on OpenCV like this: ``` os.environ["OMP_NUM_THREADS"] = '1' cv2.setNumThreads(0) ``` Unfortunately after 43-th batch I've got: > File "/home/user/kpavel/mxnet_27_06/python/mxnet/gluon/data/dataloader.py", line 218, in __next__ > idx, batch = self._data_queue.get() > File "/home/user/anaconda3/envs/mx/lib/python3.5/multiprocessing/queues.py", line 337, in get > return ForkingPickler.loads(res) > File "/home/user/kpavel/mxnet_27_06/python/mxnet/gluon/data/dataloader.py", line 55, in rebuild_ndarray > fd = fd.detach() > File "/home/user/anaconda3/envs/mx/lib/python3.5/multiprocessing/resource_sharer.py", line 57, in detach > with _resource_sharer.get_connection(self._id) as conn: > File "/home/user/anaconda3/envs/mx/lib/python3.5/multiprocessing/resource_sharer.py", line 87, in get_connection > c = Client(address, authkey=process.current_process().authkey) > File "/home/user/anaconda3/envs/mx/lib/python3.5/multiprocessing/connection.py", line 493, in Client > answer_challenge(c, authkey) > File "/home/user/anaconda3/envs/mx/lib/python3.5/multiprocessing/connection.py", line 732, in answer_challenge > message = connection.recv_bytes(256) # reject large message > File "/home/user/anaconda3/envs/mx/lib/python3.5/multiprocessing/connection.py", line 216, in recv_bytes > buf = self._recv_bytes(maxlength) > File "/home/user/anaconda3/envs/mx/lib/python3.5/multiprocessing/connection.py", line 407, in _recv_bytes > buf = self._recv(4) > File "/home/user/anaconda3/envs/mx/lib/python3.5/multiprocessing/connection.py", line 379, in _recv > chunk = read(handle, remaining) > ConnectionResetError: [Errno 104] Connection reset by peer
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services
