mfiore commented on issue #13945: dataloader crashes with threads and slow downs with processes URL: https://github.com/apache/incubator-mxnet/issues/13945#issuecomment-459270811 @zhreshold thanks for your answer! The dataset is about 178k examples so with a batch size of 32 it's more than 5k batches. Putting the mutex in the line you pointed solves the crash problem. Now the behaviors with threads or multiprocess is the same, with slow downs every few batches.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services
