ThomasDelteil commented on issue #13447: Rewrite dataloader, improves responsiveness and reliability URL: https://github.com/apache/incubator-mxnet/pull/13447#issuecomment-445433076 Yes I think the problem with multi-threading is to make sure the user understand that get a real parallel processing boost, it needs to be done in the underlying libraries like opencv to bypass the GIL. I think there's a case for it but it shouldn't be the default and has appropriate warning in the documentation. I believe different pickling protocols can be chosen to trade off compute for memory @leezu ? Maybe we could also expose that to the users
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services
