Ishitori commented on issue #11165: ImageIter much slower than ImageRecordIter URL: https://github.com/apache/incubator-mxnet/issues/11165#issuecomment-409385195 @solin319, have you tried other approaches, like creating your own custom iterator, which can do the reading in the background? I mean, if you use HDD, then using 48 threads to read random images from disk can significantly reduce the speed. Did you try to reduce the number of threads to, for example, 2, to see the difference in loading speed? Also, how much processing do you do when you read a raw image? Again, maybe you need to have a specialized pipeline: a bunch of threads who are reading files from disk, and another bunch which do processing.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services
