braindotai commented on issue #15655: Performance regression for gluon dataloader with large batch size URL: https://github.com/apache/incubator-mxnet/issues/15655#issuecomment-515695177 Data loaders are different from Data iterators. Data iterators first load all the data in to the memory before iterating it, while data loaders load data into memory as they iterate the data, hence they are slower than data iterators. __So why we use data loaders if data iterators are faster?__ Some times when we deal with massive datasets that can't be loaded into memory, so for these data sets data iterators wouldn't work. In practice we should use data iterators if our dataset can be loaded into memory (for example small datasets), otherwise we have to use data loaders. For more info regarding __Data iterators Vs Data loaders__ check [this](https://mxnet.incubator.apache.org/versions/master/architecture/note_data_loading.html) out.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services
