zhreshold commented on a change in pull request #12131: [MXNET-737][WIP] Add
last batch handle for imageiter
URL: https://github.com/apache/incubator-mxnet/pull/12131#discussion_r209706537
##########
File path: python/mxnet/image/image.py
##########
@@ -1207,8 +1244,31 @@ def next(self):
except StopIteration:
if not i:
raise StopIteration
+ return i
- return io.DataBatch([batch_data], [batch_label], batch_size - i)
+ def next(self):
+ """Returns the next batch of data."""
+ batch_size = self.batch_size
+ c, h, w = self.data_shape
+ batch_data = nd.empty((batch_size, c, h, w))
+ batch_label = nd.empty(self.provide_label[0][1])
+ index = 0
+ i = self.iterate(batch_data, batch_label)
+
+ # handle padding for sequential read
+ if self.seq is None:
+ pad = batch_size - i
+ # pad the last batch by creating a new MXRecordIO
+ self._imgrec = recordio.MXRecordIO(self.path_imgrec, 'r')
+ if pad != 0:
+ _ = self.iterate(batch_data, batch_label, i, self._imgrec)
+
+ if self.last_batch_handle != 'roll_over':
Review comment:
is it possible to cache last batch in memory if roll_over? I think
open/closing recordio is problematic
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
