electriclilies opened a new pull request #7710: URL: https://github.com/apache/tvm/pull/7710
In this PR, I introduce the DataLoader class. The DataLoader is an abstract class intended to wrap datasets from other machine learning frameworks so that they can be used interchangeably within TVM for any data-aware tasks. I also provide three implementations of the DataLoader class: TFDataLoader NumpyDataLoader, and RandomDataLoader. The TFDataLoader wraps Tensorflow datasets. The NumpyDataLoader wraps numpy arrays of data in NCHW form (where N is the total number of datapoints). Keras datasets provide data in this form-- the Numpy DataLoader is intended for use with Keras datasets, but could also be used with any other dataset stored in a similar fashion. The RandomDataLoader takes in a list of shapes and produces random outputs that correspond to those shapes. This class is useful for testing code, especially if you are not at a point where you want to go to the effort of downloading a real dataset. The DataLoader class was originally designed for data-aware quantization. I think it could also be useful for training, and for making accuracy testing scripts more robust and general. @mbrookhart @jwfromm @altanh Please take a look and let me know what you think! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected]
