arina-ielchiieva commented on a change in pull request #1978: DRILL-7578: HDF5
Metadata Queries Fail with Large Files
URL: https://github.com/apache/drill/pull/1978#discussion_r383908562
##########
File path:
contrib/format-hdf5/src/main/java/org/apache/drill/exec/store/hdf5/HDF5BatchReader.java
##########
@@ -467,6 +503,12 @@ private void projectDataset(RowSetLoader rowWriter,
String datapath) {
String fieldName = HDF5Utils.getNameFromPath(datapath);
IHDF5Reader reader = hdf5Reader;
HDF5DataSetInformation dsInfo =
reader.object().getDataSetInformation(datapath);
+
+ // If the dataset is larger than 16MB, do not project the dataset
+ if (dsInfo.getSize() > 16777216) {
Review comment:
Use constant please instead of number, this would be more readable.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
