paul-rogers commented on a change in pull request #1978: DRILL-7578: HDF5
Metadata Queries Fail with Large Files
URL: https://github.com/apache/drill/pull/1978#discussion_r384868605
##########
File path:
contrib/format-hdf5/src/main/java/org/apache/drill/exec/store/hdf5/HDF5BatchReader.java
##########
@@ -1069,26 +1118,30 @@ private void getAndMapCompoundData(String path,
List<String> fieldNames, IHDF5Re
for (int col = 0; col < values[row].length; col++) {
assert fieldNames != null;
currentFieldName = fieldNames.get(col);
- ArrayWriter innerWriter = listWriter.array(currentFieldName);
- if (values[row][col] instanceof Integer) {
- innerWriter.scalar().setInt((Integer) values[row][col]);
- } else if (values[row][col] instanceof Short) {
- innerWriter.scalar().setInt((Short) values[row][col]);
- } else if (values[row][col] instanceof Byte) {
- innerWriter.scalar().setInt((Byte) values[row][col]);
- } else if (values[row][col] instanceof Long) {
- innerWriter.scalar().setLong((Long) values[row][col]);
- } else if (values[row][col] instanceof Float) {
- innerWriter.scalar().setDouble((Float) values[row][col]);
- } else if (values[row][col] instanceof Double) {
- innerWriter.scalar().setDouble((Double) values[row][col]);
- } else if (values[row][col] instanceof BitSet || values[row][col]
instanceof Boolean) {
- innerWriter.scalar().setBoolean((Boolean) values[row][col]);
- } else if (values[row][col] instanceof String) {
- innerWriter.scalar().setString((String) values[row][col]);
- }
- if (col == values[row].length) {
- innerWriter.save();
+ try {
+ ArrayWriter innerWriter = listWriter.array(currentFieldName);
+ if (values[row][col] instanceof Integer) {
+ innerWriter.scalar().setInt((Integer) values[row][col]);
+ } else if (values[row][col] instanceof Short) {
+ innerWriter.scalar().setInt((Short) values[row][col]);
+ } else if (values[row][col] instanceof Byte) {
+ innerWriter.scalar().setInt((Byte) values[row][col]);
+ } else if (values[row][col] instanceof Long) {
+ innerWriter.scalar().setLong((Long) values[row][col]);
+ } else if (values[row][col] instanceof Float) {
+ innerWriter.scalar().setDouble((Float) values[row][col]);
+ } else if (values[row][col] instanceof Double) {
+ innerWriter.scalar().setDouble((Double) values[row][col]);
+ } else if (values[row][col] instanceof BitSet ||
values[row][col] instanceof Boolean) {
+ innerWriter.scalar().setBoolean((Boolean) values[row][col]);
+ } else if (values[row][col] instanceof String) {
+ innerWriter.scalar().setString((String) values[row][col]);
+ }
+ if (col == values[row].length) {
+ innerWriter.save();
+ }
+ } catch (Exception e) {
+ logger.warn("Drill does not support maps and lists in HDF5
Compound fields. Skipping: {}/{}", resolvedPath, currentFieldName);
Review comment:
Are we guaranteed that the only exception is for this one use case? (No NPE,
OOM, etc.?) Or, should we catch a more specific error?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
