Github user ajantha-bhat commented on a diff in the pull request:
https://github.com/apache/carbondata/pull/2869#discussion_r229246953
--- Diff:
hadoop/src/main/java/org/apache/carbondata/hadoop/api/CarbonFileInputFormat.java
---
@@ -145,9 +158,33 @@ public CarbonTable
getOrCreateCarbonTable(Configuration configuration) throws IO
externalTableSegments.add(seg);
}
}
- // do block filtering and get split
- List<InputSplit> splits =
- getSplits(job, filter, externalTableSegments, null,
partitionInfo, null);
+ List<InputSplit> splits = new ArrayList<>();
+ if (isSDK) {
--- End diff --
This is SDK logic, not fileInputFormat logic.
you can do this in CarbonReaderBuilder.build() --> here already
getSplits() is called, you can check if filters are there, call
FileInputFormat.getSplits() else call a method getAllFileSplit() which gives
all the files without blocklet loading
Also we don't need isSDK flag also
---
