Github user ravipesala commented on a diff in the pull request:
https://github.com/apache/carbondata/pull/2565#discussion_r206008805
--- Diff:
hadoop/src/main/java/org/apache/carbondata/hadoop/api/CarbonInputFormat.java ---
@@ -492,6 +493,26 @@ protected Expression getFilterPredicates(Configuration
configuration) {
return prunedBlocklets;
}
+ private List<ExtendedBlocklet> intersectFilteredBlocklets(CarbonTable
carbonTable,
+ List<ExtendedBlocklet> previousDataMapPrunedBlocklets,
+ List<ExtendedBlocklet> otherDataMapPrunedBlocklets) {
+ List<ExtendedBlocklet> prunedBlocklets = null;
+ if (BlockletDataMapUtil
+ .isCacheLevelBlock(carbonTable,
BlockletDataMapFactory.CACHE_LEVEL_BLOCKLET)) {
+ prunedBlocklets = new ArrayList<>(otherDataMapPrunedBlocklets);
+ // add blocklets from previous dataMap that are not filtered by
other dataMaps
+ for (ExtendedBlocklet previousBlocklet :
previousDataMapPrunedBlocklets) {
+ if (!otherDataMapPrunedBlocklets.contains(previousBlocklet)) {
+ prunedBlocklets.add(previousBlocklet);
--- End diff --
It supposed to be same as with blocklet level. why adding extra non pruned
blocklets should be added to the list? Any use case for this check?
---
