Github user ravipesala commented on a diff in the pull request:
https://github.com/apache/carbondata/pull/3029#discussion_r244319031
--- Diff:
processing/src/main/java/org/apache/carbondata/processing/merger/CarbonCompactionExecutor.java
---
@@ -126,17 +128,24 @@ public CarbonCompactionExecutor(Map<String,
TaskBlockInfo> segmentMapping,
// for each segment get taskblock info
TaskBlockInfo taskBlockInfo = taskMap.getValue();
Set<String> taskBlockListMapping = taskBlockInfo.getTaskSet();
+ // Check if block needs sorting or not
+ boolean sortingRequired =
+ CarbonCompactionUtil.isRestructured(listMetadata,
carbonTable.getTableLastUpdatedTime())
+ || !CarbonCompactionUtil.isSorted(taskBlockInfo);
--- End diff --
Here we are reading each and every carbondata file footer, it will impact
the compaction performance. I feel we should discuss and consider adding isSort
flag also to the carbonindex file to simplify it
---
