Github user manishgupta88 commented on a diff in the pull request:
https://github.com/apache/carbondata/pull/1812#discussion_r184401040
--- Diff:
processing/src/main/java/org/apache/carbondata/processing/merger/CarbonDataMergerUtil.java
---
@@ -444,6 +450,26 @@ public int compare(LoadMetadataDetails seg1,
LoadMetadataDetails seg2) {
});
}
+ /**
+ * This method will return the list of loads which are specified by user
in SQL.
+ *
+ * @param listOfSegments
+ * @param segmentIds
+ * @return
+ */
+ private static List<LoadMetadataDetails>
identitySegmentsToBeMergedBasedOnSpecifiedSegments(
+ List<LoadMetadataDetails> listOfSegments,
+ Set<String> segmentIds) {
+ List<LoadMetadataDetails> listOfSegmentsSpecified =
+ new ArrayList<>(CarbonCommonConstants.DEFAULT_COLLECTION_SIZE);
+ for (LoadMetadataDetails detail : listOfSegments) {
+ if (isSegmentValid(detail) &&
segmentIds.contains(detail.getLoadName())) {
+ listOfSegmentsSpecified.add(detail);
+ }
+ }
+ return listOfSegmentsSpecified;
--- End diff --
In case of custom compaction user is completely aware of the segments
provided in the Alter SQL. Therefore in the segment List provided by the user
if any segment is found invalid we should throw exception and compaction
process should be aborted
---
