shenjiayu17 commented on a change in pull request #3986:
URL: https://github.com/apache/carbondata/pull/3986#discussion_r513249713
##########
File path:
processing/src/main/java/org/apache/carbondata/processing/merger/CarbonDataMergerUtil.java
##########
@@ -1138,73 +1126,43 @@ private static Boolean
checkUpdateDeltaFilesInSeg(Segment seg,
}
/**
- * Check is the segment passed qualifies for IUD delete delta compaction or
not i.e.
- * if the number of delete delta files present in the segment is more than
- * numberDeltaFilesThreshold.
+ * Check whether the segment passed qualifies for IUD delete delta
compaction or not,
+ * i.e., if the number of delete delta files present in the segment is more
than
+ * numberDeltaFilesThreshold, this segment will be selected.
*
- * @param seg
- * @param segmentUpdateStatusManager
- * @param numberDeltaFilesThreshold
- * @return
+ * @param seg segment to be qualified
+ * @param segmentUpdateStatusManager segments & blocks details management
+ * @param numberDeltaFilesThreshold threshold of delete delta files
+ * @return block list of the segment
*/
- private static boolean checkDeleteDeltaFilesInSeg(Segment seg,
+ private static List<String> checkDeleteDeltaFilesInSeg(Segment seg,
SegmentUpdateStatusManager segmentUpdateStatusManager, int
numberDeltaFilesThreshold) {
+ List<String> blockLists = new ArrayList<>();
Set<String> uniqueBlocks = new HashSet<String>();
List<String> blockNameList =
segmentUpdateStatusManager.getBlockNameFromSegment(seg.getSegmentNo());
-
- for (final String blockName : blockNameList) {
-
- CarbonFile[] deleteDeltaFiles =
+ for (String blockName : blockNameList) {
+ List<String> deleteDeltaFiles =
segmentUpdateStatusManager.getDeleteDeltaFilesList(seg, blockName);
- if (null != deleteDeltaFiles) {
+ if (null != deleteDeltaFiles && deleteDeltaFiles.size() >
numberDeltaFilesThreshold) {
Review comment:
Added test case "test IUD Horizontal Compaction when
CarbonCommonConstants.DELETE_DELTAFILE_COUNT_THRESHOLD_IUD_COMPACTION >= 3"
in HorizontalCompactionTestCase
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
