maytasm commented on a change in pull request #10900:
URL: https://github.com/apache/druid/pull/10900#discussion_r585837666
##########
File path:
integration-tests/src/test/java/org/apache/druid/tests/indexer/ITCompactionTaskTest.java
##########
@@ -87,11 +92,68 @@ public void testCompactionWithSegmentGranularity() throws
Exception
}
@Test
- public void testCompactionWithGranularitySpec() throws Exception
+ public void testCompactionWithSegmentGranularityInGranularitySpec() throws
Exception
{
loadDataAndCompact(INDEX_TASK, INDEX_QUERIES_RESOURCE,
COMPACTION_TASK_WITH_GRANULARITY_SPEC, GranularityType.MONTH);
}
+ @Test
+ public void testCompactionWithQueryGranularityInGranularitySpec() throws
Exception
+ {
+ try (final Closeable ignored = unloader(fullDatasourceName)) {
+ loadData(INDEX_TASK);
+ // 4 segments across 2 days
+ checkNumberOfSegments(4);
+ List<String> expectedIntervalAfterCompaction =
coordinator.getSegmentIntervals(fullDatasourceName);
+ expectedIntervalAfterCompaction.sort(null);
+
+ checkQueryGranularity(SEGMENT_METADATA_QUERY_RESOURCE,
GranularityType.SECOND.name(), 4);
+ String queryResponseTemplate =
getQueryResponseTemplate(INDEX_QUERIES_RESOURCE);
+ queryHelper.testQueriesFromString(queryResponseTemplate);
+ // QueryGranularity was SECOND, now we will change it to HOUR
+ compactData(COMPACTION_TASK_WITH_GRANULARITY_SPEC, null,
GranularityType.HOUR);
+
+ // The original 4 segments should be compacted into 2 new segments
+ checkNumberOfSegments(2);
+ queryResponseTemplate =
getQueryResponseTemplate(INDEX_QUERIES_HOUR_RESOURCE);
+ queryHelper.testQueriesFromString(queryResponseTemplate);
+ checkQueryGranularity(SEGMENT_METADATA_QUERY_RESOURCE,
GranularityType.HOUR.name(), 2);
+ checkCompactionIntervals(expectedIntervalAfterCompaction);
+ }
+ }
+
+ @Test
+ public void
testCompactionWithSegmentGranularityAndQueryGranularityInGranularitySpec()
throws Exception
+ {
+ try (final Closeable ignored = unloader(fullDatasourceName)) {
+ loadData(INDEX_TASK);
+ // 4 segments across 2 days
+ checkNumberOfSegments(4);
+ List<String> expectedIntervalAfterCompaction =
coordinator.getSegmentIntervals(fullDatasourceName);
+ expectedIntervalAfterCompaction.sort(null);
+
+ checkQueryGranularity(SEGMENT_METADATA_QUERY_RESOURCE,
GranularityType.SECOND.name(), 4);
+ String queryResponseTemplate =
getQueryResponseTemplate(INDEX_QUERIES_RESOURCE);
+ queryHelper.testQueriesFromString(queryResponseTemplate);
+ compactData(COMPACTION_TASK_WITH_GRANULARITY_SPEC, GranularityType.YEAR,
GranularityType.YEAR);
+
+ // The original 4 segments should be compacted into 1 new segment
+ checkNumberOfSegments(1);
+ queryResponseTemplate =
getQueryResponseTemplate(INDEX_QUERIES_YEAR_RESOURCE);
+ queryHelper.testQueriesFromString(queryResponseTemplate);
+ checkQueryGranularity(SEGMENT_METADATA_QUERY_RESOURCE,
GranularityType.YEAR.name(), 1);
+
+ List<String> newIntervals = new ArrayList<>();
+ for (String interval : expectedIntervalAfterCompaction) {
+ for (Interval newinterval :
GranularityType.YEAR.getDefaultGranularity().getIterable(new Interval(interval,
ISOChronology.getInstanceUTC()))) {
+ newIntervals.add(newinterval.toString());
+ }
+ }
+ expectedIntervalAfterCompaction = newIntervals;
+ checkCompactionIntervals(expectedIntervalAfterCompaction);
+ }
+ }
+
Review comment:
That functionality is not changed in this PR. It will be the same as
doing a reindex task with larger query granularity to a smaller query
granularity. Compaction task will reports the result of the index task (which
would be the same as doing a reindex task)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
