keith-turner commented on code in PR #5395:
URL: https://github.com/apache/accumulo/pull/5395#discussion_r1990208020
##########
test/src/main/java/org/apache/accumulo/test/functional/CompactionIT.java:
##########
@@ -797,6 +804,108 @@ public void testGetActiveCompactions() throws Exception {
}
}
+ @Test
+ public void testMigrationCancelCompaction() throws Exception {
+
+ // This test creates 40 tablets w/ slow iterator, causes 40 compactions to
start, and then
+ // starts a new tablet server. Some of the tablets should migrate to the
new tserver and cancel
+ // their compaction. Because the test uses a slow iterator, if close
blocks on compaction then
+ // the test should timeout. Two tables are used to have different iterator
settings inorder to
+ // test the two different way compactions can be canceled. Compactions can
be canceled by thread
+ // interrupt or by a check that is done after a compaction iterator
returns a key value.
+
+ final String[] tables = this.getUniqueNames(2);
+ try (AccumuloClient client =
Accumulo.newClient().from(getClientProps()).build()) {
+ client.instanceOperations().setProperty(
+ Property.TSERV_COMPACTION_SERVICE_DEFAULT_EXECUTORS.getKey(),
+ "[{'name':'any','numThreads':20}]".replaceAll("'", "\""));
+
+ SortedSet<Text> splits = IntStream.range(1, 20).mapToObj(i ->
String.format("%06d", i * 1000))
+ .map(Text::new).collect(Collectors.toCollection(TreeSet::new));
+
+ // This iterator is intended to cover the case of a compaction being
canceled by thread
+ // interrupt.
+ IteratorSetting setting1 = new IteratorSetting(50, "sleepy",
SlowIterator.class);
+ setting1.addOption("sleepTime", "300000");
+ setting1.addOption("seekSleepTime", "3000");
+ SlowIterator.sleepUninterruptibly(setting1, false);
+
+ client.tableOperations().create(tables[0], new
NewTableConfiguration().withSplits(splits)
+ .attachIterator(setting1, EnumSet.of(IteratorScope.majc)));
+
+ // This iterator is intended to cover the case of compaction being
canceled by the check after
+ // a key value is returned. The iterator is configured to ignore
interrupts.
+ IteratorSetting setting2 = new IteratorSetting(50, "sleepy",
SlowIterator.class);
+ setting2.addOption("sleepTime", "2000");
+ setting2.addOption("seekSleepTime", "2000");
+ SlowIterator.sleepUninterruptibly(setting2, true);
+
+ client.tableOperations().create(tables[1], new
NewTableConfiguration().withSplits(splits)
+ .attachIterator(setting2, EnumSet.of(IteratorScope.majc)));
+
+ // write files to each tablet, should cause compactions to start
+ for (var table : tables) {
+ for (int round = 0; round < 5; round++) {
+ try (var writer = client.createBatchWriter(table)) {
+ for (int i = 0; i < 20_000; i++) {
+ Mutation m = new Mutation(String.format("%06d", i));
+ m.put("f", "q", "v");
+ writer.addMutation(m);
+ }
+ }
+ client.tableOperations().flush(table, null, null, true);
+ }
+ }
+
+ assertEquals(2, client.instanceOperations().getTabletServers().size());
+
+ var ctx = (ClientContext) client;
+ var tableId1 = ctx.getTableId(tables[0]);
+ var tableId2 = ctx.getTableId(tables[1]);
+
+ Wait.waitFor(() -> {
+ var runningCompactions =
client.instanceOperations().getActiveCompactions().stream()
+ .map(ac -> ac.getTablet().getTable())
+ .filter(tid -> tid.equals(tableId1) ||
tid.equals(tableId2)).count();
+ log.debug("Running compactions {}", runningCompactions);
+ return runningCompactions == 40;
Review Comment:
This test usually passes for me, every once in a while it will get stuck
here w/ less than the expected 40 compactions. Probably some issue w/ the
propagation of compaction config changes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscr...@accumulo.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
