ashish-kumar-sharma commented on a change in pull request #2651:
URL: https://github.com/apache/hive/pull/2651#discussion_r712682222
##########
File path: ql/src/test/org/apache/hadoop/hive/ql/txn/compactor/TestCleaner.java
##########
@@ -604,4 +608,113 @@ public void tearDown() throws Exception {
compactorTestCleanup();
}
+ @Test
+ public void NoCleanupAfterMajorCompaction() throws Exception {
+ Map<String, String> parameters = new HashMap<>();
+
+ //With no cleanup true
+ parameters.put("no_cleanup", "true");
+ Table t = newTable("default", "dcamc", false, parameters);
+
+ addBaseFile(t, null, 20L, 20);
+ addDeltaFile(t, null, 21L, 22L, 2);
+ addDeltaFile(t, null, 23L, 24L, 2);
+ addBaseFile(t, null, 25L, 25);
+
+ burnThroughTransactions("default", "dcamc", 25);
+
+ CompactionRequest rqst = new CompactionRequest("default", "dcamc",
CompactionType.MAJOR);
+ compactInTxn(rqst);
+
+ startCleaner();
+ // Check there are no compactions requests left.
+ ShowCompactResponse rsp = txnHandler.showCompact(new ShowCompactRequest());
+ Assert.assertEquals(1, rsp.getCompactsSize());
+ Assert.assertEquals(TxnStore.SUCCEEDED_RESPONSE,
rsp.getCompacts().get(0).getState());
+
+ // Check that the files are not removed
+ List<Path> paths = getDirectories(conf, t, null);
+ Assert.assertEquals(4, paths.size());
+
+ //With no clean up false
+ t = ms.getTable(new GetTableRequest("default", "dcamc"));
+ t.getParameters().put("no_cleanup", "false");
+ ms.alter_table("default", "dcamc", t);
+ rqst = new CompactionRequest("default", "dcamc", CompactionType.MAJOR);
+ compactInTxn(rqst);
+
+ startCleaner();
+ // Check there are no compactions requests left.
+ rsp = txnHandler.showCompact(new ShowCompactRequest());
+ Assert.assertEquals(2, rsp.getCompactsSize());
+ Assert.assertEquals(TxnStore.SUCCEEDED_RESPONSE,
rsp.getCompacts().get(0).getState());
+
+ // Check that the files are not removed
+ paths = getDirectories(conf, t, null);
+ Assert.assertEquals(1, paths.size());
+ Assert.assertEquals("base_25", paths.get(0).getName());
+ }
Review comment:
@sankarh Already there is test. In this test compactor is running 2 time
first run with no_cleanup=true where the 3 delta is file is merged into base
and not deleted. In second compaction run with no_cleanup=false there was no
delta file to merged in base file but the cleaner clean the 3 delta file.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
