deniskuzZ commented on a change in pull request #2651:
URL: https://github.com/apache/hive/pull/2651#discussion_r712203180
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/txn/compactor/Cleaner.java
##########
@@ -179,6 +180,13 @@ private void clean(CompactionInfo ci, long minOpenTxnGLB,
boolean metricsEnabled
txnHandler.markCleaned(ci);
return;
}
+ if (MetaStoreUtils.isNoCleanUpSet(t.getParameters())) {
+ // The table was marked no clean up true.
+ LOG.info("Skipping " + ci.getFullTableName() + " clean up, as
NO_CLEANUP set to true");
+ txnHandler.markCleaned(ci);
Review comment:
if we won't call ````markCleaned```` that would lead to the accumulation
of COMPACTION_QUEUE entries in READY_FOR_CLEANING state (mil of duplicates) +
infinite re-try every 5 sec (default).
However, if we do - some of the obsolete files could stay forever when there
are no new writes after the user re-enables the config. I don't think this is
applicable to the use-case this JIRA is trying to address.
PS: would it be sufficient to have a config that disables the Cleaner
completely?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
