kuczoram commented on code in PR #6498:
URL: https://github.com/apache/hive/pull/6498#discussion_r3309592893
##########
ql/src/java/org/apache/hadoop/hive/ql/txn/compactor/handler/CompactionCleaner.java:
##########
@@ -99,6 +101,18 @@ private void clean(CompactionInfo ci, long minOpenTxn,
boolean metricsEnabled) t
LOG.info("Starting cleaning for {}, based on min open {}", ci,
(ci.minOpenWriteId > 0) ? "writeId: " + ci.minOpenWriteId : "txnId: "
+ minOpenTxn);
+ if (ci.nextTxnId == 0 && ci.txnId > 0 &&
+ (ci.type == CompactionType.MAJOR || ci.type == CompactionType.MINOR ||
ci.type == CompactionType.REBALANCE)) {
+ TxnStatus status = txnHandler.getTransactionStatus(ci.txnId);
Review Comment:
It checks for the transaction state only for compactions which don't have
nextTxnId. If I understand it correctly if a compaction's txn is committed
successfully, the nextTxnId is filled. So it will mean just one additional HMS
call for compactions without successfully committed txn. For compaction's with
comitted txn, it will do nothing.
I tried an other approach as well, when I made the compactions failed in the
AbortTxnsFunction, but I didn't really like that either. I didn't find a way to
call the markFailed from there, so I either had to extend the SQLs there or
fetch the compactioninfo for all txn ids and then check if the condition's are
matched. Like it has to be a compaction, not a soft-delete and it has to be in
ready-for-cleaning state, because if it is in working state, we cannot fail it,
as it could be revoked. At the end I felt that going with this approach could
have more side effects. So I went with this one, but I can go back to making
the failure in AbortTxnsFunction if you'd like that.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
