[
https://issues.apache.org/jira/browse/HIVE-27020?focusedWorklogId=857253&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-857253
]
ASF GitHub Bot logged work on HIVE-27020:
-----------------------------------------
Author: ASF GitHub Bot
Created on: 16/Apr/23 18:13
Start Date: 16/Apr/23 18:13
Worklog Time Spent: 10m
Work Description: SourabhBadhya commented on code in PR #4091:
URL: https://github.com/apache/hive/pull/4091#discussion_r1167991491
##########
standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/txn/TxnUtils.java:
##########
@@ -60,20 +60,20 @@
public class TxnUtils {
private static final Logger LOG = LoggerFactory.getLogger(TxnUtils.class);
- public static ValidTxnList createValidTxnListForCleaner(GetOpenTxnsResponse
txns, long minOpenTxnGLB) {
- long highWaterMark = minOpenTxnGLB - 1;
+ public static ValidTxnList
createValidTxnListForCompactionCleaner(GetOpenTxnsResponse txns, long
minOpenTxn) {
+ long highWatermark = minOpenTxn - 1;
long[] abortedTxns = new long[txns.getOpen_txnsSize()];
BitSet abortedBits = BitSet.valueOf(txns.getAbortedBits());
int i = 0;
for(long txnId : txns.getOpen_txns()) {
- if(txnId > highWaterMark) {
+ if(txnId > highWatermark) {
Review Comment:
Done.
##########
standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/txn/TxnUtils.java:
##########
@@ -60,20 +60,20 @@
public class TxnUtils {
private static final Logger LOG = LoggerFactory.getLogger(TxnUtils.class);
- public static ValidTxnList createValidTxnListForCleaner(GetOpenTxnsResponse
txns, long minOpenTxnGLB) {
- long highWaterMark = minOpenTxnGLB - 1;
+ public static ValidTxnList
createValidTxnListForCompactionCleaner(GetOpenTxnsResponse txns, long
minOpenTxn) {
+ long highWatermark = minOpenTxn - 1;
long[] abortedTxns = new long[txns.getOpen_txnsSize()];
BitSet abortedBits = BitSet.valueOf(txns.getAbortedBits());
int i = 0;
for(long txnId : txns.getOpen_txns()) {
- if(txnId > highWaterMark) {
+ if(txnId > highWatermark) {
break;
}
if(abortedBits.get(i)) {
Review Comment:
Done
##########
standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/txn/TxnUtils.java:
##########
@@ -82,7 +82,29 @@ public static ValidTxnList
createValidTxnListForCleaner(GetOpenTxnsResponse txns
bitSet.set(0, abortedTxns.length);
//add ValidCleanerTxnList? - could be problematic for all the places that
read it from
// string as they'd have to know which object to instantiate
- return new ValidReadTxnList(abortedTxns, bitSet, highWaterMark,
Long.MAX_VALUE);
+ return new ValidReadTxnList(abortedTxns, bitSet, highWatermark,
Long.MAX_VALUE);
+ }
+
+ public static ValidTxnList
createValidTxnListForAbortedTxnCleaner(GetOpenTxnsResponse txns, long
minOpenTxn) {
+ long highWatermark = minOpenTxn - 1;
+ long[] exceptions = new long[txns.getOpen_txnsSize()];
+ int i = 0;
+ BitSet abortedBits = BitSet.valueOf(txns.getAbortedBits());
+ // getOpen_txns() guarantees that the list contains only aborted & open
txns.
+ // exceptions list must contain both txn types since validWriteIdList
filters out the aborted ones and valid ones for that table.
+ // If a txn is not in exception list, it is considered as a valid one and
thought of as an uncompacted write.
+ // See TxnHandler#getValidWriteIdsForTable() for more details.
+ for(long txnId : txns.getOpen_txns()) {
Review Comment:
Done
Issue Time Tracking
-------------------
Worklog Id: (was: 857253)
Time Spent: 13h (was: 12h 50m)
> Implement a separate handler to handle aborted transaction cleanup
> ------------------------------------------------------------------
>
> Key: HIVE-27020
> URL: https://issues.apache.org/jira/browse/HIVE-27020
> Project: Hive
> Issue Type: Sub-task
> Reporter: Sourabh Badhya
> Assignee: Sourabh Badhya
> Priority: Major
> Labels: pull-request-available
> Time Spent: 13h
> Remaining Estimate: 0h
>
> As described in the parent task, once the cleaner is separated into different
> entities, implement a separate handler which can create requests for aborted
> transactions cleanup. This would move the aborted transaction cleanup
> exclusively to the cleaner.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
