[
https://issues.apache.org/jira/browse/HIVE-27020?focusedWorklogId=855762&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-855762
]
ASF GitHub Bot logged work on HIVE-27020:
-----------------------------------------
Author: ASF GitHub Bot
Created on: 10/Apr/23 10:58
Start Date: 10/Apr/23 10:58
Worklog Time Spent: 10m
Work Description: deniskuzZ commented on code in PR #4091:
URL: https://github.com/apache/hive/pull/4091#discussion_r1161633548
##########
standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/txn/CompactionTxnHandler.java:
##########
@@ -162,31 +162,33 @@ public Set<CompactionInfo> findPotentialCompactions(int
abortedThreshold,
}
rs.close();
- // Check for aborted txns: number of aborted txns past threshold and
age of aborted txns
- // past time threshold
- boolean checkAbortedTimeThreshold = abortedTimeThreshold >= 0;
- String sCheckAborted = "SELECT \"TC_DATABASE\", \"TC_TABLE\",
\"TC_PARTITION\", " +
- "MIN(\"TXN_STARTED\"), COUNT(*) FROM \"TXNS\", \"TXN_COMPONENTS\" " +
- " WHERE \"TXN_ID\" = \"TC_TXNID\" AND \"TXN_STATE\" = " +
TxnStatus.ABORTED + " " +
- "GROUP BY \"TC_DATABASE\", \"TC_TABLE\", \"TC_PARTITION\" " +
- (checkAbortedTimeThreshold ? "" : " HAVING COUNT(*) > " +
abortedThreshold);
-
- LOG.debug("Going to execute query <{}>", sCheckAborted);
- rs = stmt.executeQuery(sCheckAborted);
- long systemTime = System.currentTimeMillis();
- while (rs.next()) {
- boolean pastTimeThreshold =
- checkAbortedTimeThreshold && rs.getLong(4) +
abortedTimeThreshold < systemTime;
- int numAbortedTxns = rs.getInt(5);
- if (numAbortedTxns > abortedThreshold || pastTimeThreshold) {
- CompactionInfo info = new CompactionInfo();
- info.dbname = rs.getString(1);
- info.tableName = rs.getString(2);
- info.partName = rs.getString(3);
- info.tooManyAborts = numAbortedTxns > abortedThreshold;
- info.hasOldAbort = pastTimeThreshold;
- LOG.debug("Found potential compaction: {}", info);
- response.add(info);
+ if (!MetastoreConf.getBoolVar(conf,
ConfVars.COMPACTOR_CLEAN_ABORTS_USING_CLEANER)) {
Review Comment:
no need for that, leads to code duplication
##########
standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/txn/CompactionTxnHandler.java:
##########
@@ -464,6 +466,54 @@ public List<CompactionInfo> findReadyToClean(long
minOpenTxnWaterMark, long rete
}
}
+ @Override
+ @RetrySemantics.ReadOnly
+ public List<AcidTxnInfo> findReadyToCleanForAborts(long
abortedTimeThreshold, int abortedThreshold) throws MetaException {
Review Comment:
rename `findReadyToCleanAborts`
Issue Time Tracking
-------------------
Worklog Id: (was: 855762)
Time Spent: 6h 50m (was: 6h 40m)
> Implement a separate handler to handle aborted transaction cleanup
> ------------------------------------------------------------------
>
> Key: HIVE-27020
> URL: https://issues.apache.org/jira/browse/HIVE-27020
> Project: Hive
> Issue Type: Sub-task
> Reporter: Sourabh Badhya
> Assignee: Sourabh Badhya
> Priority: Major
> Labels: pull-request-available
> Time Spent: 6h 50m
> Remaining Estimate: 0h
>
> As described in the parent task, once the cleaner is separated into different
> entities, implement a separate handler which can create requests for aborted
> transactions cleanup. This would move the aborted transaction cleanup
> exclusively to the cleaner.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
