[
https://issues.apache.org/jira/browse/HIVE-27020?focusedWorklogId=857252&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-857252
]
ASF GitHub Bot logged work on HIVE-27020:
-----------------------------------------
Author: ASF GitHub Bot
Created on: 16/Apr/23 18:13
Start Date: 16/Apr/23 18:13
Worklog Time Spent: 10m
Work Description: SourabhBadhya commented on code in PR #4091:
URL: https://github.com/apache/hive/pull/4091#discussion_r1167991431
##########
standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/txn/CompactionTxnHandler.java:
##########
@@ -702,6 +699,102 @@ public void markCleaned(CompactionInfo info) throws
MetaException {
}
}
+ @Override
+ public void markCleanedForAborts(AcidTxnInfo info) throws MetaException {
+ // Do cleanup of TXN_COMPONENTS table
+ LOG.debug("Running markCleanedForAborts with CompactionInfo: {}", info);
+ try {
+ Connection dbConn = null;
+ try {
+ dbConn = getDbConn(Connection.TRANSACTION_READ_COMMITTED,
connPoolCompaction);
+ markAbortCleaned(dbConn, info);
+ LOG.debug("Going to commit");
+ dbConn.commit();
+ } catch (SQLException e) {
+ LOG.error("Unable to delete from txn components due to {}",
e.getMessage());
+ LOG.debug("Going to rollback");
+ rollbackDBConn(dbConn);
+ checkRetryable(e, "markCleanedForAborts(" + info + ")");
+ throw new MetaException("Unable to connect to transaction database " +
+ e.getMessage());
+ } finally {
+ closeDbConn(dbConn);
+ }
+ } catch (RetryException e) {
+ markCleanedForAborts(info);
+ }
+ }
+
+ private void markAbortCleaned(Connection dbConn, AcidTxnInfo info) throws
MetaException, RetryException {
Review Comment:
Renamed. Done.
##########
standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/txn/TxnStore.java:
##########
@@ -516,6 +516,19 @@ Set<CompactionInfo> findPotentialCompactions(int
abortedThreshold, long abortedT
@RetrySemantics.ReadOnly
List<CompactionInfo> findReadyToClean(long minOpenTxnWaterMark, long
retentionTime) throws MetaException;
+ /**
+ * Find the aborted entries in TXN_COMPONENTS which can be used to
+ * clean directories belonging to transactions in aborted state.
+ * @param abortedTimeThreshold Age of table/partition's oldest aborted
transaction involving a given table
+ * or partition that will trigger cleanup.
+ * @param abortedThreshold Number of aborted transactions involving a given
table or partition
+ * that will trigger cleanup.
+ * @return Information of potential abort items that needs to be cleaned.
+ * @throws MetaException
+ */
+ @RetrySemantics.ReadOnly
+ List<AcidTxnInfo> findReadyToCleanForAborts(long abortedTimeThreshold, int
abortedThreshold) throws MetaException;
Review Comment:
Renamed. Done.
##########
standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/txn/TxnStore.java:
##########
@@ -541,6 +554,15 @@ Set<CompactionInfo> findPotentialCompactions(int
abortedThreshold, long abortedT
@RetrySemantics.CannotRetry
void markCleaned(CompactionInfo info) throws MetaException;
+ /**
+ * This will remove an aborted entries from TXN_COMPONENTS table after
+ * the aborted directories are removed from the filesystem.
+ * @param info info on the aborted directories cleanup that needs to be
removed
+ * @throws MetaException
+ */
+ @RetrySemantics.CannotRetry
+ void markCleanedForAborts(AcidTxnInfo info) throws MetaException;
Review Comment:
Removed the API. Extended `markCleaned` with a boolean `isAbortOnly` flag.
Done.
Issue Time Tracking
-------------------
Worklog Id: (was: 857252)
Time Spent: 12h 50m (was: 12h 40m)
> Implement a separate handler to handle aborted transaction cleanup
> ------------------------------------------------------------------
>
> Key: HIVE-27020
> URL: https://issues.apache.org/jira/browse/HIVE-27020
> Project: Hive
> Issue Type: Sub-task
> Reporter: Sourabh Badhya
> Assignee: Sourabh Badhya
> Priority: Major
> Labels: pull-request-available
> Time Spent: 12h 50m
> Remaining Estimate: 0h
>
> As described in the parent task, once the cleaner is separated into different
> entities, implement a separate handler which can create requests for aborted
> transactions cleanup. This would move the aborted transaction cleanup
> exclusively to the cleaner.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
