[
https://issues.apache.org/jira/browse/HIVE-27637?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17813561#comment-17813561
]
Zsolt Miskolczi commented on HIVE-27637:
----------------------------------------
[~sbadhya] , thank you for the idea of improvement and a review.
> Compare highest write ID of compaction records when trying to perform abort
> cleanup
> -----------------------------------------------------------------------------------
>
> Key: HIVE-27637
> URL: https://issues.apache.org/jira/browse/HIVE-27637
> Project: Hive
> Issue Type: Task
> Components: Hive
> Reporter: Zsolt Miskolczi
> Assignee: Zsolt Miskolczi
> Priority: Major
> Labels: pull-request-available
> Fix For: 4.0.0
>
>
> Compare highest write ID of compaction records when trying to get the
> potential table/partitions for abort cleanup.
> Idea: If there exists a highest write ID of a record in COMPACTION_QUEUE for
> a table/partition which is greater than the max(aborted write ID) for that
> table/partition, then we can potentially ignore abort cleanup for such
> tables/partitions. This is because compaction will perform cleanup of
> obsolete deltas and aborted deltas hence doing abort cleanup is redundant
> here.
> This is more of an optimisation since it can potentially save some filesystem
> operations (mainly file-listing during construction of Acid state).
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
