[
https://issues.apache.org/jira/browse/HIVE-8258?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alan Gates updated HIVE-8258:
-----------------------------
Attachment: HIVE-8258.patch
This patch changes the cleaner to snapshot all of the locks held on a table or
partition when the request comes in. It then watches for those locks to expire
and then does the cleaning. This keeps the cleaner from removing files that
are still being read but at the same time allows the cleaner to ignore any
subsequent locks. This works since any process that obtained a lock after the
cleaner will be reading the newer versions of the file.
> Compactor cleaners can be starved on a busy table or partition.
> ---------------------------------------------------------------
>
> Key: HIVE-8258
> URL: https://issues.apache.org/jira/browse/HIVE-8258
> Project: Hive
> Issue Type: Bug
> Components: Transactions
> Affects Versions: 0.13.1
> Reporter: Alan Gates
> Assignee: Alan Gates
> Priority: Critical
> Attachments: HIVE-8258.patch
>
>
> Currently the cleaning thread in the compactor does not run on a table or
> partition while any locks are held on this partition. This leaves it open to
> starvation in the case of a busy table or partition. It only needs to wait
> until all locks on the table/partition at the time of the compaction have
> expired. Any jobs initiated after that (and thus any locks obtained) will be
> for the new versions of the files.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
