[
https://issues.apache.org/jira/browse/HIVE-19124?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16451231#comment-16451231
]
Eugene Koifman commented on HIVE-19124:
---------------------------------------
a simpler approach would have been to address HIVE-18570 first by requiring X
lock IOW, then by definition there would be no open txns at the time IOW runs
(and similarly no open write txns when X lock is replaced by Write lock which
still allows readers). Unfortunately it's no longer possible since HIVE-18825
moved snapshot acquisition to be before lock acquisition.
I think a larger issue (including Gopal's proposal) is that it gets the wrong
HWM. If you look at Worker,
it creates a Compaction writeIldList and stores the HWM in the metastore. This
is important so that Cleaner later doesn't clean above this mark, i.e. the
highest level compaction looked at.
In this patch, the MM compactor IOW query creates it's own Snapshot which may
be different. Perhaps a more consistent approach is to set the snapshot on the
Driver in compaction mode from Worker to make this consistent.
> implement a basic major compactor for MM tables
> -----------------------------------------------
>
> Key: HIVE-19124
> URL: https://issues.apache.org/jira/browse/HIVE-19124
> Project: Hive
> Issue Type: Bug
> Components: Transactions
> Reporter: Sergey Shelukhin
> Assignee: Sergey Shelukhin
> Priority: Major
> Labels: mm-gap-2
> Attachments: HIVE-19124.01.patch, HIVE-19124.02.patch,
> HIVE-19124.03.patch, HIVE-19124.03.patch, HIVE-19124.04.patch,
> HIVE-19124.05.patch, HIVE-19124.06.patch, HIVE-19124.07.patch,
> HIVE-19124.patch
>
>
> For now, it will run a query directly and only major compactions will be
> supported.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
