[
https://issues.apache.org/jira/browse/HIVE-29186?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
tanishqchugh updated HIVE-29186:
--------------------------------
Description:
In case of multiple HS2 instances, if there is an HMS instance restart (Running
the initiator) it leads to changing the state of all compaction jobs of state
'Working' in Compaction Queue back to Initiated.
If the workers are running in HS2, which survives the HMS restart it tends to
continue the compaction job, but another worker can pick up the same job as it
is in initiated state. It leads to multiple workers carrying out same
compaction job, leading to duplicate Dirs/records.
was:
In case of multiple HS2 instances, if there is an HMS instance restart it leads
to changing the state of all compaction jobs of state 'Working' in Compaction
Queue back to Initiated.
If the workers are running in HS2, which survives the HMS restart it tends to
continue the compaction job, but another worker can pick up the same job as it
is in initiated state. It leads to multiple workers carrying out same
compaction job, leading to duplicate Dirs/records.
> Compactor workers should be revoked only if running in metastore in case of
> HMS instance restart
> ------------------------------------------------------------------------------------------------
>
> Key: HIVE-29186
> URL: https://issues.apache.org/jira/browse/HIVE-29186
> Project: Hive
> Issue Type: Bug
> Reporter: tanishqchugh
> Assignee: tanishqchugh
> Priority: Major
>
> In case of multiple HS2 instances, if there is an HMS instance restart
> (Running the initiator) it leads to changing the state of all compaction jobs
> of state 'Working' in Compaction Queue back to Initiated.
> If the workers are running in HS2, which survives the HMS restart it tends to
> continue the compaction job, but another worker can pick up the same job as
> it is in initiated state. It leads to multiple workers carrying out same
> compaction job, leading to duplicate Dirs/records.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
