[
https://issues.apache.org/jira/browse/HIVE-11515?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14700569#comment-14700569
]
Siddharth Seth commented on HIVE-11515:
---------------------------------------
[~navis] - Do you have more details on how the query fails in the situations
where you see this occur ? Is there an exception or does the query hang or some
other manifestation ?
I think the patch is primarily moving the registerForVertexNotifications into
the constructor - which is fine. However, the condition where an event is
received before registering for vertex success notifications is already handled
in checkForSourceCompletion
{code}
int expectedEvents = numExpectedEventsPerSource.get(name).getValue();
if (expectedEvents < 0) {
// Expected events not updated yet - vertex SUCCESS notification not
received.
return;
} else {
{code}
Even if all the events were to come in before registering for the notification,
when prune is finally called - and a notification is received, these events
will be processed.
On the Tez side, it makes sure to send in events only after the Initializer has
been constructed - that's the HiveSplitGenerator.
> Still some possible race condition in DynamicPartitionPruner
> ------------------------------------------------------------
>
> Key: HIVE-11515
> URL: https://issues.apache.org/jira/browse/HIVE-11515
> Project: Hive
> Issue Type: Bug
> Components: Query Processor, Tez
> Reporter: Navis
> Assignee: Navis
> Priority: Minor
> Attachments: HIVE-11515.1.patch.txt
>
>
> Even after HIVE-9976, I could see race condition in DPP sometimes. Hard to
> reproduce but it seemed related to the fact that prune() is called by
> thread-pool. With some delay in queue, events from fast tasks are arrived
> before prune() is called.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
