[
https://issues.apache.org/jira/browse/IMPALA-8845?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16905607#comment-16905607
]
Tim Armstrong commented on IMPALA-8845:
---------------------------------------
Yeah, I think the issue in IMPALA-3990 should be extremely rare and could be
mitigated by increasing the stream cache duration, so there's probably a good
argument for not worrying about it. It just never sat that well with me because
it was hard to reason about the behaviour. I think the problem that this JIRA
tracks is a much bigger issue in practice.
Agree with your analysis that the problem is that there's not necessarily
enough buffering on the coordinator's side of the exchange for all the sender
fragments to send their final batch and clean themselves up.
We could also consider something like ignoring all error statuses from
fragments once the last row has been appended to the PRS (we already do
something like this when the client has hit EOS)
> Close ExecNode tree prior to calling FlushFinal in FragmentInstanceState
> ------------------------------------------------------------------------
>
> Key: IMPALA-8845
> URL: https://issues.apache.org/jira/browse/IMPALA-8845
> Project: IMPALA
> Issue Type: Sub-task
> Components: Backend
> Reporter: Sahil Takiar
> Assignee: Sahil Takiar
> Priority: Major
>
> While testing IMPALA-8818, I found that IMPALA-8780 does not always cause all
> non-coordinator fragments to shutdown. In certain setups, TopN queries
> ({{select * from [table] order by [col] limit [limit]}}) where all results
> are successfully spooled, still keep non-coordinator fragments alive.
> The issue is that sometimes the {{DATASTREAM SINK}} for the TopN <-- Scan
> Node fragment ends up blocking waiting for a response to a {{TransmitData()}}
> RPC. This prevents the fragment from shutting down.
> I haven't traced the issue exactly, but what I *think* is happening is that
> the {{MERGING-EXCHANGE}} operator in the coordinator fragment hits {{eos}}
> whenever it has received enough rows to reach the limit defined in the query,
> which could occur before the {{DATASTREAM SINK}} sends all the rows from the
> TopN / Scan Node fragment.
> So the TopN / Scan Node fragments end up hanging until they are explicitly
> closed.
> The fix is to close the {{ExecNode}} tree in {{FragmentInstanceState}} as
> eagerly as possible. Moving the close call to before the call to
> {{DataSink::FlushFinal}} fixes the issue. It has the added benefit that it
> shuts down and releases all {{ExecNode}} resources as soon as it can. When
> result spooling is enabled, this is particularly important because
> {{FlushFinal}} might block until the consumer reads all rows.
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
