[
https://issues.apache.org/jira/browse/IMPALA-6984?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16692357#comment-16692357
]
Michael Ho edited comment on IMPALA-6984 at 11/19/18 10:19 PM:
---------------------------------------------------------------
Now that IMPALA-4063 is fixed, we should have a guarantee that the a final
profile will be sent even when the fragment instances are cancelled. This is
guaranteed as cancellation won't have any effect until all fragment instances
on a backend have finished calling {{FIS:Prepare()}} so the query state thread
will definitely send one report upon completion of all fragment instances.
However, the logic in {{Coordinator::BackendState::Done()}} is preventing the
final profile from being applied as {{status_}} is already set to
{{CANCELLED}}. It may need to be fixed up in order for the coordinator to
cleanly cancel the fragment instances on EOS.
{noformat}
inline bool Coordinator::BackendState::IsDone() const {
return num_remaining_instances_ == 0 || !status_.ok();
}
{noformat}
was (Author: kwho):
Now that IMPALA-4063 is fixed, we should have a guarantee that the a final
profile will be sent even when the fragment instances are cancelled. This is
guaranteed as cancellation won't have any effect until the FIS on a backend has
passed the preparation stage so the query state thread will definitely send one
report upon completion of all fragment instancse.
However, the logic in {{Coordinator::BackendState::Done()}} is preventing the
final profile from being applied as {{status_}} is already set to
{{CANCELLED}}. It may need to be fixed up in order for the coordinator to
cleanly cancel the fragment instances on EOS.
{noformat}
inline bool Coordinator::BackendState::IsDone() const {
return num_remaining_instances_ == 0 || !status_.ok();
}
{noformat}
> Coordinator should cancel backends when returning EOS
> -----------------------------------------------------
>
> Key: IMPALA-6984
> URL: https://issues.apache.org/jira/browse/IMPALA-6984
> Project: IMPALA
> Issue Type: Improvement
> Components: Backend
> Affects Versions: Impala 3.0
> Reporter: Dan Hecht
> Priority: Major
> Labels: query-lifecycle
>
> Currently, the Coordinator waits for backends rather than proactively
> cancelling them in the case of hitting EOS. There's a tangled mess that makes
> it tricky to proactively cancel the backends related to how
> {{Coordinator::ComputeQuerySummary()}} works – we can't update the summary
> until the profiles are no longer changing (which also makes sense given that
> we want the exec summary to be consistent with the final profile). But we
> current tie together the FIS status and the profile, and cancellation of
> backends causes the FIS to return CANCELLED, which then means that the
> remaining FIS on that backend won't produce a final profile.
> With the rework of the protocol for IMPALA-2990 we should make it possible to
> sort this out such that a final profile can be requested regardless of how a
> FIS ends execution.
> This also relates to IMPALA-5783.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
