[
https://issues.apache.org/jira/browse/IMPALA-5555?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16846139#comment-16846139
]
Ethan commented on IMPALA-5555:
-------------------------------
A solution has been implemented in the linked issue. Specifically, an end time
value is set in the summary profile managed by the client request state. This
value records the time at which all backend fragment instances have stopped
sending status reports (the end time of a query).
> Add timeline event to query profile to indicate that it is finished
> -------------------------------------------------------------------
>
> Key: IMPALA-5555
> URL: https://issues.apache.org/jira/browse/IMPALA-5555
> Project: IMPALA
> Issue Type: Improvement
> Components: Backend
> Affects Versions: Impala 2.8.0
> Reporter: Jenny Kim
> Assignee: Ethan
> Priority: Major
> Labels: observability, supportability
>
> Hue added a feature where after a user runs a query in Impala, we check the
> Query Profile (from the ImpalaD Web UI) for the RowsProduced statistic (from
> the Coordinator Fragment) and report that back as the total rows returned.
> We're noticing that for some long running queries, the RowsProduced will be
> incorrect (reporting 4 despite getting 198 rows) *right* after the query is
> complete, but will be correct a few seconds later (validated by checking the
> query profile manually). We discovered that by adding a latency of a few
> seconds, we can usually get the correct RowsProduced.
> But I was wondering if there's something smarter we can do, by checking
> either a value in the query profile itself, or somewhere else. We tried
> checking the hasResults value on the Thrift result handle as well as the
> status of the operation handle, but unfortunately these don't seem to have
> any effect (i.e. - they can be True or SUCCESSFUL even though the query
> profile doesn't have the right RowsProduced number).
> Can something be added to the Query Profile itself to indicate that the
> RowsProduced is correct?
> EDIT: Even though the original intent to guarantee that the value for
> RowsProduced was final by relying on profile finalization is not the right
> way to go as documented in the discussion below; it still makes sense to add
> a profile finalization counter to indicate that the final update has been
> recieved from the last fragment.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
