[
https://issues.apache.org/jira/browse/IMPALA-6924?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16709552#comment-16709552
]
ASF subversion and git services commented on IMPALA-6924:
---------------------------------------------------------
Commit 91c754bf0ac707d0b2cee23c11ef41d7c81dd0f0 in impala's branch
refs/heads/master from [~twmarshall]
[ https://git-wip-us.apache.org/repos/asf?p=impala.git;h=91c754b ]
IMPALA-6924: Add child queries to profile in compute stats
COMPUTE STATS triggers two child queries which do the actual stats
calculation. This patch fetches the profiles for the child queries and
adds them to the profile for the COMPUTE STATS to make it easier to
debug issues with the child queries.
To enable this, this patch also adds a 'format' parameter to
GetRuntimeProfile(), which allows clients to retrieve the profile as
either a pretty printed string (currently the only option), as a
base64 encoded string, or as a thrift structure. This allows the child
query to add the profile directly as a child of the parent query's
profile.
Note that the 'format' parameter is only available for the HiveServer2
client and not for Beeswax. This is because Thrift does not appear to
have proper support for default parameters to service methods, so
adding the 'format' parameter would not be backwards compatible with
existing Impala Beeswax clients. This does not affect HiveServer2
as it takes a 'request' struct as its only parameter and Thrift does
support default values for struct fields.
This patch also fixes a bug where '__isset' was not being set in the
Thrift runtime profile for the exec summry.
Testing:
- Ran COMPUTE STATS and verified that the profile contains the
expected output both when the child queries succeed and when they
fail.
- Added an e2e test that runs a COMPUTE STATS and checks that there
are three unique query ids in the profile.
- Added a BE test that verifies the archive string (de)serialization
functions work.
Change-Id: I5006c3b366d381eed4687e550cdfc463be3d1350
Reviewed-on: http://gerrit.cloudera.org:8080/11977
Reviewed-by: Impala Public Jenkins <[email protected]>
Tested-by: Impala Public Jenkins <[email protected]>
> Compute stats profiles should include reference to child queries
> ----------------------------------------------------------------
>
> Key: IMPALA-6924
> URL: https://issues.apache.org/jira/browse/IMPALA-6924
> Project: IMPALA
> Issue Type: Improvement
> Components: Backend
> Affects Versions: Impala 3.0, Impala 2.12.0
> Reporter: Tim Armstrong
> Assignee: Thomas Tauber-Marshall
> Priority: Major
> Labels: observability, supportability
> Fix For: Impala 3.2.0
>
>
> "Compute stats" queries spawn off child queries that do most of the work.
> It's non-trivial to track down the child queries and get their profiles if
> something goes wrong. We really should have, at a minimum, the query IDs of
> the child queries in the parent's profile and vice-versa.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
