Hello Andrew Sherman, Tim Armstrong, Impala Public Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/11977
to look at the new patch set (#6).
Change subject: IMPALA-6924: Add child queries to profile in compute stats
......................................................................
IMPALA-6924: Add child queries to profile in compute stats
COMPUTE STATS triggers two child queries which do the actual stats
calculation. This patch fetches the profiles for the child queries and
adds them to the profile for the COMPUTE STATS to make it easier to
debug issues with the child queries.
To enable this, this patch also adds a 'format' parameter to
GetRuntimeProfile(), which allows clients to retrieve the profile as
either a pretty printed string (currently the only option), as a
base64 encoded string, or as a thrift structure. This allows the child
query to add the profile directly as a child of the parent query's
profile.
Note that the 'format' parameter is only available for the HiveServer2
client and not for Beeswax. This is because Thrift does not appear to
have proper support for default parameters to service methods, so
adding the 'format' parameter would not be backwards compatible with
existing Impala Beeswax clients. This does not affect HiveServer2
as it takes a 'request' struct as its only parameter and Thrift does
support default values for struct fields.
This patch also fixes a bug where '__isset' was not being set in the
Thrift runtime profile for the exec summry.
Testing:
- Ran COMPUTE STATS and verified that the profile contains the
expected output both when the child queries succeed and when they
fail.
- Added an e2e test that runs a COMPUTE STATS and checks that there
are three unique query ids in the profile.
- Added a BE test that verifies the archive string (de)serialization
functions work.
Change-Id: I5006c3b366d381eed4687e550cdfc463be3d1350
---
M be/src/service/child-query.cc
M be/src/service/child-query.h
M be/src/service/client-request-state.cc
M be/src/service/impala-beeswax-server.cc
M be/src/service/impala-hs2-server.cc
M be/src/service/impala-http-handler.cc
M be/src/service/impala-server.cc
M be/src/service/impala-server.h
M be/src/util/runtime-profile-test.cc
M be/src/util/runtime-profile.cc
M be/src/util/runtime-profile.h
M common/thrift/ImpalaService.thrift
M common/thrift/RuntimeProfile.thrift
M tests/query_test/test_observability.py
14 files changed, 178 insertions(+), 20 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/77/11977/6
--
To view, visit http://gerrit.cloudera.org:8080/11977
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I5006c3b366d381eed4687e550cdfc463be3d1350
Gerrit-Change-Number: 11977
Gerrit-PatchSet: 6
Gerrit-Owner: Thomas Marshall <thomasmarsh...@cmu.edu>
Gerrit-Reviewer: Andrew Sherman <asher...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <impala-public-jenk...@cloudera.com>
Gerrit-Reviewer: Thomas Marshall <thomasmarsh...@cmu.edu>
Gerrit-Reviewer: Tim Armstrong <tarmstr...@cloudera.com>
