Impala Public Jenkins has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/11977 )
Change subject: IMPALA-6924: Add child queries to profile in compute stats ...................................................................... IMPALA-6924: Add child queries to profile in compute stats COMPUTE STATS triggers two child queries which do the actual stats calculation. This patch fetches the profiles for the child queries and adds them to the profile for the COMPUTE STATS to make it easier to debug issues with the child queries. To enable this, this patch also adds a 'format' parameter to GetRuntimeProfile(), which allows clients to retrieve the profile as either a pretty printed string (currently the only option), as a base64 encoded string, or as a thrift structure. This allows the child query to add the profile directly as a child of the parent query's profile. Note that the 'format' parameter is only available for the HiveServer2 client and not for Beeswax. This is because Thrift does not appear to have proper support for default parameters to service methods, so adding the 'format' parameter would not be backwards compatible with existing Impala Beeswax clients. This does not affect HiveServer2 as it takes a 'request' struct as its only parameter and Thrift does support default values for struct fields. This patch also fixes a bug where '__isset' was not being set in the Thrift runtime profile for the exec summry. Testing: - Ran COMPUTE STATS and verified that the profile contains the expected output both when the child queries succeed and when they fail. - Added an e2e test that runs a COMPUTE STATS and checks that there are three unique query ids in the profile. - Added a BE test that verifies the archive string (de)serialization functions work. Change-Id: I5006c3b366d381eed4687e550cdfc463be3d1350 Reviewed-on: http://gerrit.cloudera.org:8080/11977 Reviewed-by: Impala Public Jenkins <[email protected]> Tested-by: Impala Public Jenkins <[email protected]> --- M be/src/service/child-query.cc M be/src/service/child-query.h M be/src/service/client-request-state.cc M be/src/service/impala-beeswax-server.cc M be/src/service/impala-hs2-server.cc M be/src/service/impala-http-handler.cc M be/src/service/impala-server.cc M be/src/service/impala-server.h M be/src/util/runtime-profile-test.cc M be/src/util/runtime-profile.cc M be/src/util/runtime-profile.h M common/thrift/ImpalaService.thrift M common/thrift/RuntimeProfile.thrift M tests/query_test/test_observability.py 14 files changed, 178 insertions(+), 20 deletions(-) Approvals: Impala Public Jenkins: Looks good to me, approved; Verified -- To view, visit http://gerrit.cloudera.org:8080/11977 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: I5006c3b366d381eed4687e550cdfc463be3d1350 Gerrit-Change-Number: 11977 Gerrit-PatchSet: 8 Gerrit-Owner: Thomas Marshall <[email protected]> Gerrit-Reviewer: Andrew Sherman <[email protected]> Gerrit-Reviewer: Impala Public Jenkins <[email protected]> Gerrit-Reviewer: Thomas Marshall <[email protected]> Gerrit-Reviewer: Tim Armstrong <[email protected]>
