Impala Public Jenkins has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/16881 )
Change subject: IMPALA-9865: part 2/2: add verbosity to profile tool ...................................................................... IMPALA-9865: part 2/2: add verbosity to profile tool Adds a --profile_verbosity option for impala-profile-tool with the following levels: * 0: minimal * 1: legacy - matches old output, this is the default still * 2: default - basic descriptive stats, used for V2 profile. * 3: extended * 4: full This will help with transition to the V2 profile because we can have a nice, high-level, readable text profile by default with the option to produce more detailed profiles and alternate views of the profile from the thrift profile. Use the profile version in impala-profile-tool to dump the more verbose output for the V2 profile while preserving the same output for the legacy profile. Reduce verbosity of v2 profile output - only include mean/min/max by default. I intend to refine the output at the different verbosity levels for the v2 profiles further as part of IMPALA-9382, it is still fairly noisy. Fix output with/without gen_experimental_profile - there was a small difference in that the summary stats were not output in the averaged profile. Testing: * Add an end-to-end test that generates output for a small profile log and compares against expected files. * Tweak other profile tests to reflect changes to output. Change-Id: I82618a813e29af7996dfaed78873b2a73bc0231d Reviewed-on: http://gerrit.cloudera.org:8080/16881 Reviewed-by: Impala Public Jenkins <[email protected]> Tested-by: Impala Public Jenkins <[email protected]> --- M be/src/util/impala-profile-tool.cc M be/src/util/runtime-profile-counters.h M be/src/util/runtime-profile-test.cc M be/src/util/runtime-profile.cc M be/src/util/runtime-profile.h M bin/jenkins/dockerized-impala-run-tests.sh M bin/rat_exclude_files.txt A testdata/impala-profiles/README A testdata/impala-profiles/impala_profile_log_tpcds_compute_stats A testdata/impala-profiles/impala_profile_log_tpcds_compute_stats.expected.json A testdata/impala-profiles/impala_profile_log_tpcds_compute_stats.expected.pretty.json A testdata/impala-profiles/impala_profile_log_tpcds_compute_stats.expected.pretty_extended.json A testdata/impala-profiles/impala_profile_log_tpcds_compute_stats.expected.txt A testdata/impala-profiles/impala_profile_log_tpcds_compute_stats_default.expected.txt A testdata/impala-profiles/impala_profile_log_tpcds_compute_stats_extended.expected.txt M testdata/workloads/tpch/queries/runtime-profile-aggregated.test A tests/observability/test_profile_tool.py M tests/query_test/test_scanners.py 18 files changed, 95,019 insertions(+), 125 deletions(-) Approvals: Impala Public Jenkins: Looks good to me, approved; Verified -- To view, visit http://gerrit.cloudera.org:8080/16881 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: I82618a813e29af7996dfaed78873b2a73bc0231d Gerrit-Change-Number: 16881 Gerrit-PatchSet: 10 Gerrit-Owner: Tim Armstrong <[email protected]> Gerrit-Reviewer: Impala Public Jenkins <[email protected]> Gerrit-Reviewer: Joe McDonnell <[email protected]> Gerrit-Reviewer: Riza Suminto <[email protected]> Gerrit-Reviewer: Tim Armstrong <[email protected]>
