Surya Hebbar has uploaded a new patch set (#8). (
http://gerrit.cloudera.org:8080/24576 )
Change subject: IMPALA-15174: Add AGGREGATED_PROFILE query option to toggle
profile formats
......................................................................
IMPALA-15174: Add AGGREGATED_PROFILE query option to toggle profile formats
With this change, the aggregated runtime profile is exposed as a query
option named 'AGGREGATED_PROFILE', whose default value is set according
to the 'aggregated_profile' startup flag (which defaults to 'false').
It has also been added as an attribute to the query state record.
The 'aggregated_profile' flag serves as a replacement for the
'gen_experimental_profile' flag.
To help users and tests differentiate between profile types, an info
string 'Profile Type' has been added to the execution profile. The
supported values of this info string are represented by the following
enum values:
RuntimeProfileBase::Type:
AGGREGATED - Aggregated profile
UNAGGREGATED - Traditional profile
Dependencies on the global 'gen_experimental_profile' startup flag have
been refactored as follows:
1. 'DefaultVerbosity' method -
This method was removed. The appropriate verbosity is now determined
by the profile type and passed explicitly at runtime.
2. 'profile_version' attribute -
The global, startup-flag-based profile version is now set based on
the session-level "Profile Version" info string.
Note: "Profile Version" is set based on the profile type.
Allowing to switch profile formats at runtime enables us to run
end-to-end tests that are not yet compatible with the aggregated profile.
This aids in the transition to the aggregated profile (i.e., IMPALA-9846).
Testing:
- Updated the Thrift, text, and JSON outputs for the profile tool tests
to align with the latest format changes.
Change-Id: I96aceda1b044ae53d09e68cb78678fc2930326dd
---
M be/src/experiments/data-provider-test.cc
M be/src/experiments/tuple-splitter-test.cc
M be/src/runtime/coordinator-backend-state.cc
M be/src/runtime/coordinator-backend-state.h
M be/src/runtime/coordinator.cc
M be/src/service/impala-server.cc
M be/src/service/query-options.cc
M be/src/service/query-options.h
M be/src/service/query-state-record.cc
M be/src/service/query-state-record.h
M be/src/util/impala-profile-tool.cc
M be/src/util/runtime-profile-archive.cc
M be/src/util/runtime-profile-test.cc
M be/src/util/runtime-profile.cc
M be/src/util/runtime-profile.h
M common/protobuf/control_service.proto
M common/thrift/ImpalaService.thrift
M common/thrift/Query.thrift
M testdata/impala-profiles/README
M testdata/impala-profiles/impala_profile_log_tpcds_compute_stats
M testdata/impala-profiles/impala_profile_log_tpcds_compute_stats.expected.json
M
testdata/impala-profiles/impala_profile_log_tpcds_compute_stats.expected.pretty.json
M testdata/impala-profiles/impala_profile_log_tpcds_compute_stats.expected.txt
M
testdata/impala-profiles/impala_profile_log_tpcds_compute_stats_default.expected.txt
M
testdata/impala-profiles/impala_profile_log_tpcds_compute_stats_extended.expected.pretty.json
M
testdata/impala-profiles/impala_profile_log_tpcds_compute_stats_extended.expected.txt
M testdata/impala-profiles/impala_profile_log_tpcds_compute_stats_v2
M
testdata/impala-profiles/impala_profile_log_tpcds_compute_stats_v2.expected.json
M
testdata/impala-profiles/impala_profile_log_tpcds_compute_stats_v2_default.expected.txt
M
testdata/impala-profiles/impala_profile_log_tpcds_compute_stats_v2_extended.expected.pretty.json
M
testdata/impala-profiles/impala_profile_log_tpcds_compute_stats_v2_extended.expected.txt
M tests/custom_cluster/test_executor_groups.py
M tests/custom_cluster/test_observability.py
M tests/custom_cluster/test_query_log.py
M tests/custom_cluster/test_runtime_profile.py
35 files changed, 75,765 insertions(+), 65,632 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/76/24576/8
--
To view, visit http://gerrit.cloudera.org:8080/24576
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I96aceda1b044ae53d09e68cb78678fc2930326dd
Gerrit-Change-Number: 24576
Gerrit-PatchSet: 8
Gerrit-Owner: Surya Hebbar <[email protected]>
Gerrit-Reviewer: Csaba Ringhofer <[email protected]>
Gerrit-Reviewer: Impala Public Jenkins <[email protected]>
Gerrit-Reviewer: Kurt Deschler <[email protected]>
Gerrit-Reviewer: Surya Hebbar <[email protected]>
