Impala Public Jenkins has submitted this change and it was merged. (
http://gerrit.cloudera.org:8080/11534 )
Change subject: IMPALA-7622: adds profile metrics when fetching incremental
stats
......................................................................
IMPALA-7622: adds profile metrics when fetching incremental stats
When computing incremental statistics by fetching the stats directly
from catalogd, a potentially expensive RPC is made from the impalad
coordinator to catalogd. This change adds metrics to the frontend
section of the profile to track how long the request takes, the size
of the compressed bytes received, and the number of partitions received.
The profile for a 'compute incremental ...' command on a table with
no statistics looks like this:
Frontend:
- StatsFetch.CompressedBytes: 0
- StatsFetch.TotalPartitions: 24
- StatsFetch.NumPartitionsWithStats: 0
- StatsFetch.Time: 26ms
And the profile looks as follows when the table has stats, so the stats
are fetched:
Frontend:
- StatsFetch.CompressedBytes: 24622
- StatsFetch.TotalPartitions: 23
- StatsFetch.NumPartitionsWithStats: 23
- StatsFetch.Time: 14ms
Testing:
- manual inspection
- e2e test to check the profile
Change-Id: Ic9b268548c7a98c751eb99855ee08313d1d5a903
Reviewed-on: http://gerrit.cloudera.org:8080/11534
Reviewed-by: Vuk Ercegovac <[email protected]>
Tested-by: Impala Public Jenkins <[email protected]>
---
M fe/src/main/java/org/apache/impala/analysis/ComputeStatsStmt.java
M tests/common/custom_cluster_test_suite.py
M tests/custom_cluster/test_pull_stats.py
3 files changed, 93 insertions(+), 3 deletions(-)
Approvals:
Vuk Ercegovac: Looks good to me, approved
Impala Public Jenkins: Verified
--
To view, visit http://gerrit.cloudera.org:8080/11534
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: merged
Gerrit-Change-Id: Ic9b268548c7a98c751eb99855ee08313d1d5a903
Gerrit-Change-Number: 11534
Gerrit-PatchSet: 4
Gerrit-Owner: Vuk Ercegovac <[email protected]>
Gerrit-Reviewer: Bharath Vissapragada <[email protected]>
Gerrit-Reviewer: Impala Public Jenkins <[email protected]>
Gerrit-Reviewer: Tianyi Wang <[email protected]>
Gerrit-Reviewer: Vuk Ercegovac <[email protected]>
