Vuk Ercegovac has uploaded this change for review. (
http://gerrit.cloudera.org:8080/11534
Change subject: IMPALA-7622: adds profile metrics when fetching incremental
stats
......................................................................
IMPALA-7622: adds profile metrics when fetching incremental stats
When computing incremental statistics by fetching the stats directly
from catalogd, a potentially expensive RPC is made from the impalad
coordinator to catalogd. This change adds metrics to the frontend
section of the profile to track how long the request takes, the size
of the compressed bytes received, and the number of partitions received.
The profile for a 'compute incremental ...' command on a table with
no statistics looks like this:
Frontend:
- StatsFetch.Bytes: 0
- StatsFetch.NumPartitionsReceived: 24
- StatsFetch.NumPartitionsWithStats: 0
- StatsFetch.Time: 26ms
And the profile looks as follows when the table has stats, so the stats
are fetched:
Frontend:
- StatsFetch.Bytes: 24622
- StatsFetch.NumPartitionsReceived: 23
- StatsFetch.NumPartitionsWithStats: 23
- StatsFetch.Time: 14ms
Testing:
- manual inspection
- e2e test to check the profile
Change-Id: Ic9b268548c7a98c751eb99855ee08313d1d5a903
---
M fe/src/main/java/org/apache/impala/analysis/ComputeStatsStmt.java
M tests/common/custom_cluster_test_suite.py
M tests/custom_cluster/test_pull_stats.py
3 files changed, 72 insertions(+), 3 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/34/11534/1
--
To view, visit http://gerrit.cloudera.org:8080/11534
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newchange
Gerrit-Change-Id: Ic9b268548c7a98c751eb99855ee08313d1d5a903
Gerrit-Change-Number: 11534
Gerrit-PatchSet: 1
Gerrit-Owner: Vuk Ercegovac <[email protected]>
