[
https://issues.apache.org/jira/browse/IMPALA-12385?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17793952#comment-17793952
]
ASF subversion and git services commented on IMPALA-12385:
----------------------------------------------------------
Commit d75807a195273cdba29e33f00b7b6f9bee012f62 in impala's branch
refs/heads/master from Kurt Deschler
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=d75807a19 ]
IMPALA-12385: Enable Periodic metrics by default
This patch enables periodic metrics in query profiles by default and
changes the metric collectors to be more suitable for mixed workloads.
-Changed default of resource_trace_ratio to 1.
-Changed profile metrics to use sampling counters which can automatically
resize for long queries.
-Reduced profile metric samping interval to 50ms to support short-running
queries.
-Changed fragment metrics to use the same sampling interval as periodic
metrics.
-Added singleton PeriodicCounterUpdater and thread for updating system
(KRPC) metrics at a different period than fragment metrics.
-Added new flag periodic_system_counter_update_period_ms for configuring
system metric update period. Default is 500ms.
Example profile output:
- HostCpuIoWaitPercentage (400.000ms): 1, 0, 2, 3, 4, 6, 5, 2, 1, ...
- HostCpuSysPercentage (400.000ms): 1, 12, 10, 11, 11, 5, 3, 3, 3, ...
- HostCpuUserPercentage (400.000ms): 8, 46, 39, 39, 39, 29, 22, 23, ...
Testing:
-Updated runtime-profile-test and test_observability.py for new defaults
-Manual inspection of query profile metrics for long-running queries
-Tested for performance regression using 50 concurrent TPCH Q1 and with
periodic_counter_update_period_ms=1 to verify that the periodic update
loop does not affect peformance or become a bottleneck
Change-Id: Ic8e5cbfd4b324081158574ceb8f4b3a062a69fd1
Reviewed-on: http://gerrit.cloudera.org:8080/20377
Tested-by: Impala Public Jenkins <[email protected]>
Reviewed-by: Michael Smith <[email protected]>
> Enable Periodic metrics by default
> ----------------------------------
>
> Key: IMPALA-12385
> URL: https://issues.apache.org/jira/browse/IMPALA-12385
> Project: IMPALA
> Issue Type: Improvement
> Components: Backend
> Reporter: Kurt Deschler
> Assignee: Kurt Deschler
> Priority: Major
> Fix For: Impala 4.4.0
>
>
> Periodic metrics currently require user to set resource_trace_ratio and in
> many cases to restart backends with different
> periodic_counter_update_period_ms settings. Changing the defaults and tuning
> the collection mechanisms will make metrics always available and they can be
> turned back off if not desired.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
