[
https://issues.apache.org/jira/browse/IMPALA-7799?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16672291#comment-16672291
]
Philip Zeyliger commented on IMPALA-7799:
-----------------------------------------
I've used a handful of systems that dump these to logs. In the absence of
fancier stuff, I suspect a thread which dumps the metrics-json to the log file
every minute would be basically ok.
> Store periodic snapshots of Impalad metrics
> -------------------------------------------
>
> Key: IMPALA-7799
> URL: https://issues.apache.org/jira/browse/IMPALA-7799
> Project: IMPALA
> Issue Type: Improvement
> Components: Backend
> Affects Versions: Impala 3.0, Impala 2.12.0, Impala 3.1.0
> Reporter: Michael Ho
> Priority: Critical
>
> Currently, each Impala demon exposes a set of metrics exposed via the debug
> webpage metrics page. While this may be very helpful for development, there
> are many incidents in which one may want to record these metrics to do
> postmortem analysis for various issues.
> We should consider taking periodic snapshots of these Impalad metrics and
> archive them for a certain retention period to allow for postmortem analysis.
> We need to be mindful of the space usage concern (e.g. using compressed json
> ?)
> To enable easier analysis, one may need to build a tool (or use some existing
> off-the-shell libraries) to show the collected snapshots as time series and
> calculate various statistics (e.g. mean, median, min, max etc).
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
