[
https://issues.apache.org/jira/browse/IMPALA-7975?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16779118#comment-16779118
]
ASF subversion and git services commented on IMPALA-7975:
---------------------------------------------------------
Commit 360b23bfa4eeb337cce380512f106a692bf3c49b in impala's branch
refs/heads/master from Vihang Karajgaonkar
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=360b23b ]
IMPALA-7975 : Improve supportability of the automatic invalidate feature
This patch adds supportability metrics to make it easier to find the
state of Metastore events processor. It adds the following metrics
1. Total number of events received so far
This is not super useful in itself but is helpful to see if there are
events being received or not. A more usable metric which publishes the
rate of such events received is also published. See details for it
below.
2. 1min, 5min and 15min rate of events received
Exponential moving average of number of events received in last 1min,
5min and 15min. This can be used to determine if there are spikes in
event processor activity during certain hours of the day
3. Total number of events skipped so far
Events can be skipped based on certain flags are table and database
level. This metric keeps a count of total number of events skipped so
far. Useful to make decisions like - If most of the events are being
skipped, may be just turn off the event processing. If most of the
events are not skipped, do we need to add flags on certain databases?
4. Status of events processor
Easy way to determine what is the current state of metastore events
processor. Possible states could be STOPPED, ACTIVE, ERROR,
NEEDS_INVALIDATE
5. Events fetch and process durations in seconds
Average duration to fetch a batch of events and process it.
These metrics are displayed in the metrics page under events
metricsgroup. The refresh logic piggybacks on the catalogservermetrics
refresh thread.
Additionally, it also registers a url handler specific for events at
/events. The /events page provides a more detailed view of the metrics
of the event processor which include (min, max, mean, median) of the
durations and rate metrics for all the counters. In the future this page
can be enhanced to show the top-n tables with most number of invalidates
due to the events, etc.
Patch adds few tests to make sure the value of these event metrics are
as expected.
Change-Id: I23cb3aa866879eca03c64ab881796eaa9caa0337
Reviewed-on: http://gerrit.cloudera.org:8080/12549
Reviewed-by: Impala Public Jenkins <[email protected]>
Tested-by: Impala Public Jenkins <[email protected]>
> Improve supportability of the automatic invalidate feature
> ----------------------------------------------------------
>
> Key: IMPALA-7975
> URL: https://issues.apache.org/jira/browse/IMPALA-7975
> Project: IMPALA
> Issue Type: Sub-task
> Reporter: Vihang Karajgaonkar
> Assignee: Vihang Karajgaonkar
> Priority: Major
>
> Some of the things which can be done to improve supportability of this
> feature:
> * Add metrics to detect issues pertaining to this feature
> # Time taken to fetch the notifications (Would be useful to have average,
> min, max)
> # Time taken to process a batch of events received
> # Number of times particular table was invalidated (would be useful to have
> some rate metric like number_of_invalidates/per_hour)
> * Ability to turn ON/OFF for this feature (possibly without the need of a
> restart)
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
