aweisberg commented on code in PR #3068:
URL: https://github.com/apache/cassandra/pull/3068#discussion_r1476249964
##########
src/java/org/apache/cassandra/metrics/CassandraMetricsRegistry.java:
##########
@@ -250,6 +271,43 @@ private void removeAlias(MetricName name)
if (mBeanServer.isRegistered(name.getMBeanName()))
MBeanWrapper.instance.unregisterMBean(name.getMBeanName(),
MBeanWrapper.OnException.IGNORE);
}
+
+ /**
+ * Very infrequently used meters generate a linear amount of tick work
based on how long it has been
+ * since the meter was last marked or read. On scales of a year this can
be enough to cause the first request
+ * that needs to mark the meter to time out. Once a day read every meter
to force them to run Meter.tickIfNecessary
+ * so we only ever run at most one day worth of tick work per meter in the
request path.
+ *
+ * This can be removed if we ever upgrade and switch the default
MovingAverage from EWMA to SlidingWindowTimeAverages
+ */
+ private void tickMeters()
+ {
+ List<Throwable> failures = new ArrayList<>();
+ int droppedFailures = 0;
+ for (Meter meter : getMeters().values())
+ {
+ try
+ {
+ meter.getOneMinuteRate();
+ }
+ catch (Throwable t)
+ {
+ if (failures.size() < 10)
+ failures.add(t);
+ else
+ droppedFailures++;
+ }
+ }
+ if (!failures.isEmpty())
+ {
+ Throwable failure = null;
+ for (Throwable t : failures)
+ failure = Throwables.merge(failure, t);
+ // To avoid the scheduled task being cancelled don't leak
exceptions
+ // Runs only once a day so noise is not an issue
+ logger.error(String.format("Had error(s) attempting to tick meter.
Dropped %d exceptions.", droppedFailures), failure);
Review Comment:
It's not something that is ever expected so if it does happen things could
be pretty broken not just with this ticking functionality. Since it will only
log once a day it seems more appropriate to make it an error so it's more
visible and can get fixed.
Warn would be fine, but makes it a little easier for errors to get lost in
noise. This is once a day so it wouldn't contribute a lot of noise.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
