Erik Krogen created MAPREDUCE-6919: -------------------------------------- Summary: ShuffleMetrics.ShuffleConnections Gauge Metric Rises Infinitely Key: MAPREDUCE-6919 URL: https://issues.apache.org/jira/browse/MAPREDUCE-6919 Project: Hadoop Map/Reduce Issue Type: Bug Components: mrv2 Reporter: Erik Krogen
We recently noticed that the mapred.ShuffleMetrics.ShuffleConnections metric rises indefinitely (see attached graph), despite supposedly being a gauge measuring the number of currently open connections: {code:title=ShuffleHandler.java} @Metric("# of current shuffle connections") MutableGaugeInt shuffleConnections; {code} It seems this is because the metric is incremented once for each map file sent, but decremented once for each request. Thus a request which fetches multiple map files permanently increments shuffleConnections by (mapsFetched - 1). -- This message was sent by Atlassian JIRA (v6.4.14#64029) --------------------------------------------------------------------- To unsubscribe, e-mail: mapreduce-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: mapreduce-dev-h...@hadoop.apache.org