s0nskar commented on code in PR #3221:
URL: https://github.com/apache/celeborn/pull/3221#discussion_r2063366602
##########
worker/src/main/scala/org/apache/celeborn/service/deploy/worker/WorkerSource.scala:
##########
@@ -111,8 +111,12 @@ class WorkerSource(conf: CelebornConf) extends
AbstractSource(conf, Role.WORKER)
val applicationIds =
appActiveConnections.get(client.getChannel.id().asLongText())
val applicationId = Utils.splitShuffleKey(shuffleKey)._1
if (applicationIds != null && !applicationIds.contains(applicationId)) {
- addCounter(ACTIVE_CONNECTION_COUNT, Map(applicationLabel ->
applicationId))
- incCounter(ACTIVE_CONNECTION_COUNT, 1, Map(applicationLabel ->
applicationId))
+ var labels = Map.empty[String, String]
+ if (conf.metricsPushApplicationId) {
+ labels += (applicationLabel -> applicationId)
+ addCounter(ACTIVE_CONNECTION_COUNT, labels)
Review Comment:
@turboFei Adding an API for checking top application consumption on a worker
can be helpful for debugging the usage, i filed a ticket for the same earlier –
https://issues.apache.org/jira/browse/CELEBORN-1975
In our case, for debugging we check userIdentifier consumption and figure
out applicationId using spark listener audit logs.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
