kaushik-develop commented on a change in pull request #12178:
URL: https://github.com/apache/pulsar/pull/12178#discussion_r717748796
##########
File path:
pulsar-functions/worker/src/main/java/org/apache/pulsar/functions/worker/SchedulerManager.java
##########
@@ -415,6 +633,34 @@ private void compactAssignmentTopic() {
}
}
+ protected int updateWorkerDrainMap() {
+ long startTime = System.nanoTime();
+ int numRemovedWorkerIds = 0;
+
+ if (drainOpStatusMap.size() > 0) {
Review comment:
I didn't quite understand the question: "Is there a reason we should
synchronize this with drain operation?"
updateWorkerDrainMap() does a periodic cleanup of stale information about
drained workers. We need to do the operation some time after the drain has
finished, when the drained worker is removed from the cluster. Since there is
currently no hook (that I know of) into the SchedulerManager when a worker is
added to, or removed from the cluster, the cleanup is done through a periodic
poll (updateWorkerDrainMap).
The drain operation adds a record into the concurrent map [drainOpStatusMap]
when a worker is drained. An implicit assumption in the system is that the
drained worker will be removed from the system, soon, by an external
orchestrator. When the drained worker is seen to be removed from the system,
the drainOpStatusMap is cleaned up of stale information in the
updateWorkerDrainMap() code.
PLMK if I misunderstood the qs.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
