homatthew commented on code in PR #3708:
URL: https://github.com/apache/gobblin/pull/3708#discussion_r1239159140
##########
gobblin-cluster/src/main/java/org/apache/gobblin/cluster/GobblinClusterManager.java:
##########
@@ -504,6 +506,18 @@ private static void printUsage(Options options) {
formatter.printHelp(GobblinClusterManager.class.getSimpleName(), options);
}
+ public void disableLiveHelixInstances() {
+ HelixManager helixManager = this.multiManager.getJobClusterHelixManager();
Review Comment:
Upon further investigation, this job cluster manager should be connected
elsewhere. But I see another issue where you are fetching all live instances
(include itself!). Which means that when the AM calls this method, it will
disable itself.
Please take a look at the below code
https://github.com/apache/gobblin/blob/5af6bca57df909e44b995e5b2d667c70e0399877/gobblin-yarn/src/main/java/org/apache/gobblin/yarn/YarnAutoScalingManager.java#L187C1-L196C6
This code fetches all live participants that are task runners. So we should
only disable those instances.
And then we can also rename this method to
`disableTaskRunnersFromPreviousExecutions`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
