[
https://issues.apache.org/jira/browse/GOBBLIN-1841?focusedWorklogId=867042&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-867042
]
ASF GitHub Bot logged work on GOBBLIN-1841:
-------------------------------------------
Author: ASF GitHub Bot
Created on: 23/Jun/23 00:57
Start Date: 23/Jun/23 00:57
Worklog Time Spent: 10m
Work Description: homatthew commented on code in PR #3708:
URL: https://github.com/apache/gobblin/pull/3708#discussion_r1239159140
##########
gobblin-cluster/src/main/java/org/apache/gobblin/cluster/GobblinClusterManager.java:
##########
@@ -504,6 +506,18 @@ private static void printUsage(Options options) {
formatter.printHelp(GobblinClusterManager.class.getSimpleName(), options);
}
+ public void disableLiveHelixInstances() {
+ HelixManager helixManager = this.multiManager.getJobClusterHelixManager();
Review Comment:
Upon further investigation, this job cluster manager should be connected
elsewhere. But I see another issue where you are fetching all live instances
(include itself!). Which means that when the AM calls this method, it will
disable itself.
Please take a look at the below code
https://github.com/apache/gobblin/blob/5af6bca57df909e44b995e5b2d667c70e0399877/gobblin-yarn/src/main/java/org/apache/gobblin/yarn/YarnAutoScalingManager.java#L187C1-L196C6
This code fetches all live participants that are task runners. We should
only disable taskrunners, so let's do that instead.
And then we can also rename this method to
`disableTaskRunnersFromPreviousExecutions`.
Issue Time Tracking
-------------------
Worklog Id: (was: 867042)
Time Spent: 50m (was: 40m)
> Move disabling of current live instances to the GobblinClusterManager startup
> -----------------------------------------------------------------------------
>
> Key: GOBBLIN-1841
> URL: https://issues.apache.org/jira/browse/GOBBLIN-1841
> Project: Apache Gobblin
> Issue Type: Improvement
> Reporter: Matthew Ho
> Priority: Major
> Time Spent: 50m
> Remaining Estimate: 0h
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
