[
https://issues.apache.org/jira/browse/GOBBLIN-780?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hung Tran resolved GOBBLIN-780.
-------------------------------
Resolution: Fixed
Fix Version/s: 0.15.0
Issue resolved by pull request #2644
[https://github.com/apache/incubator-gobblin/pull/2644]
> Handle scenarios that cause the YarnAutoScalingManager to be stuck
> ------------------------------------------------------------------
>
> Key: GOBBLIN-780
> URL: https://issues.apache.org/jira/browse/GOBBLIN-780
> Project: Apache Gobblin
> Issue Type: Task
> Reporter: Hung Tran
> Priority: Major
> Fix For: 0.15.0
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Issue 1: The YarnAutoScalingRunnable is run in a fixed schedule by a
> ScheduledExecutorService in YarnAutoScalingManager. If the runnable
> encounters an exception the the executor service will stop scheduling it.
> Catch all exceptions in the runnable, log, and do not re-raise.
> Issue 2: The auto scaler may reduce the container count to 0. Helix will not
> schedule any flows if there are no participants connected. This results in
> the auto scaler keeping the container count at 0 and no progress is made. Fix
> this by not allowing the container count to be reduced below 1.
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
