[
https://issues.apache.org/jira/browse/GOBBLIN-780?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hung Tran updated GOBBLIN-780:
------------------------------
Summary: Handle scenarios that cause the YarnAutoScalingManager to be stuck
(was: Handle scenarios that causes the YarnAutoScalingManager to be stuck)
> Handle scenarios that cause the YarnAutoScalingManager to be stuck
> ------------------------------------------------------------------
>
> Key: GOBBLIN-780
> URL: https://issues.apache.org/jira/browse/GOBBLIN-780
> Project: Apache Gobblin
> Issue Type: Task
> Reporter: Hung Tran
> Priority: Major
>
> Issue 1: The YarnAutoScalingRunnable is run in a fixed schedule by a
> ScheduledExecutorService in YarnAutoScalingManager. If the runnable
> encounters an exception the the executor service will stop scheduling it.
> Catch all exceptions in the runnable, log, and do not re-raise.
> Issue 2: The auto scaler may reduce the container count to 0. Helix will not
> schedule any flows if there are no participants connected. This results in
> the auto scaler keeping the container count at 0 and no progress is made. Fix
> this by not allowing the container count to be reduced below 1.
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
