[
https://issues.apache.org/jira/browse/FLINK-31529?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17702645#comment-17702645
]
Weihua Hu commented on FLINK-31529:
-----------------------------------
Thanks all for reply.
If introducing a new option is acceptable, I would like to take this issue.
In batch mode, the queue resources is insufficient in most case.
----
yes.
In detail, we have two kinds of YARN clusters: Streaming and Batch. The
Streaming cluster provide guaranteed resources to make all jobs long-running.
The Batch cluster does not guarantee that jobs could get resources Immediately,
these application will queued in YARN scheduler. This strategy maximizes
resource utilization.
In addition, we do have a platform to schedule flink batch jobs. These jobs
will run at hourly or daily levels.
> Let yarn client exit early before JobManager running
> ----------------------------------------------------
>
> Key: FLINK-31529
> URL: https://issues.apache.org/jira/browse/FLINK-31529
> Project: Flink
> Issue Type: Improvement
> Components: Deployment / YARN
> Reporter: Weihua Hu
> Priority: Major
>
> Currently the YarnClusterDescriptor always wait yarn application status to be
> RUNNING even if we use the detach mode.
> In batch mode, the queue resources is insufficient in most case. So the job
> manager may take a long time to wait resources. And client also keep waiting
> too. If flink client is killed(some other reason), the cluster will be
> shutdown too.
> We need an option to let Flink client exit early. Use the detach option or
> introduce a new option are both OK.
> Looking forward other suggestions
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
