[
https://issues.apache.org/jira/browse/FLINK-8137?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Joshua Griffith updated FLINK-8137:
-----------------------------------
Description:
When submitting a new {{FlinkPlan}} using the {{StandaloneClusterClient}}, the
JobManager's REST API appears to hang until the job is submitted. The
submission time may be large enough to cause timeouts if the {{getStatistics}}
and {{createInputSplits}} methods of a job's input formats perform
time-intensive tasks like running external queries. This is exacerbated when a
job contains many such input formats since they appear to be initialized
sequentially. For a particular job with over 100 inputs, it's typical for the
API (and consequently the web UI) to be non-responsive for 45–60 seconds.
Would it make sense for tasks to have a {{Configuring}} state before the
{{Created}} state to provide greater visibility and indicate that the
JobManager is still healthy?
was:
When submitting a new {{FlinkPlan}} using the {{StandaloneClusterClient}}, the
JobManager's REST API appears to hang until the job is submitted. The
submission time may be large enough to cause timeouts if the {{getStatistics}}
and {{createInputSplits}} methods of a job's {{RichInputFormat}} s perform
time-intensive tasks like running external queries. This is exacerbated when a
job contains many such input formats since they appear to be initialized
sequentially. For a particular job with over 100 inputs, it's typical for the
API (and consequently the web UI) to be non-responsive for 45–60 seconds.
Would it make sense for tasks to have a {{Configuring}} state before the
{{Created}} state to provide greater visibility and indicate that the
JobManager is still healthy?
> Flink JobManager API non-responsive during job submission
> ---------------------------------------------------------
>
> Key: FLINK-8137
> URL: https://issues.apache.org/jira/browse/FLINK-8137
> Project: Flink
> Issue Type: Bug
> Components: Client, Job-Submission, JobManager, REST, Webfrontend
> Affects Versions: 1.3.2
> Environment: Flink 1.3.2 running a batch job in Kubernetes.
> Reporter: Joshua Griffith
> Priority: Minor
>
> When submitting a new {{FlinkPlan}} using the {{StandaloneClusterClient}},
> the JobManager's REST API appears to hang until the job is submitted. The
> submission time may be large enough to cause timeouts if the
> {{getStatistics}} and {{createInputSplits}} methods of a job's input formats
> perform time-intensive tasks like running external queries. This is
> exacerbated when a job contains many such input formats since they appear to
> be initialized sequentially. For a particular job with over 100 inputs, it's
> typical for the API (and consequently the web UI) to be non-responsive for
> 45–60 seconds.
> Would it make sense for tasks to have a {{Configuring}} state before the
> {{Created}} state to provide greater visibility and indicate that the
> JobManager is still healthy?
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
