Thanks Lijie/Biao
To put it simply, what we like to achieve is to replace env.runAsync() with
some code that uses application mode.
@Lijie: When I set deployment mode to application I got the following
exception after submitting the job
Caused by: java.lang.IllegalStateException: No ExecutorFactory found to
execute the application.
at
org.apache.flink.core.execution.DefaultExecutorServiceLoader.getExecutorFactory(DefaultExecutorServiceLoader.java:88)
at
org.apache.flink.streaming.api.environment.StreamExecutionEnvironment.executeAsync(StreamExecutionEnvironment.java:2036)
at
org.apache.flink.streaming.api.environment.StreamExecutionEnvironment.executeAsync(StreamExecutionEnvironment.java:2015)
@Biao: So the only way currently to use application mode is to package our
main method and execute it on JM? Can this be done with Java code, only
found examples using Flink Cli?
Thanks
Tamas
On Tue, Aug 2, 2022 at 4:41 AM Biao Geng wrote:
> Hi there,
> Currently, YARN Application mode only supports running a jar job. And as
> Lijie said, the main method is executed on JM, so if I understand
> correctly, your previous way of running execute() method on client side to
> submit a job to YARN cluster may not work. A quick workaround is to create
> such a job jar for submitting your SQL jobs and you can run this job jar
> using Application Mode. Thanks to Gyula's work, you can refer to
> https://github.com/apache/flink-kubernetes-operator/blob/main/examples/flink-sql-runner-example/src/main/java/org/apache/flink/examples/SqlRunner.java
> for
> more a minimal example.
>
> Best,
> Biao Geng
>
> Lijie Wang 于2022年8月2日周二 09:49写道：
>
>> Hi,
>> I think the difference between ApplicationMode and PerJob is just where
>> the main method is executed (ApplicationMode executes on JM, PerJob
>> executes on client side). So I think your original job code should work
>> well under ApplicationMode. Did you encounter any problems?
>> You can get more details about ApplicationMode in [1].
>>
>> [1]
>> https://nightlies.apache.org/flink/flink-docs-master/docs/deployment/resource-providers/yarn/#application-mode
>>
>> Best,
>> Lijie
>>
>> Tamas Kiss 于2022年8月1日周一 19:30写道：
>>
>>> Hi Experts,
>>>
>>> We are planning to migrate our flink jobs from per-job mode to
>>> application mode in our platform. Our jobs are basically SQL scripts so we
>>> have some custom Java code to leverage Flink's SQL and Table API to build
>>> the execution environment and execute the jobs on Yarn. We would like to
>>> keep the current flow in our platform so we are looking for a way to run
>>> Flink SQL in application mode via some Java code. Does this seem to be a
>>> good approach to follow or should we look for a better solution? If this is
>>> a good choice, is there any API to support this?
>>>
>>> Thanks
>>> Tamas
>>>
>>
