Re: Flink run different jars

2021-09-29 Thread Qihua Yang
Thanks a lot Yangze. That is very helpful!

On Tue, Sep 28, 2021 at 11:11 PM Yangze Guo  wrote:

> You need to edit the conf/workers. Example of the config[1] and the
> process[2].
>
> [1]
> https://ci.apache.org/projects/flink/flink-docs-master/docs/deployment/resource-providers/standalone/overview/#starting-and-stopping-a-cluster
> [2]
> https://ci.apache.org/projects/flink/flink-docs-master/docs/deployment/resource-providers/standalone/overview/#starting-a-standalone-cluster-session-mode
>
> Best,
> Yangze Guo
>
> On Wed, Sep 29, 2021 at 1:02 PM Qihua Yang  wrote:
> >
> > Hi Yangze,
> >
> > Thanks a lot for your reply. References are very helpful!
> > Another quick question. Reference 1 can start a standalone cluster
> (session Mode). That cluster has a jobManager. I can submit job to run. How
> about taskManger? Do I need to manually start multiple taskManagers?
> > Is there a complete example to show the process?
> >
> > Thanks,
> > Qihua
> >
> >
> > On Tue, Sep 28, 2021 at 7:02 PM Yangze Guo  wrote:
> >>
> >> Hi, Qihua
> >>
> >> IIUC, what you want might be a standalone cluster[1] or session
> cluster[2][3].
> >>
> >> [1]
> https://ci.apache.org/projects/flink/flink-docs-master/docs/deployment/resource-providers/standalone/overview/
> >> [2]
> https://ci.apache.org/projects/flink/flink-docs-master/docs/deployment/resource-providers/native_kubernetes/#session-mode
> >> [3]
> https://ci.apache.org/projects/flink/flink-docs-master/docs/deployment/resource-providers/yarn/#session-mode
> >>
> >> Best,
> >> Yangze Guo
> >>
> >> On Wed, Sep 29, 2021 at 5:57 AM Qihua Yang  wrote:
> >> >
> >> > Hi,
> >> >
> >> > Is that possible to run a flink app without a job? What I am trying
> to do is I build multiple jars. And switch jar to run different jobs.
> >> > I am not sure if flink supports this mode. I saw rest API can upload
> jar, cancel job and run a jar.
> >> > Right now I can upload a jar to flink. But when I cancel a job, flink
> will restart automatically. I checked log. It show below logs. Can anyone
> help me out?
> >> >
> >> > Caused by:
> org.apache.flink.client.deployment.application.UnsuccessfulExecutionException:
> Application Status: CANCELED
> >> > at
> org.apache.flink.client.deployment.application.UnsuccessfulExecutionException.fromJobResult(UnsuccessfulExecutionException.java:73)
> >> > ... 41 common frames omitted
> >> > Caused by: org.apache.flink.runtime.client.JobCancellationException:
> Job was cancelled.
> >> > at
> org.apache.flink.runtime.jobmaster.JobResult.toJobExecutionResult(JobResult.java:149)
> >> > at
> org.apache.flink.client.deployment.application.UnsuccessfulExecutionException.fromJobResult(UnsuccessfulExecutionException.java:64)
> >> > ... 41 common frames omitted
> >> >
> >> > Thanks!
>


Re: Flink run different jars

2021-09-28 Thread Yangze Guo
You need to edit the conf/workers. Example of the config[1] and the process[2].

[1] 
https://ci.apache.org/projects/flink/flink-docs-master/docs/deployment/resource-providers/standalone/overview/#starting-and-stopping-a-cluster
[2] 
https://ci.apache.org/projects/flink/flink-docs-master/docs/deployment/resource-providers/standalone/overview/#starting-a-standalone-cluster-session-mode

Best,
Yangze Guo

On Wed, Sep 29, 2021 at 1:02 PM Qihua Yang  wrote:
>
> Hi Yangze,
>
> Thanks a lot for your reply. References are very helpful!
> Another quick question. Reference 1 can start a standalone cluster (session 
> Mode). That cluster has a jobManager. I can submit job to run. How about 
> taskManger? Do I need to manually start multiple taskManagers?
> Is there a complete example to show the process?
>
> Thanks,
> Qihua
>
>
> On Tue, Sep 28, 2021 at 7:02 PM Yangze Guo  wrote:
>>
>> Hi, Qihua
>>
>> IIUC, what you want might be a standalone cluster[1] or session 
>> cluster[2][3].
>>
>> [1] 
>> https://ci.apache.org/projects/flink/flink-docs-master/docs/deployment/resource-providers/standalone/overview/
>> [2] 
>> https://ci.apache.org/projects/flink/flink-docs-master/docs/deployment/resource-providers/native_kubernetes/#session-mode
>> [3] 
>> https://ci.apache.org/projects/flink/flink-docs-master/docs/deployment/resource-providers/yarn/#session-mode
>>
>> Best,
>> Yangze Guo
>>
>> On Wed, Sep 29, 2021 at 5:57 AM Qihua Yang  wrote:
>> >
>> > Hi,
>> >
>> > Is that possible to run a flink app without a job? What I am trying to do 
>> > is I build multiple jars. And switch jar to run different jobs.
>> > I am not sure if flink supports this mode. I saw rest API can upload jar, 
>> > cancel job and run a jar.
>> > Right now I can upload a jar to flink. But when I cancel a job, flink will 
>> > restart automatically. I checked log. It show below logs. Can anyone help 
>> > me out?
>> >
>> > Caused by: 
>> > org.apache.flink.client.deployment.application.UnsuccessfulExecutionException:
>> >  Application Status: CANCELED
>> > at 
>> > org.apache.flink.client.deployment.application.UnsuccessfulExecutionException.fromJobResult(UnsuccessfulExecutionException.java:73)
>> > ... 41 common frames omitted
>> > Caused by: org.apache.flink.runtime.client.JobCancellationException: Job 
>> > was cancelled.
>> > at 
>> > org.apache.flink.runtime.jobmaster.JobResult.toJobExecutionResult(JobResult.java:149)
>> > at 
>> > org.apache.flink.client.deployment.application.UnsuccessfulExecutionException.fromJobResult(UnsuccessfulExecutionException.java:64)
>> > ... 41 common frames omitted
>> >
>> > Thanks!


Re: Flink run different jars

2021-09-28 Thread Qihua Yang
Hi Yangze,

Thanks a lot for your reply. References are very helpful!
Another quick question. Reference 1 can start a standalone cluster (session
Mode). That cluster has a jobManager. I can submit job to run. How about
taskManger? Do I need to manually start multiple taskManagers?
Is there a complete example to show the process?

Thanks,
Qihua


On Tue, Sep 28, 2021 at 7:02 PM Yangze Guo  wrote:

> Hi, Qihua
>
> IIUC, what you want might be a standalone cluster[1] or session
> cluster[2][3].
>
> [1]
> https://ci.apache.org/projects/flink/flink-docs-master/docs/deployment/resource-providers/standalone/overview/
> [2]
> https://ci.apache.org/projects/flink/flink-docs-master/docs/deployment/resource-providers/native_kubernetes/#session-mode
> [3]
> https://ci.apache.org/projects/flink/flink-docs-master/docs/deployment/resource-providers/yarn/#session-mode
>
> Best,
> Yangze Guo
>
> On Wed, Sep 29, 2021 at 5:57 AM Qihua Yang  wrote:
> >
> > Hi,
> >
> > Is that possible to run a flink app without a job? What I am trying to
> do is I build multiple jars. And switch jar to run different jobs.
> > I am not sure if flink supports this mode. I saw rest API can upload
> jar, cancel job and run a jar.
> > Right now I can upload a jar to flink. But when I cancel a job, flink
> will restart automatically. I checked log. It show below logs. Can anyone
> help me out?
> >
> > Caused by:
> org.apache.flink.client.deployment.application.UnsuccessfulExecutionException:
> Application Status: CANCELED
> > at
> org.apache.flink.client.deployment.application.UnsuccessfulExecutionException.fromJobResult(UnsuccessfulExecutionException.java:73)
> > ... 41 common frames omitted
> > Caused by: org.apache.flink.runtime.client.JobCancellationException: Job
> was cancelled.
> > at
> org.apache.flink.runtime.jobmaster.JobResult.toJobExecutionResult(JobResult.java:149)
> > at
> org.apache.flink.client.deployment.application.UnsuccessfulExecutionException.fromJobResult(UnsuccessfulExecutionException.java:64)
> > ... 41 common frames omitted
> >
> > Thanks!
>


Re: Flink run different jars

2021-09-28 Thread Yangze Guo
Hi, Qihua

IIUC, what you want might be a standalone cluster[1] or session cluster[2][3].

[1] 
https://ci.apache.org/projects/flink/flink-docs-master/docs/deployment/resource-providers/standalone/overview/
[2] 
https://ci.apache.org/projects/flink/flink-docs-master/docs/deployment/resource-providers/native_kubernetes/#session-mode
[3] 
https://ci.apache.org/projects/flink/flink-docs-master/docs/deployment/resource-providers/yarn/#session-mode

Best,
Yangze Guo

On Wed, Sep 29, 2021 at 5:57 AM Qihua Yang  wrote:
>
> Hi,
>
> Is that possible to run a flink app without a job? What I am trying to do is 
> I build multiple jars. And switch jar to run different jobs.
> I am not sure if flink supports this mode. I saw rest API can upload jar, 
> cancel job and run a jar.
> Right now I can upload a jar to flink. But when I cancel a job, flink will 
> restart automatically. I checked log. It show below logs. Can anyone help me 
> out?
>
> Caused by: 
> org.apache.flink.client.deployment.application.UnsuccessfulExecutionException:
>  Application Status: CANCELED
> at 
> org.apache.flink.client.deployment.application.UnsuccessfulExecutionException.fromJobResult(UnsuccessfulExecutionException.java:73)
> ... 41 common frames omitted
> Caused by: org.apache.flink.runtime.client.JobCancellationException: Job was 
> cancelled.
> at 
> org.apache.flink.runtime.jobmaster.JobResult.toJobExecutionResult(JobResult.java:149)
> at 
> org.apache.flink.client.deployment.application.UnsuccessfulExecutionException.fromJobResult(UnsuccessfulExecutionException.java:64)
> ... 41 common frames omitted
>
> Thanks!


Flink run different jars

2021-09-28 Thread Qihua Yang
Hi,

Is that possible to run a flink app without a job? What I am trying to do
is I build multiple jars. And switch jar to run different jobs.
I am not sure if flink supports this mode. I saw rest API can upload jar,
cancel job and run a jar.
Right now I can upload a jar to flink. But when I cancel a job, flink will
restart automatically. I checked log. It show below logs. Can anyone help
me out?

Caused by:
org.apache.flink.client.deployment.application.UnsuccessfulExecutionException:
Application Status: CANCELED
at
org.apache.flink.client.deployment.application.UnsuccessfulExecutionException.fromJobResult(UnsuccessfulExecutionException.java:73)
... 41 common frames omitted
Caused by: org.apache.flink.runtime.client.JobCancellationException: Job
was cancelled.
at
org.apache.flink.runtime.jobmaster.JobResult.toJobExecutionResult(JobResult.java:149)
at
org.apache.flink.client.deployment.application.UnsuccessfulExecutionException.fromJobResult(UnsuccessfulExecutionException.java:64)
... 41 common frames omitted

Thanks!