Re: K8s operator - Stop Job with savepoint on session cluster via Java API
Hi Krzysztof,
Please set upgradeMode to savepoint and change state from Running to
Suspended on your application. This makes it so that you trigger an upgrade
(as at least the job state changes) and for the upgrade we explicitly
trigger a savepoint as you choose that for the upgrade mode.
Importantly you do not need to use savepointTriggerNonce for this mechanism
at all, also whether you run in a session mode or application mode is of
secondary concern.
Savepoints triggered by default are canonical, that is provided by this
mechanism. The configurations for the savepoint are taken from the Flink
configuration an the application level (potentially inherited by
operator/cluster level defaults), you cannot pass them in the spec as of
today.
Best,
Marton
On Fri, Sep 1, 2023 at 9:31 AM Krzysztof Chmielewski <
krzysiek.chmielew...@gmail.com> wrote:
> Hi thanks,
> However what you have send me is sql client. I'm looking for a way to do
> it via k8s operator's java Api.
>
> pt., 1 wrz 2023, 03:58 użytkownik Shammon FY napisał:
>
>> Hi Krzysztof,
>>
>> For the flink session cluster, you can stop the job with savepoint
>> through the statement `STOP JOB '{Your job id}' WITH SAVEPOINT;`. You can
>> refer to [1] for more information about how to do it in sql client and you
>> can also create a table environment to perform the statement in your
>> application.
>>
>> [1]
>> https://nightlies.apache.org/flink/flink-docs-master/docs/dev/table/sqlclient/#terminating-a-job
>>
>> Best,
>> Shammon FY
>>
>> On Fri, Sep 1, 2023 at 6:35 AM Krzysztof Chmielewski <
>> krzysiek.chmielew...@gmail.com> wrote:
>>
>>> Hi community,
>>> I would like to ask what is the recommended way to stop Flink job with
>>> save point on a session cluster via k8s operator Java API?
>>>
>>> Currently I'm doing this by setting savepointTriggerNonce on JobSpec
>>> object.
>>> However I've noticed that this works only if I do not include Job state
>>> change in that spec.
>>>
>>> In other words when I submit JobSpec that has state change from Running
>>> to Suspend and savepointTriggerNonce, the checkpoint is not created. Is
>>> that intended?
>>> In order to mimic [1] do I have to submit two JobSpec updates? One with
>>> savepointNonce and the second one with Job state change?
>>>
>>> A followup question, what kind of savepoint is triggered when using
>>> savepointTriggerNonce native or canonical? Also is there a way to pass
>>> --drain option or savepoint path via spec? (Not
>>> including state.savepoints.dir cluster config option)
>>>
>>> [1]
>>> https://nightlies.apache.org/flink/flink-docs-master/docs/ops/state/savepoints/#stopping-a-job-with-savepoint
>>>
>>> Thanks,
>>> Krzysztof Chmielewski
>>>
>>
Re: K8s operator - Stop Job with savepoint on session cluster via Java API
Hi thanks,
However what you have send me is sql client. I'm looking for a way to do it
via k8s operator's java Api.
pt., 1 wrz 2023, 03:58 użytkownik Shammon FY napisał:
> Hi Krzysztof,
>
> For the flink session cluster, you can stop the job with savepoint through
> the statement `STOP JOB '{Your job id}' WITH SAVEPOINT;`. You can refer to
> [1] for more information about how to do it in sql client and you can also
> create a table environment to perform the statement in your application.
>
> [1]
> https://nightlies.apache.org/flink/flink-docs-master/docs/dev/table/sqlclient/#terminating-a-job
>
> Best,
> Shammon FY
>
> On Fri, Sep 1, 2023 at 6:35 AM Krzysztof Chmielewski <
> krzysiek.chmielew...@gmail.com> wrote:
>
>> Hi community,
>> I would like to ask what is the recommended way to stop Flink job with
>> save point on a session cluster via k8s operator Java API?
>>
>> Currently I'm doing this by setting savepointTriggerNonce on JobSpec
>> object.
>> However I've noticed that this works only if I do not include Job state
>> change in that spec.
>>
>> In other words when I submit JobSpec that has state change from Running
>> to Suspend and savepointTriggerNonce, the checkpoint is not created. Is
>> that intended?
>> In order to mimic [1] do I have to submit two JobSpec updates? One with
>> savepointNonce and the second one with Job state change?
>>
>> A followup question, what kind of savepoint is triggered when using
>> savepointTriggerNonce native or canonical? Also is there a way to pass
>> --drain option or savepoint path via spec? (Not
>> including state.savepoints.dir cluster config option)
>>
>> [1]
>> https://nightlies.apache.org/flink/flink-docs-master/docs/ops/state/savepoints/#stopping-a-job-with-savepoint
>>
>> Thanks,
>> Krzysztof Chmielewski
>>
>
Re: K8s operator - Stop Job with savepoint on session cluster via Java API
Hi Krzysztof,
For the flink session cluster, you can stop the job with savepoint through
the statement `STOP JOB '{Your job id}' WITH SAVEPOINT;`. You can refer to
[1] for more information about how to do it in sql client and you can also
create a table environment to perform the statement in your application.
[1]
https://nightlies.apache.org/flink/flink-docs-master/docs/dev/table/sqlclient/#terminating-a-job
Best,
Shammon FY
On Fri, Sep 1, 2023 at 6:35 AM Krzysztof Chmielewski <
krzysiek.chmielew...@gmail.com> wrote:
> Hi community,
> I would like to ask what is the recommended way to stop Flink job with
> save point on a session cluster via k8s operator Java API?
>
> Currently I'm doing this by setting savepointTriggerNonce on JobSpec
> object.
> However I've noticed that this works only if I do not include Job state
> change in that spec.
>
> In other words when I submit JobSpec that has state change from Running to
> Suspend and savepointTriggerNonce, the checkpoint is not created. Is that
> intended?
> In order to mimic [1] do I have to submit two JobSpec updates? One with
> savepointNonce and the second one with Job state change?
>
> A followup question, what kind of savepoint is triggered when using
> savepointTriggerNonce native or canonical? Also is there a way to pass
> --drain option or savepoint path via spec? (Not
> including state.savepoints.dir cluster config option)
>
> [1]
> https://nightlies.apache.org/flink/flink-docs-master/docs/ops/state/savepoints/#stopping-a-job-with-savepoint
>
> Thanks,
> Krzysztof Chmielewski
>
K8s operator - Stop Job with savepoint on session cluster via Java API
Hi community, I would like to ask what is the recommended way to stop Flink job with save point on a session cluster via k8s operator Java API? Currently I'm doing this by setting savepointTriggerNonce on JobSpec object. However I've noticed that this works only if I do not include Job state change in that spec. In other words when I submit JobSpec that has state change from Running to Suspend and savepointTriggerNonce, the checkpoint is not created. Is that intended? In order to mimic [1] do I have to submit two JobSpec updates? One with savepointNonce and the second one with Job state change? A followup question, what kind of savepoint is triggered when using savepointTriggerNonce native or canonical? Also is there a way to pass --drain option or savepoint path via spec? (Not including state.savepoints.dir cluster config option) [1] https://nightlies.apache.org/flink/flink-docs-master/docs/ops/state/savepoints/#stopping-a-job-with-savepoint Thanks, Krzysztof Chmielewski
