[
https://issues.apache.org/jira/browse/FLINK-29464?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated FLINK-29464:
-----------------------------------
Labels: pull-request-available (was: )
> JobID generation logic could lead to state loss
> -----------------------------------------------
>
> Key: FLINK-29464
> URL: https://issues.apache.org/jira/browse/FLINK-29464
> Project: Flink
> Issue Type: Improvement
> Components: Kubernetes Operator
> Affects Versions: kubernetes-operator-1.2.0
> Reporter: Gyula Fora
> Assignee: Gyula Fora
> Priority: Blocker
> Labels: pull-request-available
> Fix For: kubernetes-operator-1.2.0
>
>
> The recently added job id override logic
> (https://issues.apache.org/jira/browse/FLINK-29109) can under certain cases
> lead to state loss.
> State loss scenario:
> 1. Either first deployment / Stateless upgrade mode used -> new jobId will be
> generated and set in jobStatus
> 2. Operator/deployment fails during or directly after successful submission
> -> status is not persisted with the generated jobId
> 3. User submits a spec update with last-state upgrade
> 4. If the job was never observed (due to a failure or early spec update) a
> last-state upgrade would be performed, deleting the Deployment and simply
> submitting the job.
> 5. The current logic would then generate a new jobid (because it's still
> empty) leading to a failure to recover the state from HA -> data loss
> There are multiple ways to solve this issue:
> a ) Record status after generating a jobid
> b ) Only ever set the status during stateless deployment
> c ) Verify no HA data is present before setting the jobid when empty
> Probably the most robust solution is a).
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
