Santwana Verma created FLINK-41283:
--------------------------------------
Summary: Last-state upgrade fails when checkpointing is disabled
and max checkpoint age is set
Key: FLINK-41283
URL: https://issues.apache.org/jira/browse/FLINK-41283
Project: Flink
Issue Type: Bug
Components: Kubernetes Operator
Reporter: Santwana Verma
If a job doesn't have checkpointing enabled and
kubernetes.operator.job.upgrade.last-state.max.allowed.checkpoint.age is set,
last-state upgrades get stuck once the job has been running longer than the max
age.
To decide whether the latest checkpoint is recent enough, the operator asks the
JobManager for checkpoint statistics. For a job without checkpointing, the
JobManager returns a 404, "Checkpointing has not been enabled."
AbstractJobReconciler.getUpgradeModeBasedOnStateAge doesn't handle that, so the
reconcile fails and the upgrade never happens. Every following reconcile fails
the same way.
We already treat this 404 as "no checkpoint" when looking up the latest
checkpoint, and FLINK-41267 does the same for the cluster info. The upgrade
path should handle it too.
I'm not sure what the upgrade should do in this case. One option is to fall
back to a savepoint, like we do when the latest checkpoint is too old. That
doesn't work for batch jobs, though. The other is to go ahead with the
last-state upgrade, since there's no checkpointed state to lose.
To reproduce: deploy a last-state job without checkpointing and with the max
age set to 1m. Wait a couple of minutes, then change the parallelism. The
upgrade doesn't go through, and the operator logs RestClientException:
[Checkpointing has not been enabled.] on every reconcile.
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
