Dennis-Mircea Ciupitu created FLINK-40558:
---------------------------------------------
Summary: Apply in-place scaling only on SCALE spec diff type
Key: FLINK-40558
URL: https://issues.apache.org/jira/browse/FLINK-40558
Project: Flink
Issue Type: Improvement
Components: Autoscaler, Kubernetes Operator
Reporter: Dennis-Mircea Ciupitu
Currently, the reconciler’s in-place scaling can be reached by a non-SCALE spec
diff type. This case can only be met when the spec diff type is IGNORE, and the
reconciliation state is UPGRADING.
In reality this state is met when the status was patched and the operator
crashed before performing the deploy (for first deployment cases) or restoring
the job/cluster, by having at that time the deployment not deployed at all or
the job/cluster fully cancelled/deleted. For a job this re-enters as an UPGRADE
diff though, since its state is recorded as SUSPENDED before deploy, so it is
already excluded, and from these crash cases only the session cluster stays on
IGNORE.
Luckily, today, as mostly everything happens synchronously, the scale method
for both native and standalone both incidentally happen to return false in this
kind of case, because:
- In native mode, a session cluster has no job spec, so the in-place scaling
support check returns false and the scale method will not return a success.
- In standalone mode, there will be no TM deployment that is found and the
scale method will not return a success.
Still, there is a current gap is for the stateless suspension which happen
async on Flink cluster side, even tho the ack was send back sync to the
operator, and this can create a discrepancy by still keeping the job in a non
terminal state that can pass the scale checks on the next reconciliation pass,
depending on how fast this might happen.
The above scenario is more like an edge one, but still, by keeping the scale
logic called only for the SCALE diff type will keep both the code and logic
clean and will help us avoiding unnecessary situations.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
