gyfora opened a new pull request, #633: URL: https://github.com/apache/flink-kubernetes-operator/pull/633
## What is the purpose of the change This PR contains 3 main parts: 1. Carry over parallelism overrides between spec changes For this we removed the logic of directly modifying the `Spec` from the autoscaler and instead extended the JobAutoscaler interface to expose the current parallelism overrides for a given resource. The core reconciler logic will then always pick up the actual overrides and apply it to the user provided spec. Overrides provided by the autoscaler are removed whenever autoscaling is disabled for the resource (this should clear the overrides for the future), allowing users to control the behaviour if necessary. As we do not rely on spec changes anymore, there is now a mechanism to trigger immediate new reconciliation ones the new parallelism overrides are available. 2. Required hardening of the scale up/override logic for 1.18 Due to 1. we will now commonly have cases where overrides are provided for non-active job vertices (that are no longer in the jobgraph). Previously these caused different errors in the logic, now we improved the scaling and observe logic to basically ignore any override / scaling request for non-active vertices. 3. Cache AutoscalerInfo (https://issues.apache.org/jira/browse/FLINK-32615) With more frequent access to the autoscalerinfo it is even more important to do this efficiently. This is a standalone improvement that could be added as a second PR if prefered . ## Verifying this change New unit tests have been added to all parts of the changed behaviour and interfaces. It was also tested in local and cloud environments. ## Does this pull request potentially affect one of the following parts: - Dependencies (does it add or upgrade a dependency): no - The public API, i.e., is any changes to the `CustomResourceDescriptors`: no - Core observer or reconciler logic that is regularly executed: yes ## Documentation - Does this pull request introduce a new feature? no - If yes, how is the feature documented? not applicable -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
