mxm commented on PR #711: URL: https://github.com/apache/flink-kubernetes-operator/pull/711#issuecomment-1810649406
>Are we talking about having a field in the ScalingExecutor? Because fetching scalingHistory won't be sufficient - we need some indication that a new rescaling was applied by the time we see the transition into RUNNING with the expected parallelism. This "flag" then needs to be cleaned. I meant a flag in the ConfigMap. I guess something like `startTime: endTime` would be sufficient? The endTime would initially be null. >General question: it feels like we are very focused on optimizing the size of this particular configmap. Can't we create a separate configmap, if this is a concern? In the past we ran into the 1MB size limit for large pipelines with hundreds of vertices. That's why we don't want to increase the state size too much. We also added compression because of this. Of course we can add a new configmap. We have just not come around to do it and it might not be a good idea to put too much load on etcd. While it would be nice to have a separate config map for the scaling metrics and the scaling history, but it is much simpler to do it in one. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
