Thanks Matthias for preparing this thorough FLIP, which has taken us reviewing the multiple component leader election.
I am totally in favor of doing the code clean-up. The current implementation does not have very good readability due to legacy compatibility. And I just have a few comments. # HA data format in the HA backend(e.g. ZK, K8s ConfigMap) We have already changed the HA data format after introducing the multiple component leader election in FLINK-24038. For K8s HA, the num of ConfigMaps reduced from 4 to 2. Since we only have one leader elector, the K8s APIServer load should also be reduced. Why do we still need to change the format again? This just prevents the LAST_STATE upgrade mode in Flink-Kubernetes-Operator when the Flink version changed, even though it is a simple job and state is compatible. # LeaderContender#initiateLeaderElection I do not get your point why we need the *initiateLeaderElection* in *LeaderContender*. AFAICS, the callback *onGrant/RevokeLeadership* could be executed as soon as the registration. # When to establish the HA backend connection +1 for establish the connection beforehand Best, Yang Matthias Pohl <matthias.p...@aiven.io.invalid> 于2023年1月5日周四 21:51写道： > Hi everyone, > I brought up FLINK-26522 [1] in the mailing list discussion about > consolidating the HighAvailabilityServices interfaces [2], previously. > There, it was concluded that the community still wants the ability to have > per-component leader election and, therefore, keep the > HighAvailabilityServices interface as is. I went back to work on > FLINK-26522 [1] to figure out how we can simplify the current codebase > keeping the decision in mind. > > I wanted to handle FLINK-26522 [1] as a follow-up cleanup task of > FLINK-24038 [3]. But while working on it, I realized that even FLINK-24038 > [3] shouldn't have been handled without a FLIP. The per-process leader > election which was introduced in FLINK-24038 [3] changed the ownership of > certain components. This is actually a change that should have been > discussed in the mailing list and deserved a FLIP. To overcome this > shortcoming of FLINK-24038 [3], I decided to prepare FLIP-285 [4] to > provide proper documentation of what happened in FLINK-24038 and what will > be manifested with resolving its follow-up FLINK-26522 [1]. > > Conceptually, this FLIP proposes moving away from Flink's support for > single-contender LeaderElectionServices and introducing multi-contender > support by disconnecting the HA-backend leader election lifecycle from the > LeaderContender's lifecycle. This allows us to provide LeaderElection per > component (as it was requested in [2]) but also enables us to utilize a > single leader election for multiple components/contenders as well without > the complexity of the code that was introduced by FLINK-24038 [3]. > > I'm looking forward to your comments. > > Matthias > > [1] https://issues.apache.org/jira/browse/FLINK-26522 > [2] https://lists.apache.org/thread/9oy2ml9s3j1v6r77h31sndhc3gw57cfm > [3] https://issues.apache.org/jira/browse/FLINK-24038 > [4] > > https://cwiki.apache.org/confluence/display/FLINK/FLIP-285%3A+Refactoring+LeaderElection+to+make+Flink+support+multi-component+leader+election+out-of-the-box >
