xkrogen commented on PR #3976: URL: https://github.com/apache/hadoop/pull/3976#issuecomment-1097422287
Sorry for the delay in my response @hfutatzhanghb. I want to make sure I understand your proposal fully. I think what you are saying is this: Currently when using `ConfiguredFailoverProxyProvider`, we have a single config, `dfs.ha.namenodes.<nsid>`, where we list all NN addresses. Since we don't know the state of any of the addresses, we have to scan all of them to find the active. Instead, we could have two configs, e.g. `dfs.ha.namenodes.<nsid>.hanodes` and `dfs.ha.namenodes.<nsid>.observers`. The first lists NNs that could be either ACTIVE or STANDBY state. The second lists only NNs that could be OBSERVER state. (alternatively, `dfs.ha.namenodes.<nsid>` could remain the same and still contain all NNs, and we could just add the additional config `dfs.ha.namenodes.<nsid>.observers` to mark which of the list are observers, or even add a new per-NN config like `dfs.namenode.observer-only.<nsid>.<nnid>`) It's not possible to transition NNs in or out of OBSERVER state w/o changing the config on the client side to move NNs between the two lists. This is okay because we assume that we have enough NNs in the ACTIVE+STANDBY po ol to meet our HA needs, and OBSERVER NNs are only used for scaling reads, not providing HA. Do I have that right? If so, I think this is a reasonable proposal. I believe our installations at LinkedIn also assume that ONNs don't participate in SbNN/ANN failover transitions. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
