Ngone51 commented on issue #24569: [SPARK-23191][CORE] Warn rather than terminate when duplicate worker register happens URL: https://github.com/apache/spark/pull/24569#issuecomment-492291920 > If we rely on zoom keeper to do leader election, why don't the workers ask zoom keeper for the leader info? This may be another implementation strategy. But if we ask workers to directly ask zk(zookeeper) for leader info, will there be too many zk clients concurrently if we have plenty of workers ? And it may pay pressure on network. (And not sure zk provides the api for leader info directly.) > Reconnecting to all masters looks wrong to me, as it's likely that more than one master thinks himself as active (the communication between zoom keeper and masters has delays). hmmm...actually, it seems to me that it is worker rather than masters thinks that more than one master is active. However, masters themselves know whether itself is active well(though, has a little delays).
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
