HxpSerein commented on issue #3905: URL: https://github.com/apache/incubator-streampark/issues/3905#issuecomment-2257981544
Thanks @1996fanrui for the comment. >1.In the current stage, each server includes master and worker, right? Yes, we will not split the console at the first stage. >2.From your design, the registry center has 2 implementations: zookeeper and etcd. I'm curious could we use jdbc related database as the registry center? Such as MySQL, PostgreSQL. The specific implementation plan is still under discussion. In my opinion, both solutions have their own advantages and disadvantages. Using a single database can simplify system architecture and management, while using ZooKeeper or etcd has distributed consistency and high availability, and is suitable for dynamic configuration and service discovery. >3.Why do we need to introduce the `Job Distribution` or `Consistent Hash`? The server needs to poll jobs to obtain their latest status. When the number of jobs is huge, monitoring can become a significant challenge. Therefore, in my opinion, consistent hashing is primarily used to ensure load balancing in monitoring. In cases of cluster expansion or disaster, the consistent hashing algorithm can help migrate job monitoring. Please correct me if any understanding is wrong, thanks~ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
