Kalvin2077 commented on PR #3858: URL: https://github.com/apache/celeborn/pull/3858#issuecomment-5811465852
> @Kalvin2077, I do not agree with the current implementation. Have you verified the heartbeat load caused by adding worker metrics in a large-scale celeborn cluster? Meanwhile, the details of this FLIP have not yet been fully aligned. @SteNicholas Thanks for the feedback. You're right. The current implementation introduces a typed WorkerStats schema, while CIP-13 still proposes a simple stats map inside WorkerStatus. I should have aligned the schema on the CIP thread before implementing it. To make sure I address the right risk: is the concern mainly the payload size of the typed schema, or the broader master-side and aggregate heartbeat cost at scale? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
