mcvsubbu commented on issue #7652: URL: https://github.com/apache/pinot/issues/7652#issuecomment-958074211
> > Have you gone through https://docs.pinot.apache.org/developers/developers-and-contributors/contribution-guidelines ? > > I would like to see a detailed design doc on what problem is being solved, and what areas are covered (more than logs? Any link with metrics? etc.) Overall, my understanding is that you are trying to ease Pinot operation. Please correct me if I am wrong. > > thanks. > > sounds good. I will create one. thanks for the pointer and the review in advance. Thank you, I can review the design doc. Please be sure to include what kiind of events you want to capture, and how it is going to help in operation. We have been thinking about this problem now, and would really like to have some long-term (months/years) data and trends captured. As examples: - controllers emit some table characteristics (number of segments, avg/min/max/median segment size, etc.) - minions emit some events for each segment they process (table name, segmentname, num records remved, etc.) - merger emits some number of segments processed - Brokers emit the RequestStatistics (or a part of it) - Multi-tenant realtime installations can benefit from knowing the "load" on each realtime server over time (how many segments are being handled, overall consumption rates, query rates, etc.) A rebalance command can analyze past trends to try to rebalance correctly, for example -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
