saurabhd336 commented on PR #9058: URL: https://github.com/apache/pinot/pull/9058#issuecomment-1196440928
@mcvsubbu 1) @npawar @Jackie-Jiang ? I might have just very rough, possibly inaccurate numbers. 2) I feel the need of a control plane level API within pinot to give an overall view into current and past state of minion tasks is of importance to us. Task generator being a key part of the entire minion task flow. While metrics can help to some extent, having details like failure stack traces etc might be difficult. This api avoids having to tally metrics and debug logs from a separate log processing system. 3) I suppose it could. But integrating the log processing framework into pinot APIs themselves might be a bit of a challenge. Having a system table for these kind of usecases might be the right way forward, such that pinot itself can store and serve debug data and status metrics for each of the components / flows. Essentially move from in-memory storage of logs and metrics into the system table @npawar @Jackie-Jiang to add more -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
