mcvsubbu edited a comment on issue #8339: URL: https://github.com/apache/pinot/issues/8339#issuecomment-1065548207
Overall +1 on improving the tool. I am not sure that adding a warning if > 500MB is the best option. In some cases, it is better that the segment is 1GB big. All these are arbitrary numbers anyway. Maybe we can compute the total number of segments and propose based on that? Alternatively, we output a link (within pinot docs, of course) on how to balance number of segments vs segment size vs number of servers vs query latency vs number of stream partitions, etc. Basically, describe that this a multi-variable equation and you can fix one but likely not control the others (e.g. if you fix latency, there may be limited things you can do to reduce cost) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
