sajjad-moradi commented on pull request #7267: URL: https://github.com/apache/pinot/pull/7267#issuecomment-895627352
> What's the scenario where this feature will be useful? > > Any drawbacks? Looks like this will make entire server not accessible during catch-up instead of just one segment. Is my understanding correct? Here's the existing behavior: if the value for config `pinot.server.starter.realtimeConsumptionCatchupWaitMs` is greater than zero (zero is the default value), the pinot-server just waits for the period specified in the config to allow ALL consumers to catch up. This requires some knowledge on how much time is good for different use cases to only consume at startup. Keep in mind that consumption might change over time and then we need to adjust the config value. The new status checker still uses that startup-consumption-timeout as the upper bound, but in each status check interval, which is 10s, it checks if all consuming segments have reached their latest stream offset. Just FYI we have 2 minutes timeout (10m for high rate) in our prod use cases. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
