npawar commented on PR #8740: URL: https://github.com/apache/pinot/pull/8740#issuecomment-1133226643
> Went over the code again, and we use 2 hard-coded times to control the consume loop: > > * idlePipeSleepTimeMillis (hard coded to 100ms): Sleep time when no new message is available from the stream > * maxIdleTime (hard coded to 3 minutes): If there is no new message for this period of time, recreate the consumer > > IMO the first time can still be hard coded, but we should make the second one configurable. Within the implementation, let's refactor it to be based on the system time instead of the idle count. Currently the idle count can be very wrong when the fetch timeout is set to be high. E.g. when the fetch timeout is set to 1 minutes, the idle count will be 2 (3 consecutive fetch try without new message will recreate the consumer...) I believe I've already done what you're suggesting. Can you take a look? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
