kfaraz commented on PR #15035: URL: https://github.com/apache/druid/pull/15035#issuecomment-1736613576
I am not entirely sure we need a config here either. Irrespective of what we name the config, it is going to cause confusion with the existing `timeoutMs`, especially since the `SamplerConfig` is used with batch too where this config would not be meaningful. I think we could just have a __fixed__ maximum number of empty polls. When we poll the topic for records, and say we don't get anything for 10 tries or say, we just wrap up the sampler. Since this is only used for sampling, there is no real obligation to keep waiting until we have read the requested number of records. Compare this to batch where if there is no file at the specified path, we just fail or return empty and don't wait for data to show up. @zachjsh , @abhishekagarwal87 , what do you think? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
