mcvsubbu commented on issue #6854: URL: https://github.com/apache/pinot/issues/6854#issuecomment-970481060
@Jackie-Jiang what was the exact scenario when consumption got "stuck" ? As of now, it is the case that we retry a few times and if the stream throws exception, then we automatically mark the state as OFFLINE in idealstate. If all replicas are marked OFFLINE, then automatic recovery happens through periodic job. If some replicas are OFFLINE, then others are allowed to complete the segment, and eventually all replicas have a copy of he completed segment. By "stuck", did you mean bad data? The only way to recover from bad data as of now is to let the periodic job keep retrying until the bad offset is retained out of the underlying stream, and it will eventually pick an offset and continue consumption. A long time to wait in production use cases. Is this the scenario you were referring to? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
