mcvsubbu opened a new issue #3594: Graceful controller shutdown during LLC segment commits URL: https://github.com/apache/incubator-pinot/issues/3594 During metadata commit of LLC segment, it is required to do three zk operations * Update of metadata for completing segment * Creation of metadata for next segment of the stream partition * Update of Idealstate for the two segments from CONSUMING to ONLINE and OFFLINE to CONSUMING These cannot be done atomically, so it is possible that the controller restarts between these operations. Periodic run of ValidationManager fixes this situation and restarts consumption for LLC segments, but we want to minimize the chances that the ValidationManager has to repair the segment. Two things can be done for this: 1. Delay the controller stop operation until currently committing segments have done so (or, until a timeout) 2. Run validation manager soon after a new controller takes over.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
