Jackie-Jiang commented on issue #6679: URL: https://github.com/apache/incubator-pinot/issues/6679#issuecomment-876650388
I feel this is way too complicated, and we should break it down to several primitive operations. Since the consuming segment can only be removed manually, and the controller won't automatically recreate it, the race condition will be easy to address: a version check when updating the ideal state is good enough. The first api is the same as I suggested above. We can provide another api to read the latest offset of each partition (all partitions if partition is not specified) so that the admin knows the offset to use if they want to skip some offsets. I don't think earliest available offset is very useful here because that can change. In order to use the earliest offset, user should delete all the completed segments for the partition, and run the first api, which should pick the earliest available offset when there is no completed segment. To summarize, automatic fix always pick the latest offset if there are completed segments, or earliest available offset if not. Manual fix requires the users to explicitly put an offset, and can only work on single partition. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
