KKcorps opened a new pull request, #19710: URL: https://github.com/apache/pinot/pull/19710
On partial upsert and out-of-order tables in `PROTECTED` mode, replacing a consuming segment with the downloaded committed copy reverts any key the replica consumed past the committed end offset back to its older segment. With `ALLOW_ALWAYS` or `ALLOW_DURING_DOWNLOAD_ONLY`, the next consuming segment starts before that download finishes and takes its validDocIds snapshot first. The revert then restores a row the snapshot already wrote as invalid, and nothing marks that older segment again, so a restart with preload drops the key on that replica. This keeps the consumer semaphore held during the download whenever the replace will revert. It uses the same `shouldRevertMetadataOnInconsistency` check the revert itself uses, now exposed on `PartitionUpsertMetadataManager` with a `false` default. Build overlap and every other table are unchanged. Operators should know that tables which get `ALLOW_ALWAYS` from `allowPartialUpsertConsumptionDuringCommit` now pause consumption during a download in `PROTECTED` mode. Tested with new unit tests in `RealtimeSegmentDataManagerTest` and `ConcurrentMapPartitionUpsertMetadataManagerTest`, and both classes pass in full. 🤖 Generated with [Claude Code](https://claude.com/claude-code) https://claude.ai/code/session_01W5J32DeWRrd7a1e3z6Y5bN -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
