LuciferYang opened a new pull request, #10295: URL: https://github.com/apache/paimon/pull/10295
### Purpose Under unaligned checkpoints the barrier can overtake buffered `KEY_PART` records, so `GlobalIndexAssigner.prepareSnapshotPreBarrier` runs `endBoostrap` (setting `bootstrap=false` and nulling `bootstrapKeys`) while `KEY_PART` records are still queued. Those late records then reach `bootstrapKey` with `bootstrap == false`, where `checkArgument(inBoostrap())` throws (and the now-null `bootstrapKeys` would NPE), crashing the assigner and re-failing on restart in the same window. Aligned checkpoints never hit this, since barrier alignment processes every pre-barrier `KEY_PART` before `endBoostrap`. This registers a late bootstrap key directly into `keyIndex` instead of asserting `inBoostrap()`. It uses first-writer-wins (register only when the key is absent), so a late `KEY_PART` cannot overwrite an entry the running assigner already assigned to a live input row, which would otherwise leave `keyIndex` pointing at a bucket the data no longer lives in and misroute later same-key records into a cross-bucket duplicate primary key. The in-bootstrap buffering path and the aligned-checkpoint path are byte-for-byte unchanged. Limitation: under unaligned-checkpoint reordering, if a data row for the same key is processed before the late `KEY_PART`, the record in the previously-assigned partition is not retracted. Preserving that ordering fully is an operator/checkpoint-layer concern beyond this assigner; this change keeps `keyIndex` self-consistent and removes the crash. This closes #10294. ### Tests - `testLateBootstrapKeyAfterEndRegistersKey` pins that a `KEY_PART` after `endBoostrap` registers instead of crashing. - `testLateBootstrapKeyDoesNotOverwriteAssignedKey` pins first-writer-wins: a late `KEY_PART` does not overwrite an already-assigned key. - `testLateBootstrapKeyRegistersAndRetractsOldPartition` pins the cross-partition registration path. ### API and Format No. ### Documentation No. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
