nielifeng commented on PR #11465: URL: https://github.com/apache/seatunnel/pull/11465#issuecomment-4998765383
Thanks @DanielLeens, you're right — the content-based tie-break broke same-millisecond replay semantics. Fixed in `6811bc3d8`. **What changed** - Added a monotonic `sequenceId` to the WAL record, stamped in `WALWriter.write` (the single disruptor consumer thread), so it reflects true append order into the WAL file. - The comparator now uses `sequenceId` as the *primary* equal-timestamp tie-break: the later append gets the larger id and sorts first, so `WALReader`'s keep-first dedup keeps the actual last write. This is correct for same-key same-ms `put→delete`, `put→put`, and `delete→put`. - Two records sharing a millisecond timestamp were necessarily written by the same `WALWriter` instance (file rotation and restart both take far longer than 1ms), so the sequence is consistent for exactly the records that need it. - Backward compatibility: `sequenceId` is declared last, and protostuff-runtime 1.8.0 assigns field tags by declaration order, so existing WAL files stay deserializable and read back as `0`. Zero-sequence (legacy) records fall through to the original stable content-based tie-breaks, so the `Comparable` contract still holds for old data. **Tests** - Added `WALReaderAndWriterTest.testReplayKeepsLastWriteForSameKeySameTimestamp` — the end-to-end same-key same-ms replay test you asked for. It writes through the real `WALWriter`/`WALReader` path and asserts the last write wins for all three orderings. Confirmed it fails without the sequence stamp (`recreated-key` resurrection is lost) and passes with it. - Added comparator-level tests for the sequence tie-break, antisymmetry with sequence ids, and the legacy zero-sequence fallback. All 11 tests in the module pass; spotless is clean. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
