SEZ9 commented on PR #13060: URL: https://github.com/apache/seatunnel/pull/13060#issuecomment-6905483611
@zephyr45 Thanks for the detailed recheck against `7c272c5a` and for rerunning the regressions locally. - **F1/F2:** Removing `validateRestoredStateOwner` and the modulo rejection, and parsing each restored state with its own saved prefix, is the direction I was asking for. The contiguous-assignment, mixed-prefix and `3 → 4 → 3 → 4 → 3` scenarios you describe are the right coverage. I'll confirm against the diff in `7c272c5a` before closing these. - **F4:** Using the stable index (Spark partition ID) for the Kafka writer, with the broker retry test showing task IDs 99 and 100 sharing stable index 0 and the abandoned record staying invisible under `read_committed`, addresses my concern. I'll verify in the diff and then close it. - **F7:** The option row plus semantics paragraph covering the ID shape, Zeta/Flink stateful recovery and the Spark speculation restriction (with ZH matching) is what I asked for. Same here: I'll check the docs change in the diff before marking it resolved. - **F3:** I accept that the fresh-writer path should not scan full history given the growing startup cost you observed across Spark micro-batches, and I agree that aligning to a checkpoint counter would not identify transactions that were never captured in checkpoint state. Your comment appears to have been cut off at "Flink's adapter counters ca…" — could you finish that thought? If the conclusion is that the `transaction.timeout.ms` window is the accepted bound for this case, please state that limitation explicitly in the Kafka sink docs next to the semantics text so users set the timeout deliberately. With that, I'm fine closing F3 as a documented limitation rather than a blocker. One item from the earlier review was not covered in your update: - **F5/F6:** the legacy shared-namespace fencing performed concurrently by every restoring writer, where sibling probes bump epochs and extend each other's `getEpoch()==0` scan. Could you confirm whether `7c272c5a` changes anything here (e.g. bounding the probe chain or having a single writer own the legacy fence), or whether you consider it acceptable as a one-time migration cost? Either answer is fine; I just need it stated on the thread. Once the F3 wording and the F5/F6 answer are in, and I've verified the above against the diff, I have nothing else outstanding on this PR. <!-- streview-comment:2377 --> -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
