SEZ9 commented on PR #13060: URL: https://github.com/apache/seatunnel/pull/13060#issuecomment-6552361193
@zephyr45 Thanks for the updates in `d8143b58` and `21d4cb19`. From your description, the generation token and job-level generated `transaction_prefix` created once in `KafkaSink` sound like the right shape for the stateless-writer fencing, Spark task-retry, and omitted-prefix scale-down concerns, and the deterministic relocation namespace with the 3 -> 2 -> 3 regression is the scenario I wanted modeled. I have not yet verified these against the diff, so a few things would help me close out the earlier points: 1. Flink scale-down restore: now that the modulo assumption is gone, which test exercises a contiguous-chunk ListState redistribution (e.g. one writer receiving states for two adjacent indices) and confirms restore succeeds rather than throwing? 2. Omitted `transaction_prefix` scale-down: which test covers restore with mixed historical per-writer generated prefixes plus the new job-level prefix? 3. Legacy shared-namespace fencing: is the concurrent probe-chain growth with parallelism (every restored writer fencing the same `<prefix>-<checkpoint>` namespace and bumping epochs its siblings observe) addressed in either commit, or still open? If open, a short note on the intended approach is fine for this PR. 4. Docs: please confirm the `transaction_prefix` option row (not only the Semantics paragraph) describes the new ID shape, and that the rescaling claim is scoped to the engines where it holds. 5. Please report back once the queued run at https://github.com/zephyr45/seatunnel/actions/runs/41218673502 finishes for head `21d4cb19869a7875f52ba5035e20f177599feaaa`. I'll do a final pass once those are in. <!-- streview-comment:2180 --> -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
