SEZ9 commented on PR #13060:
URL: https://github.com/apache/seatunnel/pull/13060#issuecomment-6620591604
@zephyr45 Thanks for the self-contained write-up against `b9ed7c1297d9` —
the red/green evidence for F1 is exactly what I was after.
On F1 specifically: the point that the connector just consumes whatever
states the engine hands it, with no modulo/index ownership check, resolves the
original concern that a contiguous-chunk (Flink-style) redistribution would
trip an `IllegalStateException`. The `3 -> 4 -> 3 -> 4 -> 3` model with
empty-state writers restarting their local checkpoint counter at 1 is the right
shape for this, and the `1 test, 1 failure` at `351106062` (`Duplicate
namespace at parallelism 3: shared-prefix-7-gtestw3r2c1`) vs. `1 test, 0
failures` at `97516b977` is a convincing red/green. I'll treat F1 as addressed
once I've read the test at the linked `KafkaSinkWriterTest.java#L336-L382`
range on the current head.
One thing: your comment is cut off right at "Current head `b9ed7" — the
result for the current head itself (and anything you had after item 1) never
made it through. Could you re-post the remainder? In particular I'd like to see:
1. The run result for the same test method at `b9ed7c1297d9`, since that
head also carries the bounded fresh-writer follow-up on top of `97516b977`.
2. Your responses to the remaining previous points, which I still have no
answer for in this thread:
- F2 — scale-down restore when `transaction_prefix` is not configured and
writers carry their own random prefixes.
- F3 — a no-state (scale-up) writer starting its sequence at 1 regardless
of the job's checkpoint id, leaving the first failed attempt unfenced. If the
"bounded fresh-writer follow-up" is meant to cover this, please say so
explicitly and point at the test.
- F4 — Spark task retries getting a fresh namespace via the taskId, so
the previous attempt's open transaction is never fenced.
- F5/F6 — concurrent legacy shared-namespace fencing growing with
parallelism.
- F7 — aligning the `transaction_prefix` option row and the engine scope
of the rescaling claim in `docs/en/connectors/sink/Kafka.md` with the new
ID-shape text.
Once the rest of the ledger is visible I'll go through it in one pass.
<!-- streview-comment:2253 -->
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]