goutamadwant commented on PR #11512: URL: https://github.com/apache/seatunnel/pull/11512#issuecomment-7002159143
@SEZ9 pointers for F1–F8, against where things stand now. The API is being reworked on #13239 into a Source-level contract (`SourceReaderMetrics` / `SourceEnumeratorMetrics`, proposal in that PR), so some of these resolve by removal rather than by a fix. - **F1 (unbounded `activeSplits`) and F8 (shallow copy of `CdcSnapshotSplitProgress`):** the per-split detail list is removed from the contract. The enumerator report carries counts only; there is no `CdcSnapshotSplitProgress` type to bound or copy. Lands in #13239 with the rework; #13240 and this PR drop the serializer block and the assigner detail code. - **F2 (no credentials in position payloads):** in the API doc on #13239, under "Positions": payloads carry offset coordinates only, never credentials, connection URLs or authentication material. Stays in the reworked doc. - **F3 (enums by name in the codec):** already in this PR's head `738c0acc46`: `CdcProgressReportSerializer` writes owner, lifecycle, assignment status and accuracy by `name()` and rejects unknown names. The rework keeps that and adds phases as strings. - **F4 (runtime collection docs, pull/derive vs the enumerator path):** the "Runtime collection in Zeta" section this PR adds to `docs/*/developer/cdc-progress.md` covers the worker pull for readers and the coordinator-owned enumerator path. It will be updated for the new names. - **F5 (which connectors implement it):** intentionally only `connector-cdc-base` and the connectors built on it (MySQL declares `MYSQL_BINLOG`; the others use their plugin name as position family). I will add that as a known-gap line in the API doc. - **F6 (`SNAPSHOT` Javadoc vs the ownership rule):** the Javadoc currently says only "reading snapshot splits". With `COMPLETED` added it still does not state the ownership rule; it will say that the consumed position is `UNAVAILABLE` during `SNAPSHOT` and that the reader does not infer enumerator state. Fixed in the rework together with the phase rename. - **F7:** both `CdcEnumeratorProgressReport` constructors in the current head document the non-negative and `assigned == completed + running` (only when all `EXACT`) rules; the rework keeps one constructor with the same rule, with `running` renamed to `inFlight`. Integration coverage for the distributed path will come with the rework of this PR rather than only in #12137. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
