SEZ9 commented on issue #12268:
URL: https://github.com/apache/seatunnel/issues/12268#issuecomment-5975883632
@goutamadwant thanks for pulling the open decisions into one table — here is
where I land on each:
- **D1 (scope)** — Confirmed. One Zeta job's Source → Transform → Sink graph
derived from `JobDAGInfo`, no column lineage in V1.
- **D2 (identity)** — Nothing to add; `(jobId, nodeId)` as identity with
`name` as a display-only label is fine.
- **D3 (snapshot-only + eager build)** — The snapshot-only read with current
execution taking precedence over history is confirmed. On the new part, I'm
fine in principle with building the snapshot once when `JobMaster` finishes
init, reusing the same call that already runs at job finish at
309b15effe99423f73b6007cbe61b97bb6af7b24. Before I mark it confirmed, please
add two things to the body: (1) what happens if that build fails during init —
it must not fail or delay job start, and the REST request should then fall
through to the existing 409/503 outcomes; (2) how the snapshot is handled when
a running job's `JobMaster` is restored after a master failover, so the
"running jobs are covered" guarantee still holds. With those written down, D3
is confirmed and Phase 2 can rely on it.
- **D4 (route/errors)** — Confirmed: `GET /job-lineage/{jobId}` under the
existing REST base path, `schemaVersion: 1`, and the 400/404/409/413/503/500
mapping with the oversized graph rejected whole via 413.
- **D5 (limits)** — OK as V1 defaults (10k nodes, 50k edges/paths, 4 KiB
strings, 8 MiB response, 4 concurrent requests, 5 s deadline). Treat them as
provisional and bring the measured numbers back here after Phase 2/3 so we can
finalize them in the body.
- **D6 (access/overlap)** — Confirmed: existing Basic Auth, fixed field
whitelist, and no dependency on the auth work tracked separately.
- **No timestamp / completeness flag in V1** — Confirmed as deliberate; a
200 is the full graph, otherwise 409/503/413.
- **Opaque dataset IDs deferred** — Agreed.
Once the two D3 clarifications are in the body, please update the table
statuses accordingly and I'll move on to reviewing the Phase 1 PR.
<!-- streview-comment:1499 -->
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
