SEZ9 commented on issue #12268:
URL: https://github.com/apache/seatunnel/issues/12268#issuecomment-5987023250
@goutamadwant — on D7–D9, directly:
This reverses D1 as recorded in your table ("no column lineage", status
"Goal supported on dev@"). That is fine as a proposal, but it means the
decision that was already taken to the dev list no longer matches the issue
body. Please post the D7–D9 change to the existing dev-list thread
(https://lists.apache.org/thread/o9z45fvngl66bljjsk70fyc09dd2okn3) and update
the D1 row so it points at D7–D9 rather than reading as settled. I don't want a
scope change of this size to live only in a comment.
On the three points as you've stated them:
- D7: `?level=column` with the default response unchanged is the right shape
— the table-level contract from D4 must stay byte-for-byte what Phases 1–3
deliver. Please state explicitly which of the D4 error codes apply when
`level=column` is requested for a job whose column data is unknown.
- D8: a new optional `SupportsColumnLineage` interface in `seatunnel-api` is
a public API addition, which is a different kind of decision from the read-only
endpoint in D1–D6. I'd like it recorded as its own open row, with the four
transforms (`Copy`, `FieldMapper`, `Filter`, `Replace`) and the "unknown"
fallback spelled out, and confirmed on the dev list before any
connector/transform code changes start.
- D9: the 100,000-column cap needs to be reconciled with D5 (8 MiB response,
413 rejected whole). Please state whether column data counts against the 8 MiB
limit and what the response is when the column cap is hit but the table-level
graph fits.
Also, please paste the measured numbers from the new Impact section into
this thread so they are reviewable alongside the D5 provisional values.
Phases 1–3 and #12619 remain unaffected as you say; nothing above blocks
them. D3 (eager snapshot build at `JobMaster` init, 409 fallback, rebuild after
failover) is still the open item I need confirmed for Phase 2.
<!-- streview-comment:1521 -->
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
