DanielLeens commented on issue #11484: URL: https://github.com/apache/seatunnel/issues/11484#issuecomment-5013611212
Thanks for opening this. This reads more like a feature request than a bug report, but the current scope is still too broad to evaluate as-is. In SeaTunnel today, we do have engine-side tracing / lineage-related groundwork such as StainTrace, but we do not have a stable REST contract for returning job lineage, and `job lineage` versus `column lineage` would lead to very different design scopes. The most important next step is to narrow the target first. It would help to clarify: 1. whether you mean pipeline-level lineage only (for example: source -> transform -> sink graph for one job) or true column-level lineage 2. whether this is expected for batch, streaming, or both 3. what concrete REST response you expect to consume downstream 4. whether the source of truth should come from runtime execution metadata, static job-plan analysis, or trace data If the immediate need is job-level runtime flow visibility, that is a much smaller and more reviewable first step than column lineage. But if the target is column lineage, that likely needs a larger design discussion or even a STIP, because it touches transform semantics, schema mapping, and compatibility of the public API. Please share one concrete usage scenario and an example response shape. That will make it much easier to judge the right scope for a first implementation. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
