shnapz commented on issue #33981: URL: https://github.com/apache/beam/issues/33981#issuecomment-3530158244
@charlespnh `OpenLineage` was provided as an example only, I don't have plans to implement it. However when starting I was aware of this ticket that aims to have OpenLineage integration in Beam 3.0, and was curious if that change can help you. As you can see from my PR it intercepts absolutely all Lineage data in a single place, IMO it is a great place to plug in anything. Metric storage seems to be a "third wheel" when the goal is to submit lineage data to OpenLineage compliant API -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
