re

As tasks require connection access, I assume connection data will somehow
> be passed as part of the
> metadata to task execution - whether it's part of the executor protocol or
> in some other way (I'm
> not an expert on that part of Airflow). Then, provided it's accessible as
> part of some execution
> context, and not only passed to the task's execute method, OpenLineage
> could utilize it.
>

It's not strictly necessary that connection info be passed "as part of task
matadata".  That would require some mechanism of declaring prior to task
execution what connections would be used.  This is a thought that has come
up when thinking about execution of non-python tasks.  But it's not
required from a technical perspective by AIP-44 because the
`get_connection` function can be made to be an RPC call so a task could
continue to retrieve connections at runtime.

Reply via email to