mobuchowski opened a new pull request, #30719:
URL: https://github.com/apache/airflow/pull/30719
Add base SqlExtractor. This provides base upon which extractors gathering
data from operators which derive from `SQLExecuteQueryOperator` are build. This
means
- passing SQL to SQL parser with right options, like dialect
- handling resulting data, constructing column level lineage facet
- constructing query and querying database for schema for the used tables
- handling errors
Then those, like `PostgresExtractor`, `TrinoExtractor` or
`RedshiftSqlExtractor` only need to define specifics for the particular
database they work on, like how to get particular hook or what's the
OpenLineage namespace for this particular database.
Right now this PR cleanly derives on
https://github.com/apache/airflow/pull/30713, and does not change files that
previous PR changed - only adds `dbapi_utils.py` `sql_extractor.py` and
corresponding tests.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
