prabhnoor0212 commented on code in PR #74776:
URL: https://github.com/apache/airflow/pull/74776#discussion_r4303502137
##########
providers/common/sql/src/airflow/providers/common/sql/operators/sql.py:
##########
@@ -321,7 +321,7 @@ def get_openlineage_facets_on_start(self) ->
OperatorLineage | None:
hook=hook,
database_info=database_info,
database=self.database,
- sqlalchemy_engine=hook.get_sqlalchemy_engine(),
+ sqlalchemy_engine=hook.get_sqlalchemy_engine() if
use_external_connection else None,
Review Comment:
Thanks, addressed in the latest commit. When `sql_use_external_connection`
is disabled, the default schema is no longer queried for Postgres, Redshift,
MSSQL and Oracle. A new get_default_schema helper in sqlparser.py returns a
static default instead: public, dbo, and the uppercased login for Oracle. Both
SQLExecuteQueryOperator and the shared helper use it. Hooks that already take
the schema from connection settings (Snowflake, Databricks, Trino, etc.) are
still asked as before, so their schema is kept. Oracle connections without a
login still query; the config description says so.
But, I'd like your view on the design, since there's a tradeoff:
1. *Keep static defaults in the OpenLineage provider (current approach).*
This is correct for the common setup, but it can be confidently wrong: Postgres
search_path defaults to "$user", public, MSSQL users can have a non-dbo default
schema, and Oracle CURRENT_SCHEMA can differ from the login. It also relies on
a dialect list so a third party hook that queries for its default schema would
still query.
2. *Let each hook opt in with an offline default-schema method,* and leave
the schema unqualified when a hook doesn't provide one. That avoids guessing
and doesn't depend on a dialect list. But from_table_meta would need to omit a
missing schema (today it renders db.None.table), and the change would touch the
postgres, amazon, microsoft.mssql and oracle providers.
Do you have a preference, or a different approach in mind?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
