mobuchowski commented on code in PR #74805:
URL: https://github.com/apache/airflow/pull/74805#discussion_r4388930188
##########
providers/openlineage/src/airflow/providers/openlineage/utils/sql_hook_lineage.py:
##########
@@ -182,7 +186,8 @@ def _get_namespace(hook_id: int) -> str | None:
database_info=database_info,
)
except Exception as e:
- log.debug("Failed to parse SQL for query %s: %s", query_count,
e)
+ log.warning("Failed to parse SQL for query %s: %s",
query_count, e)
Review Comment:
Two small things in the per-query loop, now that these are warnings:
1. "Failed to parse SQL for query %s" is the only new warning that doesn't
name the connection. `conn_id` is already available a few lines up (L172).
Something like `"Failed to parse SQL for query %s on %s: %s", query_count,
conn_id, e` would make it consistent with the other messages in this PR.
2. Only database info is cached per hook (`_get_database_info`). The dialect
and default-schema lookups run again inside `get_openlineage_facets_with_sql`
for every query. If one of them raises, a task with 500 SQL extras from the
same hook writes 500 identical "Failed to get OpenLineage database dialect or
default schema" warnings.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
