kacpermuda commented on code in PR #47500:
URL: https://github.com/apache/airflow/pull/47500#discussion_r1987043289
##########
providers/dbt/cloud/src/airflow/providers/dbt/cloud/utils/openlineage.py:
##########
@@ -110,8 +114,15 @@ def generate_openlineage_events_from_dbt_cloud_run(
# catalog is available only if docs are generated
catalog = None
- with suppress(Exception):
+ try:
+ log.debug("Retrieving information about catalog artifact from DBT.")
catalog = operator.hook.get_job_run_artifact(operator.run_id,
path="catalog.json").json()["data"]
+ except Exception: # type: ignore
+ log.info(
+ "Openlineage could not find DBT catalog artifact, usually
available when docs are generated."
+ "Proceeding with metadata extraction. "
+ "If you see error logs above about `HTTP error: Not Found` it's
safe to ignore them."
+ )
Review Comment:
These logs come from HTTPHook, that is used underneath. There is no way to
pass some arguments that deep so that It does not log this error, so my only
idea was to write some kind of logs interceptor to be used for the duration of
this call. It looked to complex compared to the benefits, so I decided against.
Do you have some simpler idea?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
