DartVeDroid commented on issue #53442:
URL: https://github.com/apache/airflow/issues/53442#issuecomment-3846610149
UPD
Welp, it's weird:
- Most logging providers' handlers currently require for the `set_context()`
method to be called, to pass `dag` and `task` information to it. But:
- Some contributor or maintainer (can't remember who it was exactly now)
has mentioned previously that `set_context()` method is not supposed to be
called by Airflow 3
- Some handlers, like `ElasticsearchTaskHandler`, could also explicitly
try to access Airflow database, which would fail, due to direct database access
being against Airflow 3 architectural principles
- Now, the `dag` and `task` information is accessible from a task itself by
calling a `airflow.sdk.get_current_context()` method, and it is possible to
create a `structlog` `processor` that would call it and enrich log JSONs. But:
- You can only enrich a `structlog` log like that, and while your code
could probably be modified to use it, you would also need to modify your
requirements' logging too, and it does not look easily done to me
- Theoretically, you could enrich all logs passed from a `task` to
`supervisor`, but that would require for
`supervisor.process_log_messages_from_subprocess()` generator method to get an
`ActivitySubprocess.ti` as an optional argument
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
