kacpermuda commented on PR #41217: URL: https://github.com/apache/airflow/pull/41217#issuecomment-2278030917
> a) limit the overhead when it is enabled (this can be done as well by caching the information - so maybe that will be enough to cache the facet. JUST retrieving all installed package information is pretty heavy, and we should do it once per interpreter run ideally. > > b) I **think** there should be a way to enable this logging independently from Airflow logs, because it might well be that just enabling debug logs will have other side effects. I added caching for the package information and introduced a new flag called `debug_mode`, which controls whether this facet is created and added. Now, it should run only once in the scheduler and once per task on the worker, but only when the `debug_mode` flag is explicitly enabled by the user. I also updated the documentation to inform users about the useful information they can provide to us when debugging. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
