github-actions[bot] opened a new pull request, #53761: URL: https://github.com/apache/airflow/pull/53761
Often remote logging is down using automatic instance profiles, but not always. If you tried to configure a logger by a connection defined in the metadata DB it would have not worked (it either caused the supervise job to fail early, or to just behave as if the connection didn't exist, depending on the hook's behaviour) Unfortunately, the way of knowing what the default connection ID various hooks use is not easily discoverable, at least not easily from the outside (we can't look at `remote.hook` as for most log providers that would try to load the connection, failing in the way we are trying to fix) so I updated the log config module to keep track of what the default conn id is for the modern log providers. Once we have the connection ID we know (or at least have a good idea that we've got the right one) we then pre-emptively check the secrets backends for it, if not found there load it from the API server, and then either way. if we find a connection we put it in the env variable so that it is available. The reason we use this approach, is that are running in the supervisor process itself, so SUPERVISOR_COMMS is not and cannot be set yet. (cherry picked from commit e4fb6862c6b16f790fed1baa4af8882125c984c9) Co-authored-by: Ash Berlin-Taylor <[email protected]> -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
