adrian-edbert commented on issue #56165: URL: https://github.com/apache/airflow/issues/56165#issuecomment-3435906994
> oh the root cause seems because the git provider version_compat providers/git/src/airflow/providers/git/version_compat.py > > is using sdk basehook object, which eventually will load secrets backend from airflow-core/src/airflow/secrets/**init**.py but only by `DEFAULT_SECRETS_SEARCH_PATH_WORKERS` > > this load git providers without the metastore secret backend, causing the githook get_connection to fails > > I'm unsure if this is expected, perhaps due to the AIP-72 progress > > I can workaround the issue by explicitly adding > > ``` > [secrets] > backend = airflow.secrets.metastore.MetastoreBackend > ``` > > on the airflow.cfg though this might be anti pattern due to the AIP I'm sorry, this workaround seems to have an unexpected consequences, since the secrets backend running session.expunge_all() and this is being called every time the git dag bundle is initialized, and it seems the session context is shared between airflow components during dag processing import_errors, on `airflow-core/src/airflow/dag_processing/collection.py`, there's a part where it called `import_error.full_file_path()` after session.add() and before session.flush(), this will call the git dag bundle init -> secrets metastore backend, causing the import_errors update / insert not flushed to db (since it got expunged by secrets metastore backend) so for now instead of using the web ui to configure the connection we decided to just use the env secret backend and deploy it along the server and worker env variable https://airflow.apache.org/docs/apache-airflow/stable/howto/connection.html#environment-variables-connections -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
