dstaple commented on issue #23361: URL: https://github.com/apache/airflow/issues/23361#issuecomment-1193403063
@potiuk Thanks a ton for the feedback! Regarding why the dag_run table is locked when `with_for_update()` is called on the TaskInstance table, I believe this is due to `lazy='joined'` in the relationship between TaskInstance and DagRun: https://github.com/apache/airflow/blob/2.2.5/airflow/models/taskinstance.py#L408 This behavior is described in the following sqlalchemy issue: https://github.com/sqlalchemy/sqlalchemy/issues/4100 The behavior was not changed after the above issue was filed, but the following warning was added to the SQLAlchemy documentation: > Using with_for_update in the context of eager loading relationships is not officially supported or recommended by SQLAlchemy and may not work with certain queries on various database backends. When with_for_update is successfully used with a query that involves joinedload(), SQLAlchemy will attempt to emit SQL that locks all involved tables. https://docs.sqlalchemy.org/en/14/orm/loading_relationships.html -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
