Andrushika commented on PR #66854: URL: https://github.com/apache/airflow/pull/66854#issuecomment-4949239693
Would it be possible to keep the TI state update and asset registration in the same transaction, but defer acquiring the TI row lock until immediately before the final state update? My current understanding of the original flow is: ```text ti_update_state() → SELECT the TI row FOR UPDATE (lock acquire) → perform asset registration → update ti.state → commit and release the lock ``` This seems to mean that when asset registration takes a long time, the TI row lock is held for that entire period, causing later requests for the same TI to wait on `SELECT ... FOR UPDATE`. I was wondering whether the ordering could instead be: ```text ti_update_state() → perform asset registration → acquire the TI row lock and re-check the state → update ti.state → commit both changes together ``` If the state is no longer valid at that point, the transaction could be rolled back together with the asset changes. Would this potentially shorten the TI lock window while preserving atomicity between the state update and asset events? I have not audited the other TI writers or the lock-ordering implications, so there may be a constraint I have overlooked. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
