potiuk commented on issue #58919: URL: https://github.com/apache/airflow/issues/58919#issuecomment-3629496888
As commented in #59183 I do no think it will work. The locks are only created when row is selected, so when there is no row found, no lock is created. With the database locks there are two approaches: 1) You run SELECT FOR UPDATE (with_row_lock) on a row(s) that exist befor the lock attempt. 2) if you do not have a row you can create a dedicated database lock (advisory lock) - we use it for database migration for example - but this one will lock all kinds of similar transactions and would not be performant. 3) You could also lock the whole table, but that's even worse Like @dstandish mentioned - locking Dag table is too "heavy". There is no other direct table with foreign key that we could lock (target_dag_id is the only one) - however in fact ADPR is linked to Asset via PartitionedAssetKeyLog and we know the asset that we are operating on (asset_id), so we could lock asset row instead. This is fine grained - you only block this particular asset from being updated in parallel. Which I think is the best approach. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
