ashb opened a new pull request #19842: URL: https://github.com/apache/airflow/pull/19842
If you are running multiple schedulers on PostgreSQL, it is likely that sooner or later you will have one scheduler fail the race to enter the critical section (which is fine, and expected). However this can end up spamming the DB logs with errors like this: ``` Nov 26 14:08:48 sinope postgres[709953]: 2021-11-26 14:08:48.672 GMT [709953] ERROR: could not obtain lock on row in relation "slot_pool" Nov 26 14:08:48 sinope postgres[709953]: 2021-11-26 14:08:48.672 GMT [709953] STATEMENT: SELECT slot_pool.pool AS slot_pool_pool, slot_pool.slots AS slot_pool_slots Nov 26 14:08:48 sinope postgres[709953]: FROM slot_pool FOR UPDATE NOWAIT Nov 26 14:08:49 sinope postgres[709954]: 2021-11-26 14:08:49.730 GMT [709954] ERROR: could not obtain lock on row in relation "slot_pool" Nov 26 14:08:49 sinope postgres[709954]: 2021-11-26 14:08:49.730 GMT [709954] STATEMENT: SELECT slot_pool.pool AS slot_pool_pool, slot_pool.slots AS slot_pool_slots Nov 26 14:08:49 sinope postgres[709954]: FROM slot_pool FOR UPDATE NOWAIT ``` If you are really unlucky that can end up happening over and over and over again. So to avoid this error, for PostgreSQL only, we first try to acquire an "advisory lock" (advisory because it's up to the application to respect it), and if we cannot raise an error _like_ would have happened from the `FOR UPDATE NOWAIT`. (We still obtain the exclusive log on the pool rows so that the rows are locked.) This PR is split in to two commits, the second obtains the lock, and the first commit refactors the existing global locks to use enums to remove magic constants as these (integer for postgres) lock ids are "global", so we need to be sure the scheduler's lock doesn't clash with the `db upgrade` lock. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
