potiuk commented on PR #25266: URL: https://github.com/apache/airflow/pull/25266#issuecomment-1195622818
> So generally the fewer things you have to lock the better, and doubly so as we aren't actually making any changes to DagRun, so taking out a lock on it as well seems like it _should_ be unnecessary. Let me (re)read that other thead. It changes the task instance state only - which means that task instance state is not "frozen" during the scheduler processing. If we remove the lock from the dagrun it might happen that task state change from "queued" to "running" for example. Which might impact scheduler's decisions. I do not think it can be a "disastrous" change, but generally it is something that scheduler does not expect when making decisions. So things like running more task instances than are allowed per dagrun or similar might happen. Yeah, less locks is better, the only problem with not having DagRun lock is that is far more difficult to reason what could happen and what can go wrong. When DagRun is "frozen", nothing wrong can happen. When it's not, well, I am not sure :) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
