potiuk commented on issue #17897:
URL: https://github.com/apache/airflow/issues/17897#issuecomment-910833359
Some update @ashb after more looking. I realized that IF I am right about
the hypothesis, we'd heave much bigger problem for multiple schedulers. so I
looked a bit deeper. While my fix is likely to help with 1 scheduler, it would
not help for more of them.
But I found that this should actually prevent updating TAGs for the same
dag_id by two parallell parsing processes (regardless if in one or more
schedulers), So I think the only explanation I see now is that row-level
locking is entirely disabled (@dimon222 - do you actually have it disabled
(scheduler/use_row_level_locking)?):
```
query = (
session.query(DagModel)
.options(joinedload(DagModel.tags, innerjoin=False))
.filter(DagModel.dag_id.in_(dag_ids))
)
orm_dags = with_row_locks(query, of=DagModel, session=session).all()
``
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
