potiuk commented on issue #19957:
URL: https://github.com/apache/airflow/issues/19957#issuecomment-1108694255
Question - I am a bit lost where we are currently and since this issue is
long and relate to other - likely fixed - problem may I have kind request - can
someone create a new issue
potiuk commented on issue #19957:
URL: https://github.com/apache/airflow/issues/19957#issuecomment-1019550207
@dwiajik @stablum - is there any chance you have some customisations
(plugins ? ) or users running DB operations (backfill? API calls? UI
modifications) that might have caused the
potiuk commented on issue #19957:
URL: https://github.com/apache/airflow/issues/19957#issuecomment-1019539469
I am afraid we need to reopen this one. IMHO #20894 has no chance of fixing
the problem because it does not change Airflow behaviour really (see discussion
in #20894). @dwiajik
potiuk commented on issue #19957:
URL: https://github.com/apache/airflow/issues/19957#issuecomment-1011857056
I actually spent some time few days ago looking at the mini-scheduler code
but I could not really find a flaw there. The fact that it did not help you
indicates that my hypothesis
potiuk commented on issue #19957:
URL: https://github.com/apache/airflow/issues/19957#issuecomment-1008022416
> I managed to run a large number tasks without airflow crashing, so
changing that setting as you suggested did indeed help! Thank you! :)
Cool! Thanks for confirming! Now
potiuk commented on issue #19957:
URL: https://github.com/apache/airflow/issues/19957#issuecomment-1005969121
@stablum @AmarEL - is it possible that you disable this configuraiton and
see if it helps?
--
This is an automated message from the Apache Git Service.
To respond to the
potiuk commented on issue #19957:
URL: https://github.com/apache/airflow/issues/19957#issuecomment-1003544650
cc: @ashb -> do you think this is plausible hypothesis ^^ ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and
potiuk commented on issue #19957:
URL: https://github.com/apache/airflow/issues/19957#issuecomment-1003544607
To bump it up - Just let us know @stablum @AmarEL if disabling
mini-scheduler works.
I **think** this might be causesd by the mini-scheduler in the task
deadlocking with
potiuk commented on issue #19957:
URL: https://github.com/apache/airflow/issues/19957#issuecomment-1002655342
I looked at the code and places where it can occure and I have a theory
what could cause it.
And maybe some of people here can verify my theory.
Could you please
potiuk commented on issue #19957:
URL: https://github.com/apache/airflow/issues/19957#issuecomment-994817635
Marked it provisionally to 2.2.4 in case we release it (it might go straight
to 2.3.0 depending on how serious things there are / whether we haave a fix and
how close we are to
potiuk commented on issue #19957:
URL: https://github.com/apache/airflow/issues/19957#issuecomment-994816151
Thanks @stablum ! Great to see all the detailed information (and the
pastebin)
@ashb - I think that one might need some closer look from both of us :) and
some hypytheses on
potiuk commented on issue #19957:
URL: https://github.com/apache/airflow/issues/19957#issuecomment-986335228
(Reason why I am asking - there are very low number of Postgres deadlock
reports we receive, because (unlike MySQL) Postgres usually shows "real"
deadlocks and we'd expect this
potiuk commented on issue #19957:
URL: https://github.com/apache/airflow/issues/19957#issuecomment-986334766
Also - is there a chance to find out from the logs what was the "other"
query that the deadlock happened on? Is this also possible that you have
another person or process (either
potiuk commented on issue #19957:
URL: https://github.com/apache/airflow/issues/19957#issuecomment-986332738
Do you happen to have one or multiple schedulers?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL
14 matches
Mail list logo
