Howdy - wanted to provide an update - the following patch applied manually to a clean 1.8.1 install addressed the issue:
https://github.com/apache/incubator-airflow/pull/2143 <https://github.com/apache/incubator-airflow/pull/2143> We have confirmed/verified this with jobs running over multiple hours on our instance - so the above pull request can close both of the below: https://issues.apache.org/jira/browse/AIRFLOW-966 <https://issues.apache.org/jira/browse/AIRFLOW-966> https://issues.apache.org/jira/browse/AIRFLOW-1258 <https://issues.apache.org/jira/browse/AIRFLOW-1258> As I have seen further discussion on 1.8.2 with the final build not being tagged yet (iirc), would it be possibly to merge this for the 1.8.2 release? As it currently is, someone using Airflow+Celery will have a broken configuration for long-running jobs - would the dev team consider this major/critical enough to include? Cheers, J > On Jul 17, 2017, at 3:15 PM, Jawahar Panchal <[email protected]> wrote: > > Hi again! > > >> On Jul 16, 2017, at 3:22 AM, Alex Guziel <[email protected] >> <mailto:[email protected]>> wrote: >> >> I think this may be related to a celery bug. I'll follow up with more >> details later. >> > > Just replying back to the note earlier in the thread - apologies for the > earlier top-posting, got a bit excited that I might have found the issue, and > of course lack of sleep results in one doing terrible, terrible things… :) > > Any idea if my suspicion around the 1h default visibility timeout between > celery/redis is the culprit? > >> On Sun, Jul 16, 2017 at 12:56 AM Jawahar Panchal <[email protected]> >> wrote: >> >>> Hi! >>> >>> I am currently running a couple of long-running tasks on a >>> database/dataset at school for a project that results in behavior/log >>> output similar to what was flagged in this bug: >>> https://issues.apache.org/jira/browse/AIRFLOW-1258 < >>> https://issues.apache.org/jira/browse/AIRFLOW-1258> >>> >>> Wasn’t sure if anyone on the list had seen anything similar, or would know >>> what I can do to possibly debug further/patch. As it takes 1hr to test a >>> change, needless to say any pointers from the dev team on the right >>> direction to look within the codebase would be much appreciated! :) >>> >>> Thanks in advance for everyone’s/anyone's time and help - am not an >>> Airflow expert, but am hopefully learning quickly enough to help resolve >>> this issue (if I am ‘barking up the right tree’ with this bug number…) >>> >>> Cheers, >>> J >>> >>> > > Cheers, > J
