Re: Potential issue with serialized DAGs in decoupled Webserver and Scheduler

2020-05-05 Thread Kaxil Naik
Amtsgericht Stuttgart HRB 23192 >> Vorstand: Rolf Najork (Vorsitzender), Dr. Markus Forschner, Dr. Heiner >> Lang, Reinhard Schäfer, Dr. Marc Wucherer >> Vorsitzender des Aufsichtsrats: Christoph Kübel >> ​ >> Von: Kaxil Naik >> Gesendet: Monday, May 4, 2020 11:3

Re: Potential issue with serialized DAGs in decoupled Webserver and Scheduler

2020-05-05 Thread Anita Fronczak
Aufsichtsrats: Christoph Kübel > ​ > Von: Kaxil Naik > Gesendet: Monday, May 4, 2020 11:38 AM > An: dev@airflow.apache.org > Cc: Niebler Thomas (DC-IH/SDL1) > Betreff: Re: Potential issue with serialized DAGs in decoupled Webserver > and Scheduler > > And there was a wid

Re: Potential issue with serialized DAGs in decoupled Webserver and Scheduler

2020-05-04 Thread Kaxil Naik
And there was a wider issue, the trigger method didn't actually used the DagBag already created for the Webserver but instead created a new one which wasn't necessary. And the existing DagBag in views.py already passed all the flags correctly :) Regards, Kaxil On Mon, May 4, 2020, 10:30 Kaxil Nai

Re: Potential issue with serialized DAGs in decoupled Webserver and Scheduler

2020-05-04 Thread Kaxil Naik
Hi Niebler, Like Ash mentioned this is fixed for 1.10.11 already. This was first reported in following Github issue: - https://github.com/apache/airflow/issues/8247 And PR that fixes it for v1-10-test: https://github.com/apache/airflow/pull/8411 And https://github.com/apache/airflow/pull/8501

Re: Potential issue with serialized DAGs in decoupled Webserver and Scheduler

2020-05-04 Thread Ash Berlin-Taylor
I think this was fixed by this PR - aiming for 1.10.11 https://github.com/apache/airflow/pull/8501 On 4 May 2020 07:41:55 BST, "Niebler Thomas (DC-IH/SDL1)" wrote: >Hi all, > >I have a probably rather special use case scenario: >Using Airflow 1.10.10, I would like to physically decouple the >We

Re: Potential issue with serialized DAGs in decoupled Webserver and Scheduler

2020-05-04 Thread Anita Fronczak
Hello, Your scenario is possible. I have a ready PR that sets store_serialized_dags in all places to appropriate value (web server, CLI, experimental API) and then dag serialization works without access to DAG files (tested). I shall push the PR today. Anita pon., 4 maj 2020, 08:42 użytkownik Ni