And there was a wider issue, the trigger method didn't actually used the DagBag already created for the Webserver but instead created a new one which wasn't necessary. And the existing DagBag in views.py already passed all the flags correctly :)
Regards, Kaxil On Mon, May 4, 2020, 10:30 Kaxil Naik <kaxiln...@gmail.com> wrote: > Hi Niebler, > > Like Ash mentioned this is fixed for 1.10.11 already. > > This was first reported in following Github issue: > > - https://github.com/apache/airflow/issues/8247 > > And PR that fixes it for v1-10-test: > https://github.com/apache/airflow/pull/8411 > > And https://github.com/apache/airflow/pull/8501 also fixes it in Master. > > Regards, > Kaxil > > > On Mon, May 4, 2020, 09:21 Ash Berlin-Taylor <a...@apache.org> wrote: > >> I think this was fixed by this PR - aiming for 1.10.11 >> >> https://github.com/apache/airflow/pull/8501 >> >> On 4 May 2020 07:41:55 BST, "Niebler Thomas (DC-IH/SDL1)" >> <thomas.nieb...@boschrexroth.de.INVALID> wrote: >> >Hi all, >> > >> >I have a probably rather special use case scenario: >> >Using Airflow 1.10.10, I would like to physically decouple the >> >Webserver and the Scheduler for some secure access reasons. According >> >to https://airflow.apache.org/docs/stable/dag-serialization.html, this >> >should be a piece of cake with Airflow 1.10.10, since DAGs are stored >> >in the Metadata database and the webserver does not need to access the >> >DAG files anymore. The metadata database is of course reachable by both >> >Airflow instances: >> > >> >Docker Image Instance: Airflow Webserver <----> Docker Image Instance: >> >Metadata database <----> Physical Machine: Airflow Scheduler >> > >> >However, every time I start a DAG manually, it crashes with a rather >> >lengthy error message: >> > >> >File "/usr/local/lib/python3.7/site-packages/airflow/www/views.py", >> >line 1255, in trigger >> > external_trigger=True >> >File "/usr/local/lib/python3.7/site-packages/airflow/utils/db.py", line >> >74, in wrapper >> > return func(*args, **kwargs) >> >File "/usr/local/lib/python3.7/site-packages/airflow/models/dag.py", >> >line 1818, in create_dagrun >> > return self.get_dag().create_dagrun(run_id=run_id, >> >AttributeError: 'NoneType' object has no attribute 'create_dagrun' >> > >> >This basically boils down to self.get_dag() not having set the Boolean >> >flag store_serialized_flags to True (or whatever value the config is >> >set to), but always using False (the default value). >> >This then leads to Airflow attempting to read the DAG file, ignoring >> >the DAG database entry and returning None, which obviously has no >> >attribute create_dagrun. >> > >> >I’ve got several questions here now: >> > >> >1. Is my scenario even possible or am I overlooking something rather >> >obvious? >> >2. Is the crashing DAG behavior intended like that? It rather seems >> >like a bug to me. >> >3. Is it worth fixing this issue (if it is one) for Airflow 1.10.x, >> >considering that Airflow 2.0.0 does not even contain the corresponding >> >classes anymore and takes a different path? >> > >> >Mit freundlichen Grüßen / Best regards >> > >> >Dr. Thomas Niebler >> >Data Scientist >> >Sales Data Lab, Analytics DC-IH/SDL1 >> > >> >Tel. +49 9352 18-2392 >> >Fax +49 9352 18-0 >> >thomas.nieb...@boschrexroth.de<mailto:thomas.nieb...@boschrexroth.de> >> >www.boschrexroth.com >> > >> >Bosch Rexroth AG >> >Partensteiner Straße 23 >> >97816 Lohr am Main >> >GERMANY >> > >> >[BOSCH REXROTH]<http://www.boschrexroth.com/> >> > >> > >> > >> >Sitz: Stuttgart, Registergericht: Amtsgericht Stuttgart HRB 23192 >> >Vorstand: Rolf Najork (Vorsitzender), Dr. Markus Forschner, Dr. Heiner >> >Lang, Reinhard Schäfer, Dr. Marc Wucherer >> >Vorsitzender des Aufsichtsrats: Christoph Kübel >> > >> >>
