ashb commented on pull request #8718: URL: https://github.com/apache/airflow/pull/8718#issuecomment-625232783
> Hi, > > I will collectively answer the questions here. > > Many of the changes that I have made have been a result of setting store_serialized_dags from conf in DagBag init method in one of squashed together commits. > Before I decided that it is better to leave it with False default I started to change it to False in all places where it broke things. > And it broke things in CLI, dag parsing code and this migration: airflow/migrations/versions/cc1e65623dc7_add_max_tries_column_to_task_instance.py What error exactly? And is that error reproducible in Airflow 1.10.10, or is it only a problem in the Cloud Composer version? > > This change is in fact noop so maybe we should drop it. > It actually is about not reading core.store_serialized_dags on server startup. I had also problems with tests because of that. But if that operation is expensive and people will not probably be turning dag serialization on and off, then maybe it is better to drop it. Airflow does not support changing config at runtime -- it is not possible. It is only the tests that are able to change settings. > > Here is a similar change to Airflow 1.10.10: > #8764 > This one actually fixes starting dag from UI. > In this change I can also see dag async dag bag loading was dropped altogether in Airflow 1.10.10. Am I correct? No, async dag bag loading was _never in_ Airflow. Either dag serialization is enabled, in which case the webserver loads it form the serialized tables in the DB, or it's not and the existing mechnaism is in place. Having a _third_ method of loading dags when dag serialization exists and fixes the performance overhead seems like extra code to maintain for little benefit. Also: there is a reasonable chance that in Airflow 2.0 serializing dags may be the _only_ option. ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected]
