coufon edited a comment on issue #5743: [AIRFLOW-5088][AIP-24] Persisting serialized DAG in DB for webserver scalability URL: https://github.com/apache/airflow/pull/5743#issuecomment-520183227 I saw two remaining issues: (1) Serialized DAGs/Operators are big with many redundant fields. We should trim the fields not used by UI, by giving a list of fields to be included in serialization: class SerializedDAG(DAG, Serialization): _included_fields = list(vars(DAG(dag_id='test')).keys())` class SerializedBaseOperator(BaseOperator, Serialization): _included_fields = list(vars(BaseOperator(task_id='test')).keys()) + [ '_dag', '_task_type', 'subdag', 'ui_color', 'ui_fgcolor', 'template_fields'] Currently we just use all fields of DAG and BaseOperator. (2) (nit) In '/tree' page, it creates a new dagbag instead of using the global dagbag (created once): dag_model = DagModel.get_dagmodel(dag_id) dag = dag_model.get_dag(DAGCACHED_ENABLED) '/graph' page uses the global dagbag. Maybe it is better to unify the behavior.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
