kaxil commented on issue #5743: [AIRFLOW-5088][AIP-24] Persisting serialized 
DAG in DB for webserver scalability
URL: https://github.com/apache/airflow/pull/5743#issuecomment-530785443
 
 
   Few things to do:
   
   - [ ] Add 
https://github.com/astronomer/airflow/commit/baf12f626e6d56dfde735faaed71b2c30cb4befb
 and add tests for it
   - [ ] Reduce the info we store in Serialized DAGs by removing all the 
default arguments that are not overridden by users. Eg `owner` in DAG & Task 
etc. This will help reduce blob size as well as reduce the time spent in 
`_deserialise` method.
   - [ ] Agree / dis-agree on using 
https://pypi.org/project/SQLAlchemy-JSONField/ instead of our code . It also 
has a nice option of specifying json library as compared to providing that info 
in the `create_engine.json_serializer` and `create_engine.json_deserializer` 
parameters in  
https://docs.sqlalchemy.org/en/13/core/type_basics.html#sqlalchemy.types.JSON
   
   cc @coufon @ashb 
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

Reply via email to