[GitHub] [airflow] potiuk commented on issue #6515: [AIRFLOW-XXX] GSoD: How to make DAGs production ready
potiuk commented on issue #6515: [AIRFLOW-XXX] GSoD: How to make DAGs production ready URL: https://github.com/apache/airflow/pull/6515#issuecomment-551906055 > I was also thinking to actually place this doc in the main section and name is **Best Practices** or something similar. I like it too :) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] potiuk commented on issue #6515: [AIRFLOW-XXX] GSoD: How to make DAGs production ready
potiuk commented on issue #6515: [AIRFLOW-XXX] GSoD: How to make DAGs production ready URL: https://github.com/apache/airflow/pull/6515#issuecomment-551725808 Great @KKcorps It could also be great to search some of the existing documentation and see if there are no contradictions practices <> examples (I believe there are a few places where you have those bad practices shown as examples :). And we can fix them together. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] potiuk commented on issue #6515: [AIRFLOW-XXX] GSoD: How to make DAGs production ready
potiuk commented on issue #6515: [AIRFLOW-XXX] GSoD: How to make DAGs production ready URL: https://github.com/apache/airflow/pull/6515#issuecomment-551469402 @KKcorps I think one of the important things to mention in this document is database access (and especially avoiding to use Airflow Variables in the DAGs (they are still ok to use in "execute" method. This is a know problem that people are complaining a lot that scheduler opens and closes a lot of connections to the database - because every time the file is parsed, and variable is reached, a database connection is opened and query executed. I think there are lots of examples around with using variables in the DAGs but this is not really a good practice and I think this is a perfect place to mention it. @kaxil @ashb @mik-laj - what are your thoughts about it ? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
