potiuk commented on issue #11708: URL: https://github.com/apache/airflow/issues/11708#issuecomment-925725607
This is a good idea to have GitSync for plugins, but I am afraid the solution where you keep your plugins together with DAGs is a very bad idea for Airflow 2, one that should be heavily discouraged. There are two reasons (actually the root cause is isolation and security): 1) Plugins should not be "modifiable" by DAG writers. Plugins give you more "powers" over Airlfow that DAGs. The Plugins can modify UI of airflow (similarly as providers - with connections) and they are executed in the context of scheduler (not in the context of FileProcessor as all DAGs are) and you should not have the same people being able to write DAGs and modify plugins. This is something that wil be only strenghtened in the future when we increase isolation of Ailrflow components and it's a prerequisite to future Airflow multi-tenancy. But more importantly: 2) In Airflow 2 DAGs are not available as mounted volume for Webserver in our Helm Chart (and they should NOT be). See the Official Helm Chart. This is precisely because of the increased isolation and the fact that webserver does not need them any more (it uses Serialized form of the DAGs from the DB). So if your Plugins modify the UI of Airflow, this solution will not work (unless you also additionally mount the DAG folder to webserver - but this violates the isolation that was improved in Airflow 2 - see point 1). Generally speaking - you SHOULD separate DAGs and Plugins. Access to modigy those two in many cases should be controlled differently for security reasons. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
