Hi all, First of all, thank you for all the comments in the proposal above. These feedbacks are very valuable and really helped me to move forward on that topic. Airflow multi-tenant model is definitely a vast project and I believe it is best to do in several AIPs to have clear scope and reduce the cognitive load on reviewers. For now, I can see at least 3 different AIPs:
1. View level access control: introduce "Tenant" entity and allow grouping of users 2. Introduction of DAG group (exact concept name TBD): allow grouping of DAGs and impose access restrictions 3. Resource level access control: Modify resources access policies based on Tenants We may decide to merge AIP 2 and 3 or add new ones depending on learnings from 1st one. To start off, I drafted an AIP on the view level access control, you can access it here: https://docs.google.com/document/d/1swNx_GTvUm456w8UKgQS1-CbGPu2OFr4l9lDaUnXbN8/edit?usp=sharing. If you agree with this proposal, would it be possible to have permissions to create an AIP? Thank you, Vincent From: "Mehta, Shubham" <shu...@amazon.com> Date: Tuesday, January 10, 2023 at 3:01 AM To: "dev@airflow.apache.org" <dev@airflow.apache.org> Cc: "Beck, Vincent" <vincb...@amazon.com>, "Mehta, Shubham" <shu...@amazon.com> Subject: Seeking Feedback for Airflow Multi-Tenant Model Proposal Hi folks, Over the past few weeks, @Vincent Beck<mailto:vincb...@amazon.com> and I have been working on a proposal for a multi-tenant model for Apache Airflow<https://docs.google.com/document/d/1n23h26p4_8F5-Cd0JGLPEnF3gumJ5hw3EpwUljz7HcE/edit?usp=sharing>. Building on AIP-43<https://cwiki.apache.org/confluence/display/AIRFLOW/AIP-43+DAG+Processor+separation> (DAG Processor separation) and AIP-44<https://cwiki.apache.org/confluence/display/AIRFLOW/AIP-44+Airflow+Internal+API> (Airflow Internal API), we aim to modify the existing Role-Based Access Control (RBAC) to provide fine-grained access control and pave the way for running Airflow in a multi-tenant fashion. Multi-tenancy support in Airflow would allow users to use a single Airflow environment to support multiple teams or business units, each with their own isolated workflows, user permissions, and data. This can offer a number of benefits including cost savings from a shared environment, improved collaboration among teams, and enhanced security through isolation, while also reducing the overall operational load. In the proposal, we outline user requirements and describe the design for view-level and resource-level access control. We intentionally did not include technical implementation details, as these will be covered in AIPs after alignment. The proposal also includes open questions and recommendations. We would like to thank Jarek, Filip, and Kaxil for providing early feedback, helping to ensure the design has no obvious flaws. Please review the proposal and provide your feedback by January 18th. We will then proceed to draft AIPs with implementation details based on the final proposal. Regards Shubham Mehta
