o-nikolas commented on PR #61646: URL: https://github.com/apache/airflow/pull/61646#issuecomment-3874462163
> > Regarding the questions: > > 1. Are queues guaranteed to be unique across teams? For example, could team_1 and team_2 both use a queue named "default"? If so, the current queue-based isolation would break down. --> Not an expert in the multi team setup ... @o-nikolas can you provide an answer? The `queue` param has always been an odd duck. Originally hardcoded into Airflow for the Celery Executor specifically, then abused for the old concrete hybrid executors (to specify which executor a task would be sent to). It's now used by only CeleryExecutor and EdgeExecutor (as far as I know). I would personally like to see this thing deprecated in favour of using the `executor_config`, but let's at least not abuse it yet again. It is not yet team-ified and I don't have any specific plans to do so. It is a property of a task, which is team-aware, so I'm not sure what benefit making it team aware would be or how that would look. Tasks will be sent to the correct executor instance for their team inside the scheduler well before the queue property is ever evaluated inside the executor (which is team aware_, so I'm not sure how it would help in this situation or what it would mean to make it team aware. > 3. Alternatively, would it be more appropriate to add a team_name column to EdgeJobModel / EdgeWorkerModel for explicit team-level filtering, rather than relying on queue-based inference? --> without knowing the answers from above... I assume so. Which would be a blocker until the DB migration tooling in integrated in Edge which is still pending from [Add EdgeDBManager for provider-specific database migrations #61155](https://github.com/apache/airflow/pull/61155) - before this not merged we should not extend the DB scheme. I believe this is the best answer. Let's not abuse queue again and implement this right :smile: -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
