justinpakzad commented on code in PR #69768:
URL: https://github.com/apache/airflow/pull/69768#discussion_r3566672795
##########
airflow-core/src/airflow/dag_processing/dagbag.py:
##########
@@ -161,6 +162,30 @@ def _validate_executor_fields(dag: DAG, bundle_name: str |
None = None) -> None:
)
+def _assign_default_team_pools(
+ dag: DAG,
+ bundle_name: str | None = None,
+) -> None:
+ """Assign the default team pool to tasks that do not explicitly specify a
pool."""
+ dag_team_name = None
+
+ if conf.getboolean("core", "multi_team"):
+ if bundle_name:
+ from airflow.dag_processing.bundles.manager import
DagBundlesManager
+
+ bundle_manager = DagBundlesManager()
+ bundle_config = bundle_manager._bundle_config[bundle_name]
+
+ dag_team_name = bundle_config.team_name
+
+ if not dag_team_name:
+ return
+
+ for task in dag.tasks:
+ if task.pool == Pool.DEFAULT_POOL_NAME:
+ task.pool = Pool.get_default_team_pool_name(dag_team_name)
Review Comment:
The addition to the `team create` command would only create the default
pools for new teams going forward. That would mean for already existing teams
(without explicitly assigned pools), their tasks would be assigned a pool that
doesn't actually exist. Users would need to manually create the team default
pools. I know multi-team is still "experimental" and relatively new, so this
might not be an issue, but worth flagging.
Another potential edge case is if a user explicitly assigns the global
default pool to a task, this would actually overwrite it with the default team
pool. Probably not very common, but worth mentioning as this would effectively
mean that the default global pool would be unreachable for team Dags.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
