This is an automated email from the ASF dual-hosted git repository.
vincbeck pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/airflow.git
The following commit(s) were added to refs/heads/main by this push:
new d3ea3ef0bcc Add documentation for team-based asset event filtering
(#65690)
d3ea3ef0bcc is described below
commit d3ea3ef0bccb23786e5b69a5534806ed6ed67c5e
Author: Vincent <[email protected]>
AuthorDate: Thu May 7 10:44:38 2026 -0400
Add documentation for team-based asset event filtering (#65690)
---
.../docs/authoring-and-scheduling/assets.rst | 47 +++++++
airflow-core/docs/core-concepts/multi-team.rst | 139 +++++++++++++++++++--
.../example_dags/example_asset_allow_teams.py | 75 +++++++++++
docs/spelling_wordlist.txt | 2 +
4 files changed, 256 insertions(+), 7 deletions(-)
diff --git a/airflow-core/docs/authoring-and-scheduling/assets.rst
b/airflow-core/docs/authoring-and-scheduling/assets.rst
index a764bc33449..8e5cf356394 100644
--- a/airflow-core/docs/authoring-and-scheduling/assets.rst
+++ b/airflow-core/docs/authoring-and-scheduling/assets.rst
@@ -402,6 +402,53 @@ As mentioned in :ref:`Fetching information from previously
emitted asset events<
events = inlet_events[AssetAlias("example-alias")]
last_row_count = events[-1].extra["row_count"]
+.. _asset_allow_producer_teams:
+
+Cross-team asset event filtering with ``allow_producer_teams``
+--------------------------------------------------------------
+
+.. versionadded:: 3.3.0
+
+When :doc:`Multi-Team mode </core-concepts/multi-team>` is enabled, asset
events are filtered by team
+membership. By default, a consuming Dag only receives asset events produced by
Dags within the same team
+or by global (teamless) Dags. This prevents unintended cross-team triggers.
+
+To allow specific other teams to produce events that trigger your Dag, use the
``allow_producer_teams`` parameter
+on the ``Asset`` definition:
+
+.. code-block:: python
+
+ from airflow.sdk import Asset
+
+ shared_data = Asset(
+ name="my_data",
+ uri="s3://bucket/shared/data.csv",
+ allow_producer_teams=["team_analytics", "team_ml"],
+ )
+
+In this example, asset events produced by Dags belonging to ``team_analytics``
or ``team_ml`` will be
+accepted by any consuming Dag that schedules on ``shared_data``, in addition
to events from the consuming
+Dag's own team.
+
+Default behavior
+~~~~~~~~~~~~~~~~
+
+When ``allow_producer_teams`` is not specified (or set to an empty list), the
default same-team filtering applies.
+The rules depend on whether the producer and consumer have a team association:
+
+- **Both have the same team**: The event is always delivered.
+- **Producer has a team, consumer has a different team**: The event is blocked
(unless the
+ producer's team is in the asset's ``allow_producer_teams``).
+- **Producer has no team (global Dag)**: The event is delivered to all
consumers, regardless of
+ the consumer's team. Global Dags act as shared infrastructure that any team
can depend on.
+- **Consumer has no team (global Dag)**: The consumer accepts events from any
source,
+ regardless of the producer's team. Teamless consumers act as shared
infrastructure that any
+ team can feed into.
+- **Neither has a team**: The event is delivered (both are global).
+
+When Multi-Team mode is disabled, ``allow_producer_teams`` is ignored and all
asset events are delivered to all
+consuming Dags, preserving backward compatibility.
+
Asset partitions
----------------
diff --git a/airflow-core/docs/core-concepts/multi-team.rst
b/airflow-core/docs/core-concepts/multi-team.rst
index 609a79cdf18..88a99e2be40 100644
--- a/airflow-core/docs/core-concepts/multi-team.rst
+++ b/airflow-core/docs/core-concepts/multi-team.rst
@@ -503,16 +503,130 @@ When Multi-Team mode is enabled, the scheduler performs
additional logic to dete
teams share the same metadata database and common Airflow infrastructure.
For absolutely strict security
requirements, consider separate Airflow deployments.
-Architecture
-------------
+.. _multi-team-asset-event-filtering:
-The following diagram shows a Multi-Team Airflow deployment with resource
isolation between teams.
+Team-Based Asset Event Filtering
+---------------------------------
-The components in blue are the shared components and those in green are the
team components (note the green shadow box
-indicating more than one team is present in the architecture).
+When Multi-Team mode is enabled, asset events are filtered by team membership
before they trigger
+downstream Dag runs. This prevents asset events produced by one team's Dags
from unintentionally
+triggering Dag runs for a different team.
-.. image:: /img/multi_team_arch_diagram.png
- :alt: Multi-Team Airflow Architecture showing resource isolation between
teams
+Default Behavior
+^^^^^^^^^^^^^^^^
+
+By default, a consuming Dag only receives asset events from producers within
the same team or from Dags with no team association, i.e. global Dags.
+
+Cross-Team Opt-In with ``allow_producer_teams``
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+To allow specific teams to produce events that trigger consumers on a given
asset from another team, use the
+``allow_producer_teams`` parameter on the ``Asset`` definition:
+
+.. code-block:: python
+
+ from airflow.sdk import Asset
+
+ shared_data = Asset(
+ name="shared_data",
+ uri="s3://bucket/shared/data.csv",
+ allow_producer_teams=["team_analytics", "team_ml"],
+ )
+
+With this configuration, asset events from ``team_analytics`` or ``team_ml``
will be accepted by any
+consuming Dag that schedules on ``shared_data``, in addition to events from
the consumer's own team.
+
+See :ref:`Cross-team asset event filtering with allow_producer_teams
<asset_allow_producer_teams>` in the Assets
+documentation for usage details and validation rules.
+
+Behavioral Rules
+^^^^^^^^^^^^^^^^
+
+The following table describes the complete filtering logic:
+
+.. list-table::
+ :header-rows: 1
+ :widths: 20 20 20 15 25
+
+ * - Producer
+ - Consumer
+ - ``allow_producer_teams``
+ - Result
+ - Reason
+ * - Team A (DAG)
+ - Team A
+ - (any)
+ - ✅ Allowed
+ - Same team
+ * - Team A (DAG)
+ - Team B
+ - ``[]``
+ - ❌ Blocked
+ - Different team, no opt-in
+ * - Team A (DAG)
+ - Team B
+ - ``["team_a"]``
+ - ✅ Allowed
+ - Cross-team opt-in
+ * - (no team, DAG)
+ - Team B
+ - (any)
+ - ✅ Allowed
+ - Global producer
+ * - Team A (DAG)
+ - (no team)
+ - (any)
+ - ✅ Allowed
+ - Teamless consumer accepts events from any DAG producer
+ * - (no team, DAG)
+ - (no team)
+ - (any)
+ - ✅ Allowed
+ - Both global
+ * - Team A (API)
+ - Team A
+ - (any)
+ - ✅ Allowed
+ - Same team
+ * - Team A (API)
+ - Team B
+ - ``["team_a"]``
+ - ✅ Allowed
+ - Cross-team opt-in
+ * - Team A (API)
+ - (no team)
+ - (any)
+ - ✅ Allowed
+ - Teamless consumer accepts events from any source
+ * - (no team, API)
+ - Team B
+ - (any)
+ - ❌ Blocked
+ - Teamless API user cannot trigger team-bound consumer
+ * - (no team, API)
+ - (no team)
+ - (any)
+ - ✅ Allowed
+ - Both global
+
+Key rules:
+
+- **Same team**: Always allowed.
+- **Global (teamless) DAG producer**: Triggers all consumers regardless of
team.
+- **Teamless API user**: Can only trigger teamless consumers. Unlike a
teamless DAG — which is
+ deployed by a platform operator and intentionally shared — an API user
without a team has no
+ verified team affiliation, so their events are restricted to teamless
consumers to
+ prevent unscoped access to team-bound pipelines.
+- **Teamless consumer**: Accepts events from any source (DAG or API),
regardless of team.
+- **Cross-team via** ``allow_producer_teams``: Allowed when the producer's
team is listed in the asset's ``allow_producer_teams``.
+- **Multi-Team disabled**: All filtering is skipped; existing behavior is
preserved.
+
+API-Triggered Events
+^^^^^^^^^^^^^^^^^^^^
+
+When a user creates an asset event via the REST API, the user's team is
resolved from the auth manager.
+The same filtering rules apply, with one distinction: a teamless API user can
only trigger teamless
+consumers, whereas a teamless DAG producer is treated as global and can
trigger any consumer.
Important Considerations
------------------------
@@ -535,3 +649,14 @@ Global Uniqueness of Identifiers
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
**Dag IDs, Variable keys, and Connection IDs must be unique across the entire
Airflow deployment**, regardless of which team owns them. This is similar to
how S3 bucket names are globally unique across all AWS accounts. You should
establish naming conventions within your organization to avoid naming conflicts
(e.g. prefix identifiers with the team name)
+
+Architecture
+------------
+
+The following diagram shows a Multi-Team Airflow deployment with resource
isolation between teams.
+
+The components in blue are the shared components and those in green are the
team components (note the green shadow box
+indicating more than one team is present in the architecture).
+
+.. image:: /img/multi_team_arch_diagram.png
+ :alt: Multi-Team Airflow Architecture showing resource isolation between
teams
diff --git a/airflow-core/src/airflow/example_dags/example_asset_allow_teams.py
b/airflow-core/src/airflow/example_dags/example_asset_allow_teams.py
new file mode 100644
index 00000000000..0dc71db8d20
--- /dev/null
+++ b/airflow-core/src/airflow/example_dags/example_asset_allow_teams.py
@@ -0,0 +1,75 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+"""
+Example DAG demonstrating cross-team asset triggering with
``allow_producer_teams``.
+
+When Multi-Team mode is enabled (``[core] multi_team = True``), asset events
are filtered by team
+membership. By default, a consuming DAG only receives events from DAGs within
the same team.
+
+Usage:
+ - ``team_analytics_producer`` (belonging to ``team_analytics``) produces
events on ``shared_data``.
+ - ``team_ml_consumer`` (belonging to ``team_ml``) consumes ``shared_data``.
+ - Because ``shared_data`` has ``allow_producer_teams=["team_analytics"]``,
events from ``team_analytics``
+ are accepted by ``team_ml_consumer``.
+ - Without ``allow_producer_teams``, the cross-team event would be blocked.
+"""
+
+from __future__ import annotations
+
+import pendulum
+
+from airflow.providers.standard.operators.bash import BashOperator
+from airflow.sdk import DAG, Asset
+
+# [START asset_allow_producer_teams]
+# Define an asset that accepts events from team_analytics in addition to the
consumer's own team.
+shared_data = Asset(
+ name="shared_data",
+ uri="s3://data-lake/shared/output.csv",
+ allow_producer_teams=["team_analytics"],
+)
+
+# Producer DAG — belongs to team_analytics (via its DAG bundle configuration).
+# When this DAG's task completes, it emits an asset event for shared_data.
+with DAG(
+ dag_id="team_analytics_producer",
+ start_date=pendulum.datetime(2024, 1, 1, tz="UTC"),
+ schedule="@daily",
+ catchup=False,
+ tags=["team_analytics", "produces", "asset-scheduled", "allow-teams"],
+) as producer_dag:
+ BashOperator(
+ task_id="produce_shared_data",
+ outlets=[shared_data],
+ bash_command="echo 'Producing shared data for cross-team consumption'",
+ )
+
+# Consumer DAG — belongs to team_ml (via its DAG bundle configuration).
+# This DAG is triggered when shared_data is updated. Because shared_data has
+# allow_producer_teams=["team_analytics"], events from team_analytics are
accepted.
+with DAG(
+ dag_id="team_ml_consumer",
+ start_date=pendulum.datetime(2024, 1, 1, tz="UTC"),
+ schedule=[shared_data],
+ catchup=False,
+ tags=["team_ml", "consumes", "asset-scheduled", "allow-teams"],
+) as consumer_dag:
+ BashOperator(
+ task_id="consume_shared_data",
+ bash_command="echo 'Consuming shared data from team_analytics'",
+ )
+# [END asset_allow_producer_teams]
diff --git a/docs/spelling_wordlist.txt b/docs/spelling_wordlist.txt
index 8d6413a2885..b4460b21e75 100644
--- a/docs/spelling_wordlist.txt
+++ b/docs/spelling_wordlist.txt
@@ -1637,6 +1637,7 @@ tbuild
TCP
tcp
tdload
+teamless
teardown
teardowns
templatable
@@ -1747,6 +1748,7 @@ unpaused
unpausing
unpredicted
unsanitized
+unscoped
untestable
untransformed
untrusted
