Lee-W commented on code in PR #76991:
URL: https://github.com/apache/airflow/pull/76991#discussion_r4559259939
##########
airflow-core/docs/authoring-and-scheduling/assets.rst:
##########
@@ -730,6 +730,44 @@ including ``partition_key`` in the request body):
"partition_key": "us|2026-03-10T09:00:00"
}'
+.. _monitor-pending-asset-partitions:
+
+Monitoring pending partitioned runs
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+To surface partitioned consumers that have waited longer than you expect, set
+:ref:`[scheduler] asset_partition_pending_timeouts
<config:scheduler__asset_partition_pending_timeouts>`.
+The value is a JSON object mapping exact Dag IDs to positive, finite numbers
of seconds.
+The default, ``{}``, disables monitoring. For example, in ``airflow.cfg``:
+
+.. code-block:: ini
+
+ [scheduler]
+ asset_partition_pending_timeouts = {"clean_and_combine_player_stats": 7200}
Review Comment:
```suggestion
pending_partitioned_dag_run_timeout = {"clean_and_combine_player_stats":
7200}
```
this is more accurate. reading the original wording, i'm not able to get
what is actaully pending
##########
airflow-core/docs/authoring-and-scheduling/assets.rst:
##########
@@ -730,6 +730,44 @@ including ``partition_key`` in the request body):
"partition_key": "us|2026-03-10T09:00:00"
}'
+.. _monitor-pending-asset-partitions:
+
+Monitoring pending partitioned runs
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+To surface partitioned consumers that have waited longer than you expect, set
+:ref:`[scheduler] asset_partition_pending_timeouts
<config:scheduler__asset_partition_pending_timeouts>`.
+The value is a JSON object mapping exact Dag IDs to positive, finite numbers
of seconds.
+The default, ``{}``, disables monitoring. For example, in ``airflow.cfg``:
+
+.. code-block:: ini
+
+ [scheduler]
+ asset_partition_pending_timeouts = {"clean_and_combine_player_stats": 7200}
+
+Use the same setting on every scheduler. For each configured, active
partitioned Dag,
+the scheduler adds an ``asset partition pending`` warning when its oldest
pending-run
Review Comment:
same for the wording here
##########
airflow-core/docs/authoring-and-scheduling/assets.rst:
##########
@@ -730,6 +730,44 @@ including ``partition_key`` in the request body):
"partition_key": "us|2026-03-10T09:00:00"
}'
+.. _monitor-pending-asset-partitions:
+
+Monitoring pending partitioned runs
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+To surface partitioned consumers that have waited longer than you expect, set
+:ref:`[scheduler] asset_partition_pending_timeouts
<config:scheduler__asset_partition_pending_timeouts>`.
+The value is a JSON object mapping exact Dag IDs to positive, finite numbers
of seconds.
+The default, ``{}``, disables monitoring. For example, in ``airflow.cfg``:
+
+.. code-block:: ini
+
+ [scheduler]
+ asset_partition_pending_timeouts = {"clean_and_combine_player_stats": 7200}
Review Comment:
I didn't aware it's `{"asset_name": timeout}` till i read the
implementation, but asset is composed by `name, uri`, we can think of whether
there's a better way to express it. but the current `{"name": timeout}` is ok.
name is unique
##########
airflow-core/docs/authoring-and-scheduling/assets.rst:
##########
@@ -730,6 +730,44 @@ including ``partition_key`` in the request body):
"partition_key": "us|2026-03-10T09:00:00"
}'
+.. _monitor-pending-asset-partitions:
+
+Monitoring pending partitioned runs
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+To surface partitioned consumers that have waited longer than you expect, set
+:ref:`[scheduler] asset_partition_pending_timeouts
<config:scheduler__asset_partition_pending_timeouts>`.
+The value is a JSON object mapping exact Dag IDs to positive, finite numbers
of seconds.
+The default, ``{}``, disables monitoring. For example, in ``airflow.cfg``:
+
+.. code-block:: ini
+
+ [scheduler]
+ asset_partition_pending_timeouts = {"clean_and_combine_player_stats": 7200}
+
+Use the same setting on every scheduler. For each configured, active
partitioned Dag,
+the scheduler adds an ``asset partition pending`` warning when its oldest
pending-run
+row is strictly older than the configured threshold. The warning appears with
the
+Dag's other warnings in the UI and through ``GET /api/v2/dagWarnings`` in the
Review Comment:
which page in the UI or event a screenshot can help
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
