ephraimbuddy commented on PR #68517: URL: https://github.com/apache/airflow/pull/68517#issuecomment-6020830384
Pushed three fixes on top of this PR, following the dev-list agreement that batching should be opt-in: - **Opt-in batching:** `batch_asset_events` now defaults to the new `[scheduler] batch_asset_events` option (false), and setting it to true restores the old behaviour deployment-wide. - **AND/rollup exception:** `a & b` conditions and partition rollups can't run one event at a time, so they still default to batching when the argument isn't passed, and only an explicit `False` for them is rejected, which keeps existing Dags parsing. - **Bounded run creation:** with batching off, the events picked up per loop are capped by `max_dagruns_to_create_per_loop` and each Dag's `max_active_runs` headroom, and the rest stay queued. - **Partitioned pending rows:** the UI detail endpoint now takes a `partitioned_dag_run_id` (falling back to the oldest row), and enabling batching merges pending rows for the same key into the newest one without losing events. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
