Andrushika commented on issue #75865: URL: https://github.com/apache/airflow/issues/75865#issuecomment-6483614606
I found a scheduler crash on Postgres that looks related to #70972. Since #70972, `asset_dag_run_queue` keeps one row for each asset event. When the scheduler creates an asset-triggered run, it puts all queued rows of the Dag into one `IN (...)`. With many rows, Postgres rejects the statement. The scheduler exits, and it fails again on each restart. Then no Dag gets scheduled. What I saw on `apache/airflow:3.4.0rc1` with Postgres 16. I sent the events with `POST /api/v2/assets/events`. - A Dag with `schedule=[a, b]` and 9,000 queued events fails with `stack depth limit exceeded`. - A single-asset Dag with `[scheduler] batch_asset_events = True` waits at `max_active_runs` while events arrive. With 9,000 queued events, it fails with the same error. - A single-asset Dag with default settings and 66,199 queued events fails with `number of parameters must be between 0 and 65535`. On 3.3.2, the same `[a, b]` Dag with 33,500 events works fine. --- Drafted-by: Claude Opus 5.5 (reviewed by me) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
