ephraimbuddy opened a new pull request, #77254:
URL: https://github.com/apache/airflow/pull/77254

   `test_migration_stairway` fails on every MySQL and Postgres DB-core job on 
main. The stairway runs out of connections partway through (`FATAL: sorry, too 
many clients already` on Postgres, `Too many connections` on MySQL), fails 
after its retries, and leaves the database mid-migration for the tests that 
follow.
   
   The stairway fixture asked for NullPool with 
`configure_orm(disable_connection_pool=True)`. When `upgradedb()` leaves 
`_single_connection_pool()`, it rebuilds the ORM from config, so from the first 
step on every engine got the default pool. The session and migration-lock 
connection that `upgradedb()` keeps using stay bound to the engine that rebuild 
disposed. Using them afterwards re-creates a pool on that engine, and nothing 
disposes it again. Each step leaves about one idle connection that closes only 
when a full garbage collection frees the engine. In the long-running "Other" 
test process, full collections are rare enough that the stairway reaches the 
server's connection limit.
   
   Setting NullPool through `[database] sql_alchemy_pool_enabled` makes every 
rebuild keep it, so those connections close as soon as they are returned. The 
per-revision NullPool re-apply in the test loop is no longer needed.
   
   The leak predates the commits after which CI started failing; those commits 
only changed when garbage collection happens to run in that process.
   
   Run with `uv run` against local Postgres 14 and MySQL 8.0 servers, with full 
garbage collection suppressed to match CI, and on SQLite:
   
   | Backend | Before | After |
   |---|---|---|
   | Postgres 14 | idle connections grow by ~1 per step; fails at step ~95 with 
the CI error | flat at 3 for all 267 steps; passes |
   | MySQL 8.0 | 63 idle connections at step 60 | flat at 3 for all 267 steps; 
passes |
   | SQLite | not run | passes |
   
   ---
   
   ##### Was generative AI tooling used to co-author this PR?
   
   - [X] Yes — Claude Code (Opus 5.5)
   
   Generated-by: Claude Code (Opus 5.5) following [the 
guidelines](https://github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#gen-ai-assisted-contributions)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to