dosubot[bot] commented on issue #40903: URL: https://github.com/apache/superset/issues/40903#issuecomment-4659527397
<!-- Answer --> This looks like a race condition introduced by the combination of two recent PRs: 1. **[PR #40194](https://github.com/apache/superset/pull/40194)** (merged May 18) — added dynamic `connect` event listeners on engines to execute prequeries (e.g., `SET search_path`). These listeners are registered per-request and removed in a `finally` block . 2. **[PR #40237](https://github.com/apache/superset/pull/40237)** (merged May 30) — introduced process-wide SQLAlchemy engine caching so the same `Engine` instance is returned to multiple threads . The problem: when multiple threads share a cached engine, one thread can call `sqla.event.listen(engine, "connect", ...)` or `sqla.event.remove(engine, "connect", ...)` while another thread is iterating over `self.listeners` inside SQLAlchemy's `_exec_w_sync_on_first_run` during `raw_connection()`. That mutates the underlying deque mid-iteration → `RuntimeError: deque mutated during iteration`. The cache access itself is protected by `_ENGINE_CACHE_LOCK`, but the event listener registration/removal is **not** under the same (or any) lock . **Possible workarounds until a fix lands:** - Revert PR #40237 (engine caching) to confirm it's the culprit and eliminate the error. - Or set `SQLALCHEMY_POOL_SIZE = 0` / use `NullPool` for the target database to reduce concurrent connection creation contention. A proper fix would likely involve either synchronizing listener add/remove with a lock, or registering prequery listeners once at engine-creation time (inside the cache-miss path) instead of dynamically per-request. <!-- Dosu Comment Footer --> *To reply, just mention [@dosu](https://go.dosu.dev/dosubot?utm_source=github&utm_medium=bot-comment&utm_campaign=github-comment-footer-20260415&utm_content=reply-with-mention&utm_term=apache%2Fsuperset).* --- Share context across your team and agents. Try [Dosu](https://dosu.dev?utm_source=github&utm_medium=bot-comment&utm_campaign=github-comment-footer-20260415&utm_content=knowledge-infrastructure-tagline&utm_term=apache%2Fsuperset). [](https://app.dosu.dev/response-feedback?utm_source=github&utm_medium=bot-comment&utm_campaign=github-comment-footer-20260415&utm_content=knowledge-infrastructure-feedback&utm_term=apache%2Fsuperset&message_id=0745d31b-63da-41f1-a5a1-20294f61c1ca) [](https://github.dosu.com/apache/superset?utm_source=github&utm_medium=bot-comment&utm_campaign=github-comment-footer-20260415&utm_content=knowledge-infrastructure-learn-repo&utm_term=apache%2Fsuperset) [](https://app.dosu.dev/signup?referrer=openSource&source=github-footer&utm_source=github&utm_medium=bot-comment&utm_campaign=github-comment-footer-20260415&utm_content=knowledge-infrastructure-add-team&utm_term=apache%2Fsuperset) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
