dosubot[bot] commented on issue #34992: URL: https://github.com/apache/superset/issues/34992#issuecomment-3246611193
<!-- Greeting --> Hi @maxgruber19! I'm [Dosu](https://go.dosu.dev/dosubot) and Iām helping the superset team. <!-- Answer --> This behavior happens because Superset, when introspecting Trino/Iceberg tables for schema info, issues partition metadata queries like `SELECT * FROM ...$partitions` as part of its engine-specific metadata fetching logic. The reason you see three identical queries is that Superset's schema introspection calls several metadata routines in sequence (such as fetching columns, indexes, and keys), each of which may trigger the same expensive partition query via SQLAlchemy reflection and the Trino engine spec. This is not unique to your setup and is a side effect of how Superset gathers table metadata for complex backends like Trino/Iceberg[^1]. The lingering "FINISHING" state in Trino is due to Superset fetching the results and then closing the connection without explicitly closing or cancelling the Trino query. Trino expects the client to either fetch all results or explicitly close the query; otherwise, it keeps the query alive until its timeout is reached. Superset relies on SQLAlchemy/DBAPI context managers for cleanup, which may not force a query kill on the Trino side, especially for large result sets or metadata queries[^1]. There is currently no explicit mechanism in Superset to close or cancel these metadata queries after schema introspection, and no recent changes in the codebase address this. The issue is more pronounced with large-partition Iceberg tables, as the partition metadata queries are expensive and can quickly exhaust Trino's resource pools if not cleaned up promptly[^1]. If this is causing operational issues, you might consider: - Reducing the number of partitions in your Iceberg tables if possible. - Increasing Trino's resource pool limits or lowering the query timeout. - Exploring if you can patch Superset's Trino engine spec to avoid redundant partition queries or to explicitly cancel them after fetching metadata. [^1]: [Superset TrinoEngineSpec and metadata query handling](https://github.com/apache/superset/blob/829e4d92d91ceae4b43b1ed3b063ffe45377799c/superset/db_engine_specs/trino.py) <!-- Dosu Comment Footer --> *To reply, just mention [@dosu](https://go.dosu.dev/dosubot).* --- <sup>How did I do? [Good](https://app.dosu.dev/response-feedback/c08cf6b2-a0d7-42e3-a2b2-956dff67b0fb?feedback_type=great_response) | [Irrelevant](https://app.dosu.dev/response-feedback/c08cf6b2-a0d7-42e3-a2b2-956dff67b0fb?feedback_type=irrelevant_answer) | [Incorrect](https://app.dosu.dev/response-feedback/c08cf6b2-a0d7-42e3-a2b2-956dff67b0fb?feedback_type=incorrect_sources) | [Verbose](https://app.dosu.dev/response-feedback/c08cf6b2-a0d7-42e3-a2b2-956dff67b0fb?feedback_type=too_verbose) | [Hallucination](https://app.dosu.dev/response-feedback/c08cf6b2-a0d7-42e3-a2b2-956dff67b0fb?feedback_type=hallucination) | [Report š](https://app.dosu.dev/response-feedback/c08cf6b2-a0d7-42e3-a2b2-956dff67b0fb?feedback_type=bug_report) | [Other](https://app.dosu.dev/response-feedback/c08cf6b2-a0d7-42e3-a2b2-956dff67b0fb?feedback_type=other)</sup> [](https://app.dosu.dev/a28d3c7e-a9d3-459e-9fb6-3a6f9ff4f357/ask?utm_source=githu b) [](https://go.dosu.dev/discord-bot) [](https://twitter.com/intent/tweet?text=%40dosu_ai%20helped%20me%20solve%20this%20issue!&url=https%3A//github.com/apache/superset/issues/34992) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
