[
https://issues.apache.org/jira/browse/DRILL-8529?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18014601#comment-18014601
]
Charles Givre commented on DRILL-8529:
--------------------------------------
Hi Vincent,
Thanks for the idea. I think this could be an excellent contribution and am
all for it.
Before you dive into this, do take a look at the Drill metastore
([https://github.com/apache/drill/tree/master/metastore/iceberg-metastore]) as
this was an attempt to cache schema data for file based queries.
If I can assist, please let me know.
> Caching QueryPlan Results
> -------------------------
>
> Key: DRILL-8529
> URL: https://issues.apache.org/jira/browse/DRILL-8529
> Project: Apache Drill
> Issue Type: Improvement
> Components: Query Planning & Optimization
> Reporter: Vincent de Gans
> Priority: Minor
> Fix For: Future
>
>
> I propose introducing a caching mechanism for the output of `getQueryPlan()`
> in cases where:
> - The input SQL query is the same as a previously seen one
> - The schema or relevant metadata used for planning has not changed
> - The cached result has not expired, based on a configurable time-to-live
> (TTL)
> Proposed Caching Features
> - Toggle to enable or disable query plan caching
> - Configurable TTL-based invalidation
> - Optional schema metadata verification to detect changes in underlying data
> sources
> Motivation
> - Reduce planner overhead for repeated queries
> - Improve response times in environments where query plans are reused
> - Provide an optional optimization that can be enabled when beneficial
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
