[
https://issues.apache.org/jira/browse/DRILL-8529?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18014603#comment-18014603
]
Charles Givre commented on DRILL-8529:
--------------------------------------
One more thing. Recently, I added a PR which caches the schemata from the
Splunk plugin. I found in that case was that the schemata from Splunk really
don't change and that simply retrieving schema metadata was the bottleneck in
repeated Splunk queries. As a part of this, it might be worthwhile also to
incorporate some sort of schema caching for tables that don't change as well.
I've thought about doing this for the JDBC plugin since database schemata are
also very unlikely to change.
> Caching QueryPlan Results
> -------------------------
>
> Key: DRILL-8529
> URL: https://issues.apache.org/jira/browse/DRILL-8529
> Project: Apache Drill
> Issue Type: Improvement
> Components: Query Planning & Optimization
> Reporter: Vincent de Gans
> Priority: Minor
> Fix For: Future
>
>
> I propose introducing a caching mechanism for the output of `getQueryPlan()`
> in cases where:
> - The input SQL query is the same as a previously seen one
> - The schema or relevant metadata used for planning has not changed
> - The cached result has not expired, based on a configurable time-to-live
> (TTL)
> Proposed Caching Features
> - Toggle to enable or disable query plan caching
> - Configurable TTL-based invalidation
> - Optional schema metadata verification to detect changes in underlying data
> sources
> Motivation
> - Reduce planner overhead for repeated queries
> - Improve response times in environments where query plans are reused
> - Provide an optional optimization that can be enabled when beneficial
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
