[
https://issues.apache.org/jira/browse/DRILL-8555?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18125117#comment-18125117
]
ASF GitHub Bot commented on DRILL-8555:
---------------------------------------
letian-jiang commented on PR #3086:
URL: https://github.com/apache/drill/pull/3086#issuecomment-6063462661
> 8. Hard-coded allowlist of literal-only function operands
(PlanCacheParameterizer.java:142). Any function not on the list (format
strings, scale arguments, new UDFs) gets a slot and fails planning because of
https://github.com/apache/drill/pull/1. Could this come from operator operand
metadata instead?
This is a difficult problem, and I agree that a hard-coded list carries a
maintenance risk. Given Drill’s current architecture, though, I don’t see a
clearly better practical approach: DrillOptiq itself uses function-name
switches for value-dependent rewrites, so the parameterizer needs explicit
knowledge of which operands must remain literals. Moving this information into
function metadata could improve organization, but would still require manually
declaring and maintaining those rules. For now, I think keeping this list
aligned with the existing rewrites and adding targeted regression tests is a
reasonable approach, while acknowledging that it does not establish safety for
every SQL pattern.
> Physical plan cache for parameterized SQL queries
> -------------------------------------------------
>
> Key: DRILL-8555
> URL: https://issues.apache.org/jira/browse/DRILL-8555
> Project: Apache Drill
> Issue Type: New Feature
> Components: Query Planning & Optimization
> Reporter: Letian Jiang
> Priority: Major
>
> h2. Motivation
> Repeated queries that differ only in literals are a common workload, yet each
> query currently requires full planning. This adds unnecessary latency and
> wastes CPU time.
> h2. Brief Design
> Add a per-Drillbit, in-memory plan cache. Drill automatically parameterizes
> eligible SQL queries and reuses a cached physical plan with the current
> literal values, without changes to client queries.
> h2. Plan reuse safety
> * *Engine:* matches the SQL template and checks relevant option changes,
> plugin configuration changes, and whether base table versions still match.
> * *Storage plugin:* explicitly opts in, provides a plan-compatible table
> version, and is responsible for rebuilding its predicate-dependent scan work
> from the rebound expressions.
> h2. Expected benefit
> Short, frequently executed queries such as HBase reads can benefit
> substantially. Analytical queries with significant planning cost can benefit
> as well.
> h2. Implementation status
> A working implementation is available, including HBase, Paimon and Iceberg
> integrations. A pull request will follow.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
