[
https://issues.apache.org/jira/browse/DRILL-8555?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18125154#comment-18125154
]
ASF GitHub Bot commented on DRILL-8555:
---------------------------------------
letian-jiang commented on PR #3086:
URL: https://github.com/apache/drill/pull/3086#issuecomment-6064744201
I’ve rerun both benchmarks on the latest commit,
[[6ee1c592b](https://github.com/letian-jiang/drill/commit/6ee1c592b66c7e5ff16abcf7e6b0a16bf5b9dc41)](https://github.com/letian-jiang/drill/commit/6ee1c592b66c7e5ff16abcf7e6b0a16bf5b9dc41),
using the same machine, datasets, queries and sampling method as the original
measurements.
| Workload | Planning off → hit | Reduction | End-to-end off → hit |
Reduction |
|---|---:|---:|---:|---:|
| HBase point read | 48 → 14 ms | 70.8% | 63 → 29 ms | 54.0% |
| HBase range scan | 40 → 13 ms | 67.5% | 54 → 26 ms | 51.9% |
| HBase column filter | 34 → 10 ms | 70.6% | 47 → 22 ms | 53.2% |
| Iceberg TPC-H SF0.01 | 1,950 → 616 ms | 68.4% | 18,691 → 17,101 ms | 8.5% |
> Physical plan cache for parameterized SQL queries
> -------------------------------------------------
>
> Key: DRILL-8555
> URL: https://issues.apache.org/jira/browse/DRILL-8555
> Project: Apache Drill
> Issue Type: New Feature
> Components: Query Planning & Optimization
> Reporter: Letian Jiang
> Priority: Major
>
> h2. Motivation
> Repeated queries that differ only in literals are a common workload, yet each
> query currently requires full planning. This adds unnecessary latency and
> wastes CPU time.
> h2. Brief Design
> Add a per-Drillbit, in-memory plan cache. Drill automatically parameterizes
> eligible SQL queries and reuses a cached physical plan with the current
> literal values, without changes to client queries.
> h2. Plan reuse safety
> * *Engine:* matches the SQL template and checks relevant option changes,
> plugin configuration changes, and whether base table versions still match.
> * *Storage plugin:* explicitly opts in, provides a plan-compatible table
> version, and is responsible for rebuilding its predicate-dependent scan work
> from the rebound expressions.
> h2. Expected benefit
> Short, frequently executed queries such as HBase reads can benefit
> substantially. Analytical queries with significant planning cost can benefit
> as well.
> h2. Implementation status
> A working implementation is available, including HBase, Paimon and Iceberg
> integrations. A pull request will follow.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
