[
https://issues.apache.org/jira/browse/HIVE-28259?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17932881#comment-17932881
]
Stamatis Zampetakis commented on HIVE-28259:
--------------------------------------------
A small video presentation about the work that was performed here can be found
in [youtube|https://www.youtube.com/watch?v=PHm5vZ1A43I&t=6317s]. The slides
(in pdf) are attached under this JIRA.
> Common table expression detection and rewrites using CBO
> --------------------------------------------------------
>
> Key: HIVE-28259
> URL: https://issues.apache.org/jira/browse/HIVE-28259
> Project: Hive
> Issue Type: New Feature
> Components: CBO
> Reporter: Stamatis Zampetakis
> Assignee: Stamatis Zampetakis
> Priority: Major
> Labels: pull-request-available
> Fix For: 4.1.0
>
> Attachments:
> optimizing-common-table-expressions-in-apache-hive-with-calcite.pdf
>
>
> Hive already provides the means to detect and exploit CTEs via the
> {{SharedWorkOptimizer}}. The {{SharedWorkOptimizer}} relies on a series of
> heuristic transformations of the physical plan ({{Operator}} DAG) that apply
> towards the end of the planning process. The optimizer is quite powerful and
> offers various properties ({{hive.optimize.shared.work*}}) through which its
> behavior can be fine tuned by users but has also a few drawbacks:
> * not cost-based
> * limited customization
> * complex implementation
> This ticket aims to leverage CBO for detecting and exploiting common table
> expressions (CTE) in queries in an attempt to alleviate some of the
> shortcomings of the {{SharedWorkOptimizer}} and open the road for more
> powerful transformations.
> The initial work focuses on establishing the general design and the main APIs
> for CBO based CTE transformations.
> The main idea is to model CTEs as materialized views (MVs) and rely on
> existing MV rewrite logic to incorporate them in the plan in a cost-based
> fashion as a new CBO planning phase.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
