[
https://issues.apache.org/jira/browse/HIVE-28816?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17934593#comment-17934593
]
Stamatis Zampetakis commented on HIVE-28816:
--------------------------------------------
To fix the inconsistency either we have to update the description of the
property or modify the code which uses the property to use a strict inequality.
Apart from the SemanticAnalyzer, there are a few other places that make use of
the hive.optimize.cte.materialize.threshold and there strict inequality is used
so there is a bit of inconsistency even within different code paths in the
compiler.
> hive.optimize.cte.materialize.threshold behavior is inconsistent with its
> description
> -------------------------------------------------------------------------------------
>
> Key: HIVE-28816
> URL: https://issues.apache.org/jira/browse/HIVE-28816
> Project: Hive
> Issue Type: Bug
> Affects Versions: 4.0.1
> Reporter: Stamatis Zampetakis
> Assignee: Stamatis Zampetakis
> Priority: Major
>
> The description of the {{hive.optimize.cte.materialize.threshold}} property
> is shown below.
> {code:java}
> If the number of references to a CTE clause exceeds this threshold, Hive will
> materialize it before executing the main query block.
> {code}
> However, in some cases Hive will materialize a CTE even when the number of
> references to the CTE clause is exactly *equal* to the threshold.
> The respective snippet from
> [SemanticAnalyzer.java|https://github.com/apache/hive/blob/38dc6b61a695a799c447d6f90503742dcc22e7bb/ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java#L2274]
> is shown below.
> {code:java}
> if (threshold >= 0 && cte.reference >= threshold) {
> cte.materialize = !HiveConf.getBoolVar(conf,
> ConfVars.HIVE_CTE_MATERIALIZE_FULL_AGGREGATE_ONLY)
> || cte.qbExpr.getQB().getParseInfo().isFullyAggregate();
> }
> {code}
> In most cases the phrase "exceeds" a threshold stands for strict inequality
> (i.e., greater than the threshold) so in this case the description of the
> property is not consistent with the behavior of Hive.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
