[
https://issues.apache.org/jira/browse/HIVE-11405?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14646863#comment-14646863
]
Gopal V commented on HIVE-11405:
--------------------------------
thanks [~hsubramaniyan], I'm currently bypassing that with a temporary band-aid
which needs attention for correctness
{code}
---
a/ql/src/java/org/apache/hadoop/hive/ql/optimizer/stats/annotation/StatsRulesProcFactory.java
+++
b/ql/src/java/org/apache/hadoop/hive/ql/optimizer/stats/annotation/StatsRulesProcFactory.java
@@ -325,9 +325,16 @@ private long evaluateExpression(Statistics stats,
ExprNodeDesc pred,
}
} else if (udf instanceof GenericUDFOPOr) {
// for OR condition independently compute and update stats
- for (ExprNodeDesc child : genFunc.getChildren()) {
- newNumRows = StatsUtils.safeAdd(
- evaluateChildExpr(stats, child, aspCtx, neededCols, fop),
newNumRows);
+ newNumRows = stats.getNumRows();
+ Statistics orStats = stats.clone();
+ int k = 0;
+
+ for (ExprNodeDesc child :
com.google.common.collect.Lists.reverse(genFunc.getChildren())) {
+ final long branchRows = evaluateChildExpr(orStats, child, aspCtx,
neededCols, fop);
+ final long branch2Rows = (newNumRows <= branchRows) ? 0 :
(newNumRows - branchRows);
+ updateStats(orStats, branch2Rows, true, fop);
+ newNumRows = StatsUtils.safeAdd(branchRows, newNumRows);
}
} else if (udf instanceof GenericUDFOPNot) {
newNumRows = evaluateNotExpr(stats, pred, aspCtx, neededCols, fop);
{code}
> Add early termination for recursion in
> StatsRulesProcFactory$FilterStatsRule.evaluateExpression for OR expression
> ------------------------------------------------------------------------------------------------------------------
>
> Key: HIVE-11405
> URL: https://issues.apache.org/jira/browse/HIVE-11405
> Project: Hive
> Issue Type: Bug
> Reporter: Hari Sankar Sivarama Subramaniyan
> Assignee: Prasanth Jayachandran
>
> Thanks to [~gopalv] for uncovering this issue as part of HIVE-11330. Quoting
> him,
> "The recursion protection works well with an AND expr, but it doesn't work
> against
> (OR a=1 (OR a=2 (OR a=3 (OR ...)
> since the for the rows will never be reduced during recursion due to the
> nature of the OR.
> We need to execute a short-circuit to satisfy the OR properly - no case which
> matches a=1 qualifies for the rest of the filters.
> Recursion should pass in the numRows - branch1Rows for the branch-2."
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
