thomasrebele commented on code in PR #6601:
URL: https://github.com/apache/hive/pull/6601#discussion_r3580218685
##########
ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/rules/HiveFilterProjectTransposeRule.java:
##########
@@ -338,8 +338,18 @@ public void visit(RelNode node, int ordinal, RelNode
parent) {
if (node instanceof Filter) {
check((Filter) node);
} else if (node instanceof Project) {
+ Project project = (Project) node;
+ if (RexOver.containsOver(project.getProjects(), null)) {
+ // RelOptUtil.pushPastProjectUnlessBloat may rewrite RexInputRefs
inside RexOver
+ // window specs using the Project expression list. Window
partition/order
+ // expressions reference the Project input row type, so their
indexes may fall
+ // outside project.getProjects(). This visitor is only a
conservative redundancy
+ // check for IS NOT NULL predicates; if it cannot safely inspect
through a
+ // windowing Project, stop the check instead of failing query
planning.
+ return;
+ }
RexNode condition = HiveRelOptUtil.pushPastProjectUnlessBloat(
- filterCondition, (Project) node, bloat);
+ filterCondition, project, bloat);
if (condition != null) {
filterCondition = condition;
Review Comment:
I did a bit of debugging, and the problem might be around here.
pushPastProjectUnlessBloat checks for OVER and returns null. The
filterCondition is not updated, so it continues to hold the filter condition of
the parent.
For the q test file, the filterCondition corresponding to the parent is `IS
NOT NULL(row_number() OVER (PARTITION BY 0 ORDER BY $1 NULLS LAST RANGE BETWEEN
UNBOUNDED PRECEDING AND UNBOUNDED FOLLOWING))`. This will be also applied to
the child, which is `HiveProject(_o__c0=[$0])`. So a project with just one
field, making the $1 RexInputRef illegal.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
