kasakrisz commented on code in PR #4869:
URL: https://github.com/apache/hive/pull/4869#discussion_r1395692477
##########
ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/rules/views/HivePushdownSnapshotFilterRule.java:
##########
@@ -124,7 +126,8 @@ private boolean setSnapShotId(RexNode op1, RexNode op2) {
}
RelOptHiveTable hiveTable = (RelOptHiveTable) relOptTable;
-
hiveTable.getHiveTableMD().setVersionIntervalFrom(Long.toString(snapshotId));
+ String snapshotIdText = snapshotId != NULL_SNAPSHOT_ID ?
Long.toString(snapshotId) : null;
Review Comment:
Rebuilding a materialized view incrementally requires a plan which scans the
existing data stored in the MV and computes the changes from base table since
the last rebuild.
It has a Union operator. One branch of the union scans the MV the other one
is derived from the MV definition query plan. In this subplan there is a
filters on top of each TS with a predicate that helps distinguish between
existing and changes records. The predicate should have a `<=` operator. Example
```
<=($3, 42)
```
In case of Iceberg tables `42` is the snapshotId when the MV was last
rebuilt. $3 is referencing the SnaphotId virtual column.
These predicates are temporary in the plan because these are needed only for
the rewrite algorithm which generates the plan with the Union.
We can't say
```
<=($3, NULL)
```
because that always evaluated to `NULL` and the rewrite doesn't happen.
Once we have this plan the Filters are removed and the predicates are pushed
down to TS operator config to `versionIntervalFrom` property.
So the actual constant value doesn't matter. I wanted to chose something
which is not a valid iceberg snapshot Id but still an integer.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
