[
https://issues.apache.org/jira/browse/HIVE-27101?focusedWorklogId=850569&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-850569
]
ASF GitHub Bot logged work on HIVE-27101:
-----------------------------------------
Author: ASF GitHub Bot
Created on: 13/Mar/23 08:48
Start Date: 13/Mar/23 08:48
Worklog Time Spent: 10m
Work Description: kasakrisz commented on code in PR #4079:
URL: https://github.com/apache/hive/pull/4079#discussion_r1133604073
##########
ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/rules/views/HivePushdownSnapshotFilterRule.java:
##########
@@ -0,0 +1,156 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to you under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hadoop.hive.ql.optimizer.calcite.rules.views;
+
+import org.apache.calcite.plan.RelOptRule;
+import org.apache.calcite.plan.RelOptRuleCall;
+import org.apache.calcite.plan.RelOptTable;
+import org.apache.calcite.plan.RelRule;
+import org.apache.calcite.rel.RelNode;
+import org.apache.calcite.rel.metadata.RelMetadataQuery;
+import org.apache.calcite.rel.type.RelDataTypeField;
+import org.apache.calcite.rex.RexBuilder;
+import org.apache.calcite.rex.RexCall;
+import org.apache.calcite.rex.RexInputRef;
+import org.apache.calcite.rex.RexLiteral;
+import org.apache.calcite.rex.RexNode;
+import org.apache.calcite.rex.RexShuttle;
+import org.apache.calcite.rex.RexTableInputRef;
+import org.apache.calcite.sql.SqlKind;
+import org.apache.calcite.sql.type.SqlTypeFamily;
+import org.apache.hadoop.hive.ql.metadata.VirtualColumn;
+import org.apache.hadoop.hive.ql.optimizer.calcite.HiveRelFactories;
+import org.apache.hadoop.hive.ql.optimizer.calcite.RelOptHiveTable;
+import org.apache.hadoop.hive.ql.optimizer.calcite.reloperators.HiveFilter;
+
+import java.util.Set;
+
+/**
+ * Calcite rule to push down predicates contains {@link
VirtualColumn#SNAPSHOT_ID} reference to TableScan.
+ * <p>
+ * This rule traverse the logical expression in {@link HiveFilter} operators
and search for
+ * predicates like
+ * <p>
+ * <code>
+ * snapshotId <= 12345677899
+ * </code>
+ * <p>
+ * The literal is set in the {@link RelOptHiveTable#getHiveTableMD()} object
wrapped by
+ * {@link
org.apache.hadoop.hive.ql.optimizer.calcite.reloperators.HiveTableScan}
+ * and the original predicate in the {@link HiveFilter} is replaced with
literal true.
+ */
+public class HivePushdownSnapshotFilterRule extends
RelRule<HivePushdownSnapshotFilterRule.Config> {
+
+ public static final RelOptRule INSTANCE =
+ RelRule.Config.EMPTY.as(HivePushdownSnapshotFilterRule.Config.class)
+ .withRelBuilderFactory(HiveRelFactories.HIVE_BUILDER)
+ .withOperandSupplier(operandBuilder ->
operandBuilder.operand(HiveFilter.class).anyInputs())
+ .withDescription("HivePushdownSnapshotFilterRule")
+ .toRule();
+
+ public interface Config extends RelRule.Config {
+ @Override
+ default HivePushdownSnapshotFilterRule toRule() {
+ return new HivePushdownSnapshotFilterRule(this);
+ }
+ }
+
+ private HivePushdownSnapshotFilterRule(Config config) {
+ super(config);
+ }
+
+ @Override
+ public void onMatch(RelOptRuleCall call) {
+ HiveFilter filter = call.rel(0);
Review Comment:
Yes. In order to identify snapshotId references all column references should
be checked in the predicate. To do it the predicate expression has to be
traversed. `SnapshotIdShuttle` does this traversal and it also sets the
snapshotId in the source TS.
Please let me know if you have a better/faster solution to check whether a
predicate expression has snapshotId reference.
Issue Time Tracking
-------------------
Worklog Id: (was: 850569)
Time Spent: 7h 50m (was: 7h 40m)
> Support incremental materialized view rebuild when Iceberg source tables have
> insert operation only.
> ----------------------------------------------------------------------------------------------------
>
> Key: HIVE-27101
> URL: https://issues.apache.org/jira/browse/HIVE-27101
> Project: Hive
> Issue Type: Improvement
> Components: Iceberg integration, Materialized views
> Reporter: Krisztian Kasa
> Assignee: Krisztian Kasa
> Priority: Major
> Labels: pull-request-available
> Time Spent: 7h 50m
> Remaining Estimate: 0h
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
