Hello Attila Jeges, Impala Public Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/17765
to look at the new patch set (#2).
Change subject: IMPALA-10840: Add support for "FOR SYSTEM_TIME AS OF" and "FOR
SYSTEM_VERSION AS OF" for Iceberg tables
......................................................................
IMPALA-10840: Add support for "FOR SYSTEM_TIME AS OF" and "FOR SYSTEM_VERSION
AS OF" for Iceberg tables
This patch adds support "FOR SYSTEM_TIME AS OF" and
"FOR SYSTEM_VERSION AS OF" clauses for Iceberg tables. The new
clauses are part of the table ref. FOR SYSTEM_TIME AS OF conforms to the
SQL2011 standard:
https://cs.ulb.ac.be/public/_media/teaching/infoh415/tempfeaturessql2011.pdf
The timestamp given to "FOR SYSTEM_TIME AS OF" is interpreted in the
local timezone. The local timezone can be set via the query option
TIMEZONE. By default the timezone being used is the coordinator node's
local timezone.
"FOR SYSTEM_VERSION AS OF" is a non-standard extension that can be used
to query a table via a snapshot ID. HIVE-25344 also added support for
these clauses to Hive.
Sample queries:
SELECT * FROM t FOR SYSTEM_TIME AS OF now();
SELECT * FROM t FOR SYSTEM_TIME AS OF '2021-08-10 11:02:34';
SELECT * FROM t FOR SYSTEM_TIME AS OF now() - interval 10 days + interval 3
hours;
SELECT * FROM t FOR SYSTEM_VERSION AS OF 7080861547601448759;
SELECT * FROM t FOR SYSTEM_TIME AS OF now()
MINUS
SELECT * FROM t FOR SYSTEM_TIME AS OF now() - interval 1 days;
This patch uses some parts of the in-progress
IMPALA-9773 (https://gerrit.cloudera.org/#/c/13342/) developed by
Todd Lipcon and Grant Henke. This patch also resolves some TODOs of
IMPALA-9773, i.e. after this patch it'll be easier to add
time travel for Kudu tables as well.
Testing:
* added parser tests (ParserTest.java)
* added analyzer tests (AnalyzeStmtsTest.java)
* added e2e tests (test_iceberg.py)
Change-Id: Ib523c5e47b8d9c377bea39a82fe20249177cf824
---
M be/src/common/init.cc
M fe/src/main/cup/sql-parser.cup
M fe/src/main/java/org/apache/impala/analysis/AlterTableSetTblProperties.java
M fe/src/main/java/org/apache/impala/analysis/BaseTableRef.java
M fe/src/main/java/org/apache/impala/analysis/ColumnDef.java
M fe/src/main/java/org/apache/impala/analysis/RangePartition.java
M fe/src/main/java/org/apache/impala/analysis/TableRef.java
A fe/src/main/java/org/apache/impala/analysis/TimeTravelSpec.java
M fe/src/main/java/org/apache/impala/catalog/FeIcebergTable.java
M fe/src/main/java/org/apache/impala/planner/IcebergScanNode.java
M fe/src/main/java/org/apache/impala/planner/KuduScanNode.java
A fe/src/main/java/org/apache/impala/util/ExprUtil.java
M fe/src/main/java/org/apache/impala/util/IcebergUtil.java
M fe/src/main/java/org/apache/impala/util/KuduUtil.java
M fe/src/main/jflex/sql-scanner.flex
M fe/src/test/java/org/apache/impala/analysis/AnalyzeStmtsTest.java
M fe/src/test/java/org/apache/impala/analysis/ParserTest.java
M testdata/workloads/functional-query/queries/QueryTest/iceberg-negative.test
M tests/query_test/test_iceberg.py
19 files changed, 545 insertions(+), 63 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/65/17765/2
--
To view, visit http://gerrit.cloudera.org:8080/17765
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Ib523c5e47b8d9c377bea39a82fe20249177cf824
Gerrit-Change-Number: 17765
Gerrit-PatchSet: 2
Gerrit-Owner: Zoltan Borok-Nagy <[email protected]>
Gerrit-Reviewer: Attila Jeges <[email protected]>
Gerrit-Reviewer: Impala Public Jenkins <[email protected]>
Gerrit-Reviewer: Zoltan Borok-Nagy <[email protected]>
