lilei1128 commented on code in PR #16523:
URL: https://github.com/apache/iceberg/pull/16523#discussion_r3360887285
##########
spark/v4.1/spark/src/test/java/org/apache/iceberg/spark/sql/TestSelect.java:
##########
@@ -740,6 +740,49 @@ public void simpleTypesInFilter() {
sql("DROP TABLE IF EXISTS %s", tableName);
}
+ @TestTemplate
+ public void testTimeTravelFilterOnRenamedColumn() {
+ String ttTableName = tableName("tt_rename_table");
+ sql("DROP TABLE IF EXISTS %s", ttTableName);
+ sql(
+ "CREATE TABLE %s (id BIGINT, col DOUBLE) USING iceberg TBLPROPERTIES ("
+ + "'read.data-planning-mode'='distributed',"
+ + "'read.delete-planning-mode'='distributed')",
+ ttTableName);
+ sql("INSERT INTO %s VALUES (1, 100.0), (2, 200.0), (3, 0.0)", ttTableName);
+
+ TableIdentifier ttTableIdent = TableIdentifier.of(tableIdent.namespace(),
"tt_rename_table");
+ long snapshotId =
validationCatalog.loadTable(ttTableIdent).currentSnapshot().snapshotId();
+
+ sql("ALTER TABLE %s RENAME COLUMN col TO value", ttTableName);
+ sql("INSERT INTO %s VALUES (4, 400.0)", ttTableName);
+
+ Dataset<Row> df =
+ spark
+ .read()
+ .format("iceberg")
+ .option(SparkReadOptions.VERSION_AS_OF, snapshotId)
Review Comment:
Thanks for the suggestion, but SparkReadOptions.SNAPSHOT_ID ("snapshot-id")
is explicitly rejected in v4.1 — IcebergSource calls
Spark3Util.validateNoLegacyTimeTravel() which throws:
<img width="2458" height="678" alt="image"
src="https://github.com/user-attachments/assets/9b48eb6c-876a-48c7-866c-98cf8e274fa1";
/>
To make the code compatible with Spark 3.x, 4.0 and 4.1+, we switch to
VERSION_AS_OF. It accepts numeric snapshot IDs with the same semantics, and the
SQL syntax VERSION AS OF <snapshotId> works across all these versions.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
