amogh-jahagirdar commented on code in PR #15825:
URL: https://github.com/apache/iceberg/pull/15825#discussion_r3016968064
##########
spark/v4.1/spark/src/main/java/org/apache/iceberg/spark/source/SparkTable.java:
##########
@@ -353,13 +347,35 @@ private static SparkTable createWithTimestamp(Table
table, AsOfTimestamp timeTra
return new SparkTable(table, snapshotId, timeTravel);
}
+ private static Set<TableCapability> computeCapabilities(Table table) {
+ ImmutableSet.Builder<TableCapability> caps = ImmutableSet.builder();
Review Comment:
Nit: I'd prefer the full form, `tableCapabilities`
##########
spark/v4.1/spark-extensions/src/test/java/org/apache/iceberg/spark/extensions/TestMergeSchemaEvolution.java:
##########
@@ -260,6 +262,49 @@ public void testMergeWithSchemaEvolutionTypeWidening() {
sql("SELECT id, value FROM %s ORDER BY id", selectTarget()));
}
+ @TestTemplate
+ public void testMergeWithSchemaEvolutionDisabledByTableProperty() {
+ assumeThat(branch).as("Schema evolution does not work for branches
currently").isNull();
+
+ createAndInitTable(
+ "id INT, dep STRING",
+ "{ \"id\": 1, \"dep\": \"hr\" }\n" + "{ \"id\": 2, \"dep\":
\"software\" }");
+
+ sql(
+ "ALTER TABLE %s SET TBLPROPERTIES ('%s' 'false')",
Review Comment:
specifically
https://github.com/apache/spark/blob/224f30fa8a1b4912f161126df82dafccfae6d5a1/sql/api/src/main/antlr4/org/apache/spark/sql/catalyst/parser/SqlBaseParser.g4#L663
is what i mean
##########
spark/v4.1/spark-extensions/src/test/java/org/apache/iceberg/spark/extensions/TestMergeSchemaEvolution.java:
##########
@@ -260,6 +262,49 @@ public void testMergeWithSchemaEvolutionTypeWidening() {
sql("SELECT id, value FROM %s ORDER BY id", selectTarget()));
}
+ @TestTemplate
+ public void testMergeWithSchemaEvolutionDisabledByTableProperty() {
+ assumeThat(branch).as("Schema evolution does not work for branches
currently").isNull();
+
+ createAndInitTable(
+ "id INT, dep STRING",
+ "{ \"id\": 1, \"dep\": \"hr\" }\n" + "{ \"id\": 2, \"dep\":
\"software\" }");
+
+ sql(
+ "ALTER TABLE %s SET TBLPROPERTIES ('%s' 'false')",
Review Comment:
I think this form is technically valid
https://github.com/apache/spark/blob/224f30fa8a1b4912f161126df82dafccfae6d5a1/sql/api/src/main/antlr4/org/apache/spark/sql/catalyst/parser/SqlBaseParser.g4#L657
but I'd also prefer the explicit =.
##########
core/src/main/java/org/apache/iceberg/TableProperties.java:
##########
@@ -346,6 +346,10 @@ private TableProperties() {}
public static final String SPARK_WRITE_ACCEPT_ANY_SCHEMA =
"write.spark.accept-any-schema";
public static final boolean SPARK_WRITE_ACCEPT_ANY_SCHEMA_DEFAULT = false;
+ public static final String SPARK_WRITE_AUTO_SCHEMA_EVOLUTION =
Review Comment:
+1 to the enabled suffix, it better matches the naming of the other
properties we have
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
