huan233usc commented on code in PR #17957:
URL: https://github.com/apache/iceberg/pull/17957#discussion_r4040060010
##########
spark/v4.2/spark/src/main/java/org/apache/iceberg/spark/source/SparkTable.java:
##########
@@ -158,6 +169,60 @@ public Set<TableCapability> capabilities() {
return capabilities;
}
+ @Override
+ public boolean supportsColumnChange(TableChange.ColumnChange change) {
+ if (change instanceof TableChange.AddColumn) {
+ TableChange.AddColumn add = (TableChange.AddColumn) change;
+ return add.isNullable() && add.defaultValue() == null &&
canConvert(add.dataType());
Review Comment:
Spark 4.2 recursively generates schema changes for map keys.
For example, evolving a target `map<int, string>` from a source `map<long,
string>` produces `updateColumnType(["m", "key"], LongType)`. T
his implementation returns `true`, but `SchemaUpdate.apply()` explicitly
rejects(as Iceberg don't allow it) updates to map keys (`Cannot update map
keys`), so the write fails during `alterTable` instead of letting Spark skip
the unsupported evolution and attempt a cast.
The same false positive occurs for additions or type changes inside a
struct-valued map key. Could we reject changes whose field path targets the
map-key subtree and add regression coverage for it?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
