amogh-jahagirdar commented on code in PR #13310:
URL: https://github.com/apache/iceberg/pull/13310#discussion_r2205511986
##########
spark/v4.0/spark/src/main/java/org/apache/iceberg/spark/source/SparkWriteBuilder.java:
##########
@@ -183,6 +206,10 @@ private static Schema validateOrMergeWriteSchema(
UpdateSchema update =
table.updateSchema().caseSensitive(caseSensitive).unionByNameWith(newSchema);
Schema mergedSchema = update.apply();
+ if (writeIncludesRowLineage) {
Review Comment:
>is this needed? dsSchema already contains the 2 row lineage meta columns.
dsSchema does but we need the spark schema and the Iceberg `writeSchema` to
align otherwise it'll fail in the `convert` on 227
##########
spark/v4.0/spark/src/main/java/org/apache/iceberg/spark/source/SparkWriteBuilder.java:
##########
@@ -183,6 +206,10 @@ private static Schema validateOrMergeWriteSchema(
UpdateSchema update =
table.updateSchema().caseSensitive(caseSensitive).unionByNameWith(newSchema);
Schema mergedSchema = update.apply();
+ if (writeIncludesRowLineage) {
Review Comment:
>is this needed? dsSchema already contains the 2 row lineage meta columns.
dsSchema does but we need the spark schema and the Iceberg `writeSchema` to
align otherwise it'll fail in the `convert` on 227
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org
For additional commands, e-mail: issues-h...@iceberg.apache.org
