bitsondatadev commented on code in PR #8528:
URL: https://github.com/apache/iceberg/pull/8528#discussion_r1324263673
##########
docs/spark-writes.md:
##########
@@ -313,6 +313,28 @@ data.writeTo("prod.db.table")
.createOrReplace()
```
+### Schema Merge
+
+While inserting or updating Iceberg is capable of resolving schema mismatch at
runtime. If configured accordingly Iceberg will perform an automatic schema
evolution as following:
+
+* A new column is present in the source but not in the target table. The new
column is added to the target table. Column values are set to NULL in all the
Review Comment:
```suggestion
* A new column is present in the source but not in the target table. The new
column is added to the target table. Column values are set to `NULL` in all the
```
##########
docs/spark-writes.md:
##########
@@ -313,6 +313,28 @@ data.writeTo("prod.db.table")
.createOrReplace()
```
+### Schema Merge
+
+While inserting or updating Iceberg is capable of resolving schema mismatch at
runtime. If configured accordingly Iceberg will perform an automatic schema
evolution as following:
+
+* A new column is present in the source but not in the target table. The new
column is added to the target table. Column values are set to NULL in all the
+rows already present in the table
+* A column is present in the target but not in the source. The target column
value is set to NULL when inserting or left unchanged when updating the row.
+
+The target table must be configured to accept-any-schema
Review Comment:
```suggestion
The target table must be configured to accept any schema change by setting
the property `write.spark.accept-any-schema` to `true`.
```
Carful using the property as part of the sentence.
##########
docs/spark-writes.md:
##########
@@ -313,6 +313,28 @@ data.writeTo("prod.db.table")
.createOrReplace()
```
+### Schema Merge
+
+While inserting or updating Iceberg is capable of resolving schema mismatch at
runtime. If configured accordingly Iceberg will perform an automatic schema
evolution as following:
+
+* A new column is present in the source but not in the target table. The new
column is added to the target table. Column values are set to NULL in all the
+rows already present in the table
+* A column is present in the target but not in the source. The target column
value is set to NULL when inserting or left unchanged when updating the row.
Review Comment:
```suggestion
* A column is present in the target but not in the source. The target column
value is set to `NULL` when inserting or left unchanged when updating the row.
```
##########
docs/spark-writes.md:
##########
@@ -313,6 +313,28 @@ data.writeTo("prod.db.table")
.createOrReplace()
```
+### Schema Merge
+
+While inserting or updating Iceberg is capable of resolving schema mismatch at
runtime. If configured accordingly Iceberg will perform an automatic schema
evolution as following:
+
+* A new column is present in the source but not in the target table. The new
column is added to the target table. Column values are set to NULL in all the
+rows already present in the table
+* A column is present in the target but not in the source. The target column
value is set to NULL when inserting or left unchanged when updating the row.
+
+The target table must be configured to accept-any-schema
+
+```sql
+ALTER TABLE prod.db.sample SET TBLPROPERTIES (
+ 'write.spark.accept-any-schema'='true'
+)
+```
+The writer must enable the `schema-merge` option.
Review Comment:
Is `schema-merge` the config option and `mergeSchema` the session property?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
