nada-attia commented on issue #19654:
URL: https://github.com/apache/hudi/issues/19654#issuecomment-5331492244
in the passing test case (0.14.1), the following configs are set:
- hoodie.schema.on.read.enable=false
- hoodie.datasource.write.reconcile.schema=true
- hoodie.avro.schema.validate=true
in the failing test case (1.2), the following configs are set:
- hoodie.schema.on.read.enable=false
- hoodie.datasource.write.reconcile.schema=true
- hoodie.avro.schema.validate=false
note, in the 0.14 job, we set the following config which is no longer
available in 1.2.
```
public static final ConfigProperty<Boolean> MAKE_NEW_COLUMNS_NULLABLE =
ConfigProperty
.key("hoodie.datasource.write.new.columns.nullable")
.defaultValue(false)
.markAdvanced()
.sinceVersion("0.14.0")
.withDocumentation("When a non-nullable column is added to datasource
during a write operation, the write "
+ " operation will fail schema compatibility check. Set this
option to true will make the newly added "
+ " column nullable to successfully complete the write
operation.");
```
im wondering if this is related to the failure we were seeing. @ad1happy2go
can you PTAL?
for context, this is a snippet of the error:
```
Caused by: org.apache.hudi.exception.SchemaCompatibilityException: Field
address.country has no default value and is non-nullable
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
