trushev commented on issue #11794: URL: https://github.com/apache/hudi/issues/11794#issuecomment-2301853593
> @trushev hi, can you see that, why these two parameters be binded together. Hi @KnightChess , let me explain my changes in https://github.com/apache/hudi/pull/7895 According documentation of `hoodie.datasource.write.schema.allow.auto.evolution.column.drop`: "...batch's schema can have any of the columns dropped. By default, Hudi will not allow this kind of (auto) schema evolution." Default `hoodie.datasource.write.schema.allow.auto.evolution.column.drop=false`. So you would expect that if that option is not presented or false then column dropping would cause an error. But it didn't work as expected. The example from https://github.com/apache/hudi/pull/7895 silently proceeded to work without the error. It happened because of `hoodie.avro.schema.validate=false`. Pay attention that it is a default value as well! So I decoupled `hoodie.datasource.write.schema.allow.auto.evolution.column.drop` and `hoodie.avro.schema.validate`. Regardless of whether `hoodie.avro.schema.validate` is true, false, or not represented, the column dropping check must be performed if `hoodie.datasource.write.schema.allow.auto.evolution.column.drop` is false or not presented -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
