yihua commented on PR #10615: URL: https://github.com/apache/hudi/pull/10615#issuecomment-2050092146
> 1. is there any change to partitions in `hoodie.proerties`? Do we now write it as `field1:type,field2:type2` when using CustomKeyGenerator? There is no change to the table configs in `hoodie.properties`, i.e., the `hoodie.table.partition.fields` contains the comma-separated list of partition field names like `"segment,ts"` (no type for custom key generator). This PR opens the opportunity to override the `hoodie.datasource.write.partitionpath.field` with `SET TBLPROPERTIES` at the table level in the Spark catalog, so that SQL DML can derive the correct write config of the partition fields (e.g., `"segment:simple,ts:timestamp"` instead of `"segment,ts"`). > 2. Thanks for adding extensive tests. Can you please look into the failures? They seem related to the patch. Failures for Spark 3.2 and above are fixed. I'm looking into failures for older Spark versions. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
