nikoshet commented on issue #8372: URL: https://github.com/apache/hudi/issues/8372#issuecomment-1497193253
I also faced this yesterday. In order to overcome this without modifying the Hudi source code, I used the `SqlQueryBasedTransformer` and the `ComplexKeyGenerator` to extract the `year`,`month` and `day` values from the `inserted_at` column, like this: ``` --hoodie-conf hoodie.datasource.write.recordkey.field=id \ --hoodie-conf hoodie.datasource.write.hive_style_partitioning=true \ --hoodie-conf hoodie.datasource.write.partitionpath.field=year,month,day \ --hoodie-conf hoodie.datasource.write.keygenerator.class=org.apache.hudi.keygen.ComplexKeyGenerator \ --transformer-class org.apache.hudi.utilities.transform.SqlQueryBasedTransformer \ --hoodie-conf hoodie.deltastreamer.transformer.sql="SELECT * , extract(year from inserted_at) as year, extract(month from inserted_at) as month, extract(day from inserted_at) as day FROM <SRC> a;" ``` Hudi version: 0.13.0, Storage: S3, Deployment: Kubernetes -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
