deepakpanda93 commented on issue #3868: URL: https://github.com/apache/hudi/issues/3868#issuecomment-4833087957
Hi @matthiasdg Are you still seeing this on newer Hudi? Two things have changed in areas directly related to your ```Required column is missing in data file. Col: [year] error:``` - @guanlisheng's suggestion above (setting hoodie.table.partition.fields=year,month,day in hoodie.properties) — did that resolve it for you? - Since 0.10, the Spark-datasource read path and partition-field handling have been substantially reworked. In particular, partition fields and their types are now persisted in the table config, which is essentially the automatic version of that workaround. Would you be able to retry your MOR + CustomKeyGenerator (timestamp partition) + Hive-sync scenario on a recent Hudi 1.x release and let us know whether it's resolved? If it works, we can close this out; if you still hit it on 1.x, please share the hoodie.properties and the failing query and we'll dig in. Thanks for your patience on this one! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
