alexeykudinkin commented on issue #6798: URL: https://github.com/apache/hudi/issues/6798#issuecomment-1262803683
@sstimmel this is a known issue due to how Spark treats partition-columns (by default, Spark doesn't persist them in the data files, but instead encoding them into partition path). Since we're relying on some of the Spark infra to read the data to make sure that Hudi's tables are compatible w/ Spark execution engines optimizations we're unfortunately strangled by these limitations currently, but we're actively looking for solutions there. You can find more details in the HUDI-3204 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
