danny0405 commented on issue #12619: URL: https://github.com/apache/hudi/issues/12619#issuecomment-2586087999
> Does the compaction operation have to wait until all buckets have files before it can complete? Not really, it depends on what the compaction plan looks like, currently the plan scheduler would scan all the partitions for all the buckets to see if there is any log file in the latest file slice, by default the pllan include all the log files. > Is it expected behavior that Hive can only read data after the compaction operation is completed? It depends on what kind of inputformat you configured, here is a doc on how to query with Hive in Chinese: https://www.yuque.com/yuzhao-my9fz/kb/kgv2rb?singleDoc# 《Hive On Hudi》 > After compaction, is it expected that each bucket contains only one file? Is there a way to retain more historical files? If the compaction is triggered by a separate job, you may need to check the clean options specifically. BTW, the bulk_insert write parquets in your job right? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
