zhangyue19921010 commented on PR #5829: URL: https://github.com/apache/hudi/pull/5829#issuecomment-1152332531
> @zhangyue19921010 : I would like to discuss this patch w/ you. Looks like for a table w/ very large no of partitions, we are regressing after #4643. Can you take a look to see if we can come up w/ some middle ground may be. 10x is definitely bad. Hi @nsivabalan Sorry for this blocking :< The performance depends on the bottleneck which is the number of partitions or the number of data files in each partitions. It's hard to balance. But as you said 10x is too bad. We definitely need to revert this change before 0.11.1. After this maybe we can offer a config to let advanced users decide what kind of method he need to use based on their use case? or some way to detect automatically Sorry for this blocking again. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
