bvaradar commented on issue #2280: URL: https://github.com/apache/hudi/issues/2280#issuecomment-735927848
@ygordefraga : This could be coming from the increase in the number of partitions. This could be related to https://github.com/apache/hudi/issues/2269#issuecomment-733299492 Also, note that since you increased the number of partition with the additional partitioning level, keeping the same number of executors wont be exactly apples-to-apples comparison. Can you try 0.6.0 (which has incremental cleaning support) to see if you get better performance. Please note that the next version of hudi will come with consolidated metadata which will remove the listing altogether. ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected]
