namrathamyske commented on PR #7932: URL: https://github.com/apache/iceberg/pull/7932#issuecomment-1621236434
Correct, but as we see first step of exchange there are 20 partitions which is parameter controlled by config shuffle partitions number. Next step is AQEshuffleRead which coalesces 20 partitions to 1 partitions. In the above plan maybe all 20 partitions are processed by same executor, so all of them coalesced to one partitions. But if 20 partitions were spread across more executors, then number of coalesced partitions might increase as coalesce only happens locally per executor. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
