ulysses-you commented on pull request #34820: URL: https://github.com/apache/spark/pull/34820#issuecomment-987760901
thank you @maryannxue for the explanation, I see your point now. It's really hard to give the number of this change. In our production, the `advisoryPartitionSizeInBytes` value of writing job's last stage is always bigger than the default value, e.g. 512MB. The origin partitions before coalesced can also be big and so are the blocks. So I think it's more likely to take the benifits. But yes, I cann't say the number here.. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
