cloud-fan commented on pull request #28669:
URL: https://github.com/apache/spark/pull/28669#issuecomment-640574818
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub an
cloud-fan commented on pull request #28669:
URL: https://github.com/apache/spark/pull/28669#issuecomment-636294332
thanks, merging to master/3.0!
This is an automated message from the Apache Git Service.
To respond to the mes
cloud-fan commented on pull request #28669:
URL: https://github.com/apache/spark/pull/28669#issuecomment-635751861
This makes sense to me. e.g. for partitions [1, 1, 1, ..., 1, 10], which are
coalesced to `[9, 10]`, I don't think there are skewed partitions.
@JkSelf what do you think
