Github user srowen commented on the issue:
https://github.com/apache/spark/pull/19764
I'm probably not qualified to review this. I don't think you addressed
Herman's question. It wasn't about ordering or whether the same exact row maps
to the same partition, but whether all values for a key map to the same
partition. I believe that's part of the contract here. If it doesn't do that,
then, I don't see how it solves the problem you're trying to solve. Skew is
inherently a problem if you promise to put all values for a key together.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
