Github user hvanhovell commented on the issue:
https://github.com/apache/spark/pull/19175
@DonnyZone The current heavy weight approach should be better in terms of
data being scanned and moved. The main problem is just that shuffles get too
bulky. However this is a no-trivial problem to fix. So I am not against this
approach, and I think it be a useful thing to add.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
