Github user jerryshao commented on the issue:
https://github.com/apache/spark/pull/21758
I mean rddC's partitions are derived from rddA and rddB, here assuming
partitions in rddA requires barrier, but not required in rddB. So rddC's
partitions are the half barrier and half not barrier. So how do you merge such
conflict, did you mark rddC's all partitions as barrier, or only the ones
coming from rddA? Obviously partitions from rddB doesn't require barrier (just
normal tasks), forcing to barrier will require strict resource demand.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
