Github user jinxing64 commented on the issue:
https://github.com/apache/spark/pull/17276
@mridulm
Sorry for late reply. I opened the pr for
SPARK-19659(https://github.com/apache/spark/pull/16989) and make these two PRs
independent. Basically this pr is is to evaluate the performance(blocks are
shuffled to disk) and stability(size in `MapStatus` is inaccurate and OOM can
happen) of the implementation proposed in SPARK-19659.
I'd be so thankful if you have time to comment on these two PRs.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
