Github user jkbradley commented on the pull request:
https://github.com/apache/spark/pull/2595#issuecomment-57838386
@chouqin @manishamde Seconding the comments from @mengxr --- in my tests,
the current settings for maxMemoryInMB seem pretty reasonable. For a
decent-size cluster (16 nodes or so), the implementation is
communication-bound. For small maxMemoryInMB, the communication latency might
outweigh the non-latency communication time. But for 256MB, there seems to be
enough communication that latency is not an issue. But I've only tested in a
limited setting (16-node EC2 cluster).
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
