Github user aarondav commented on the pull request:
https://github.com/apache/spark/pull/3900#issuecomment-68801129
Agree with 2MB with the caveat that this could cause some slowdown for the
other code path (reading cache blocks from disk). However, memory mapping
frequently can be dangerous (it was actually due to the JVM dying with a SIGBUS
error that we added the threshold in the shuffle code path), which seems like a
worse problem to have than the possibility of an undetermined slowdown.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
