Github user tdas commented on the pull request:
https://github.com/apache/spark/pull/4167#issuecomment-72137779
This is a tricky change. I had kept it in MEMORY_ONLY_SER to make the
default usecase fast (when stuff fits in memory) fast, and the corner case
(stuff not in memory) will be slow (recomputed from raw data or intermediate
shuffle files). However writing to MEMORY_AND_DISK_SER makes even the default
usecase slow, and corner case equally fast as the default (so effectively, no
corner case). This is, for the majority of the users, a signiicant performance
regression, especially if they have not even configured their streaming
application to use fast and/or multiple disks (which is in most cases not
necessary).
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
