The epoll mode definitely exists in spark, but the official
documentation does not mention it, nor any of the other settings that
appear to be unofficially documented in:
https://github.com/jaceklaskowski/mastering-apache-spark-book/blob/master/spark-rpc-netty.adoc
I don't seem to have any partic
The spark-shell process alone shouldn't take up that much memory, at least
in my experience. Have you dumped the heap to see what's all in there? What
environment are you running spark in?
Doing stuff like RDD.collect() or .countByKey will pull potentially a lot
of data the spark-shell heap. Anoth