On 08/04/2014 10:57 PM, Michael Armbrust wrote:
If mesos is allocating a container that is exactly the same as the max
heap size then that is leaving no buffer space for non-heap JVM memory,
which seems wrong to me.
This can be a cause. I am now wondering how mesos pick up the size and
setup
If mesos is allocating a container that is exactly the same as the max heap
size then that is leaving no buffer space for non-heap JVM memory, which
seems wrong to me.
The problem here is that cacheTable is more aggressive about grabbing large
ByteBuffers during caching (which it later releases
I am not a mesos expert... but it sounds like there is some mismatch
between the size that mesos is giving you and the maximum heap size of the
executors (-Xmx).
On Fri, Aug 1, 2014 at 12:07 AM, Gurvinder Singh gurvinder.si...@uninett.no
wrote:
It is not getting out of memory exception. I am
cacheTable uses a special columnar caching technique that is optimized for
SchemaRDDs. It something similar to MEMORY_ONLY_SER but not quite. You
can specify the persistence level on the SchemaRDD itself and register that
as a temporary table, however it is likely you will not get as good
