Fine grain mode does reuse the same JVM but perhaps different placement or
different allocated cores comparing to the same total memory allocation.
Tim
Sent from my iPhone
> On Nov 3, 2015, at 6:00 PM, Reynold Xin wrote:
>
> Soren,
>
> If I understand how Mesos works correctly, even the fine
Hi,
We are using Mesos fine grained mode because we can have multiple instances of
spark to share machines and each application get resources dynamically
allocated. Thanks & Regards, Meethu M
On Wednesday, 4 November 2015 5:24 AM, Reynold Xin
wrote:
If you are using Spark with M
Soren,
If I understand how Mesos works correctly, even the fine grained mode keeps
the JVMs around?
On Tue, Nov 3, 2015 at 4:22 PM, Soren Macbeth wrote:
> we use fine-grained mode. coarse-grained mode keeps JVMs around which
> often leads to OOMs, which in turn kill the entire executor, causin
We "used" Spark on Mesos to build interactive data analysis platform
because the interactive session could be long and might not use Spark for
the entire session. It is very wasteful of resources if we used the
coarse-grained mode because it keeps resource for the entire session.
Therefore, fine-gr
we use fine-grained mode. coarse-grained mode keeps JVMs around which often
leads to OOMs, which in turn kill the entire executor, causing entire
stages to be retried. In fine-grained mode, only the task fails and
subsequently gets retried without taking out an entire stage or worse.
On Tue, Nov 3
If you are using Spark with Mesos fine grained mode, can you please respond
to this email explaining why you use it over the coarse grained mode?
Thanks.