Depends... The heartbeat you received happens due to GC pressure (probably
due to Full GC). If you increase the memory too much, the GC's may be less
frequent, but the Full GC's may take longer. Try increasing the following
confs:
spark.executor.heartbeatInterval
spark.core.connection.ack.wait.tim
I will increase memory for the job...that will also fix it right ?
On Apr 10, 2015 12:43 PM, "Reza Zadeh" wrote:
> You should pull in this PR: https://github.com/apache/spark/pull/5364
> It should resolve that. It is in master.
> Best,
> Reza
>
> On Fri, Apr 10, 2015 at 8:32 AM, Debasish Das
> w
You should pull in this PR: https://github.com/apache/spark/pull/5364
It should resolve that. It is in master.
Best,
Reza
On Fri, Apr 10, 2015 at 8:32 AM, Debasish Das
wrote:
> Hi,
>
> I am benchmarking row vs col similarity flow on 60M x 10M matrices...
>
> Details are in this JIRA:
>
> https:/