[
https://issues.apache.org/jira/browse/SPARK-17444?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15472931#comment-15472931
]
Ofer Eliassaf edited comment on SPARK-17444 at 9/8/16 7:13 AM:
---------------------------------------------------------------
sorry - i can't reproduce properly.
closing.
The problem was that i used different spar-env.sh versions on different
machines which led to undefined bahavior
was (Author: ofer):
sorry - i can't reproduce properly.
closing.
> spark memory allocation makes workers non responsive
> ----------------------------------------------------
>
> Key: SPARK-17444
> URL: https://issues.apache.org/jira/browse/SPARK-17444
> Project: Spark
> Issue Type: Bug
> Components: PySpark
> Affects Versions: 2.0.0
> Environment: spark standalone
> Reporter: Ofer Eliassaf
> Priority: Critical
>
> I am running a cluster of 3 slaves and 2 masters with spark standalone.
> total of 12 cores (4 in each machine)
> memory allocated to executors and workers are 4.5GB, and the machine has
> total of 8GB.
> steps to reproduce:
> open pyspark and point to the masters
> run the following command multiple times:
> sc.parallelize(range(1,50000000), 12).count()
> after few runs the python will stop respond.
> then exit the python shell.
> The critical issue that after this happens the cluster is not useful any more:
> There is no way to submit application or running another commands on the
> cluster etc.
> Hope this helps!
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
