[
https://issues.apache.org/jira/browse/SPARK-6284?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14902336#comment-14902336
]
Oleh Halenok commented on SPARK-6284:
-------------------------------------
Have a cluster, Mesos 0.22.1, hadoop 2.5.0, 3 masters, 8 slaves, which are 3 -
role "fast", 3 - role "*", 2 other roles.
Spark 1.5.0 is deployed to use cluster mode with dispatcher, it looks working,
however I can not get it working on the particular nodes with role "fast"
configured.
Job submitted as # bin/spark-submit --class org.apache.spark.examples.SparkPi
--master master.novalocal:7077 --deploy-mode cluster --conf
spark.mesos.role=fast http://www/spark.jar 10
Also, I've tried to configure Spark defaults for submitted jobs and added to
/opt/spark/conf/spark-defaults.conf the line "spark.mesos.role fast"
In any case after submitting job is failing with the next error throwing, it
looks driver starts with some defaults - cpu, mem, from spark ui:
State: TASK_ERROR, Message: Task uses more resources cpus(*):1; mem(*):1024
than available cpus(fast):8; mem(fast):14863; disk(fast):45243;
ports(fast):[1025-2180, 2182-3887, 3889-5049, 5052-8079, 8082-65535], Source:
SOURCE_MASTER, Reason: REASON_TASK_INVALID, Time: 1.442868857232848E9
Can you please describe how to configure/start it properly?
> Support framework authentication and role in Mesos framework
> ------------------------------------------------------------
>
> Key: SPARK-6284
> URL: https://issues.apache.org/jira/browse/SPARK-6284
> Project: Spark
> Issue Type: Improvement
> Components: Mesos
> Reporter: Timothy Chen
> Assignee: Timothy Chen
> Fix For: 1.5.0
>
>
> Support framework authentication and role in both Coarse grain and fine grain
> mode.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
