[jira] [Commented] (SPARK-17454) Add option to specify Mesos resource offer constraints
[ https://issues.apache.org/jira/browse/SPARK-17454?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15523003#comment-15523003 ] Chris Bannister commented on SPARK-17454: - Yep, it would be good to also provide isolation between tasks running on the same agent > Add option to specify Mesos resource offer constraints > -- > > Key: SPARK-17454 > URL: https://issues.apache.org/jira/browse/SPARK-17454 > Project: Spark > Issue Type: Improvement > Components: Mesos >Affects Versions: 2.0.0 >Reporter: Chris Bannister > > Currently the driver will accept offers from Mesos which have enough ram for > the executor and until its max cores is reached. There is no way to control > the required CPU's or disk for each executor, it would be very useful to be > able to apply something similar to spark.mesos.constraints to resource offers > instead of attributes on the offer. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-17454) Add option to specify Mesos resource offer constraints
[ https://issues.apache.org/jira/browse/SPARK-17454?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15517007#comment-15517007 ] Michael Gummelt commented on SPARK-17454: - So you're trying to only launch executors on nodes with a sufficient amount of disk space? > Add option to specify Mesos resource offer constraints > -- > > Key: SPARK-17454 > URL: https://issues.apache.org/jira/browse/SPARK-17454 > Project: Spark > Issue Type: Improvement > Components: Mesos >Affects Versions: 2.0.0 >Reporter: Chris Bannister > > Currently the driver will accept offers from Mesos which have enough ram for > the executor and until its max cores is reached. There is no way to control > the required CPU's or disk for each executor, it would be very useful to be > able to apply something similar to spark.mesos.constraints to resource offers > instead of attributes on the offer. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-17454) Add option to specify Mesos resource offer constraints
[ https://issues.apache.org/jira/browse/SPARK-17454?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15516685#comment-15516685 ] Chris Bannister commented on SPARK-17454: - The scheduler will write the to mesos disk sandbox if LOCAL_DIRS is not set, see SPARK-9708. Not all nodes are the same, some have small fast disks, some have large slow disks or some just have different amount of space available due to other tasks running on them. > Add option to specify Mesos resource offer constraints > -- > > Key: SPARK-17454 > URL: https://issues.apache.org/jira/browse/SPARK-17454 > Project: Spark > Issue Type: Improvement > Components: Mesos >Affects Versions: 2.0.0 >Reporter: Chris Bannister > > Currently the driver will accept offers from Mesos which have enough ram for > the executor and until its max cores is reached. There is no way to control > the required CPU's or disk for each executor, it would be very useful to be > able to apply something similar to spark.mesos.constraints to resource offers > instead of attributes on the offer. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-17454) Add option to specify Mesos resource offer constraints
[ https://issues.apache.org/jira/browse/SPARK-17454?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15482682#comment-15482682 ] Michael Gummelt commented on SPARK-17454: - As of Spark 2.0, Mesos mode supports spark.executor.cores And the scheduler doesn't reserve any disk. It just writes to the local workspace. Do you have a need for disk reservation? > Add option to specify Mesos resource offer constraints > -- > > Key: SPARK-17454 > URL: https://issues.apache.org/jira/browse/SPARK-17454 > Project: Spark > Issue Type: Improvement > Components: Mesos >Affects Versions: 2.0.0 >Reporter: Chris Bannister > > Currently the driver will accept offers from Mesos which have enough ram for > the executor and until its max cores is reached. There is no way to control > the required CPU's or disk for each executor, it would be very useful to be > able to apply something similar to spark.mesos.constraints to resource offers > instead of attributes on the offer. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
