[ https://issues.apache.org/jira/browse/SPARK-8881?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14618369#comment-14618369 ]
Nishkam Ravi commented on SPARK-8881: ------------------------------------- This isn't the best example because the third worker will get screened out. Consider the following instead: three workers with num_cores (8, 8, 3). spark.cores.maximum=8, spark.executor.cores=2. Core allocation would be (3, 3, 2). 3 executors launched instead of 4. You get the drift. > Scheduling fails if num_executors < num_workers > ----------------------------------------------- > > Key: SPARK-8881 > URL: https://issues.apache.org/jira/browse/SPARK-8881 > Project: Spark > Issue Type: Bug > Components: Deploy > Affects Versions: 1.4.0, 1.5.0 > Reporter: Nishkam Ravi > > Current scheduling algorithm (in Master.scala) has two issues: > 1. cores are allocated one at a time instead of spark.executor.cores at a time > 2. when spark.cores.max/spark.executor.cores < num_workers, executors are not > launched and the app hangs (due to 1) -- This message was sent by Atlassian JIRA (v6.3.4#6332) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org