[jira] [Assigned] (SPARK-10295) Dynamic allocation in Mesos does not release when RDDs are cached
[ https://issues.apache.org/jira/browse/SPARK-10295?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen reassigned SPARK-10295: - Assignee: Sean Owen Dynamic allocation in Mesos does not release when RDDs are cached - Key: SPARK-10295 URL: https://issues.apache.org/jira/browse/SPARK-10295 Project: Spark Issue Type: Improvement Components: Documentation, Spark Core Affects Versions: 1.5.0 Environment: Spark 1.5.0 RC1 Centos 6 java 7 oracle Reporter: Hans van den Bogert Assignee: Sean Owen Priority: Minor Fix For: 1.6.0, 1.5.1 When running spark in coarse grained mode with shuffle service and dynamic allocation, the driver does not release executors if a dataset is cached. The console output OTOH shows: 15/08/26 17:29:58 WARN SparkContext: Dynamic allocation currently does not support cached RDDs. Cached data for RDD 9 will be lost when executors are removed. However after the default of 1m, executors are not released. When I perform the same initial setup, loading data, etc, but without caching, the executors are released. Is this intended behaviour? If this is intended behaviour, the console warning is misleading. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Assigned] (SPARK-10295) Dynamic allocation in Mesos does not release when RDDs are cached
[ https://issues.apache.org/jira/browse/SPARK-10295?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-10295: Assignee: (was: Apache Spark) Dynamic allocation in Mesos does not release when RDDs are cached - Key: SPARK-10295 URL: https://issues.apache.org/jira/browse/SPARK-10295 Project: Spark Issue Type: Improvement Components: Documentation, Spark Core Affects Versions: 1.5.0 Environment: Spark 1.5.0 RC1 Centos 6 java 7 oracle Reporter: Hans van den Bogert Priority: Minor When running spark in coarse grained mode with shuffle service and dynamic allocation, the driver does not release executors if a dataset is cached. The console output OTOH shows: 15/08/26 17:29:58 WARN SparkContext: Dynamic allocation currently does not support cached RDDs. Cached data for RDD 9 will be lost when executors are removed. However after the default of 1m, executors are not released. When I perform the same initial setup, loading data, etc, but without caching, the executors are released. Is this intended behaviour? If this is intended behaviour, the console warning is misleading. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Assigned] (SPARK-10295) Dynamic allocation in Mesos does not release when RDDs are cached
[ https://issues.apache.org/jira/browse/SPARK-10295?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-10295: Assignee: Apache Spark Dynamic allocation in Mesos does not release when RDDs are cached - Key: SPARK-10295 URL: https://issues.apache.org/jira/browse/SPARK-10295 Project: Spark Issue Type: Improvement Components: Documentation, Spark Core Affects Versions: 1.5.0 Environment: Spark 1.5.0 RC1 Centos 6 java 7 oracle Reporter: Hans van den Bogert Assignee: Apache Spark Priority: Minor When running spark in coarse grained mode with shuffle service and dynamic allocation, the driver does not release executors if a dataset is cached. The console output OTOH shows: 15/08/26 17:29:58 WARN SparkContext: Dynamic allocation currently does not support cached RDDs. Cached data for RDD 9 will be lost when executors are removed. However after the default of 1m, executors are not released. When I perform the same initial setup, loading data, etc, but without caching, the executors are released. Is this intended behaviour? If this is intended behaviour, the console warning is misleading. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
