Imran Rashid created SPARK-27677: ------------------------------------ Summary: Disk-persisted RDD blocks served by shuffle service, and ignored for Dynamic Allocation Key: SPARK-27677 URL: https://issues.apache.org/jira/browse/SPARK-27677 Project: Spark Issue Type: New Feature Components: Block Manager, Spark Core Affects Versions: 2.4.3 Reporter: Imran Rashid
Disk-cached RDD blocks are currently unavailable after an executor is removed. However, when there is an external shuffle service, the data remains available on disk and could be served by the shuffle service. This would allow dynamic allocation to reclaim executors with only disk-cached blocks more rapidly, but still keep the cached data available. -- This message was sent by Atlassian JIRA (v7.6.3#76005) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org