dhruve commented on issue #22015: [SPARK-20286][SPARK-24786][Core][DynamicAllocation] Release executors on unpersisting RDD URL: https://github.com/apache/spark/pull/22015#issuecomment-464906979 The user intends to drop the cached data by explicitly calling `unpersist` on the rdd. So if the executor doesn't have any cached data on it, we can assume it is safe to give it away. However if the executor still has cached data from other rdd's, in that case we continue to honor the expiry time based on the set timeout.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
