[
https://issues.apache.org/jira/browse/SPARK-35751?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17362730#comment-17362730
]
Apache Spark commented on SPARK-35751:
--------------------------------------
User 'qfoxzjd' has created a pull request for this issue:
https://github.com/apache/spark/pull/32900
> Support Joint eviction strategies for cached RDD partitions
> -----------------------------------------------------------
>
> Key: SPARK-35751
> URL: https://issues.apache.org/jira/browse/SPARK-35751
> Project: Spark
> Issue Type: Improvement
> Components: Block Manager
> Affects Versions: 3.1.2
> Reporter: JindongZhang
> Priority: Minor
>
> As of now, there's as it were ousting procedure for cached RDD segment in
> Spark. The default RDD removal methodology is LRU .When memory space not
> adequate for RDD caching, a few allotments will be ousted, on the off chance
> that these segments are utilized once more latterly, they will be replicated
> by the Ancestry data and cached in memory once more. The replicate stage will
> bring in extra taken a toll. Be that as it may, LRU has no ensure for the
> most reduced duplicate cost. The to begin with RDD that required to be cached
> is ordinarily created by perusing from HDFS and doing a few changes. The
> perusing operation ordinarily fetched longer time than other Start
> transformations.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
