JindongZhang created SPARK-35751:
------------------------------------
Summary: Support Joint eviction strategies for cached RDD
partitions
Key: SPARK-35751
URL: https://issues.apache.org/jira/browse/SPARK-35751
Project: Spark
Issue Type: Improvement
Components: Block Manager
Affects Versions: 3.1.2
Reporter: JindongZhang
As of now, there's as it were ousting procedure for cached RDD segment in
Spark. The default RDD removal methodology is LRU .When memory space not
adequate for RDD caching, a few allotments will be ousted, on the off chance
that these segments are utilized once more latterly, they will be replicated by
the Ancestry data and cached in memory once more. The replicate stage will
bring in extra taken a toll. Be that as it may, LRU has no ensure for the most
reduced duplicate cost. The to begin with RDD that required to be cached is
ordinarily created by perusing from HDFS and doing a few changes. The perusing
operation ordinarily fetched longer time than other Start transformations.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
