Faiz Halde created SPARK-43408:
----------------------------------
Summary: Spark caching in the context of a single job
Key: SPARK-43408
URL: https://issues.apache.org/jira/browse/SPARK-43408
Project: Spark
Issue Type: Question
Components: Shuffle
Affects Versions: 3.3.1
Reporter: Faiz Halde
Does caching benefit a spark job with only a single action in it? Spark IIRC
already optimizes shuffles by persisting them onto the disk
I am unable to find a counter-example where caching would benefit a job with a
single action. In every case I can think of, the shuffle checkpoint acts as a
good enough caching mechanism in itself
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
