Liang-Chi Hsieh created SPARK-28866:
---------------------------------------
Summary: Persist item factors RDD when checkpointing in ALS
Key: SPARK-28866
URL: https://issues.apache.org/jira/browse/SPARK-28866
Project: Spark
Issue Type: Improvement
Components: ML
Affects Versions: 3.0.0
Reporter: Liang-Chi Hsieh
In ALS ML implementation, if `implicitPrefs` is false, we checkpoint the RDD of
item factors, between intervals. Before checkpointing and materializing RDD,
this RDD was not persisted. It causes recomputation. In an experiment, there is
performance difference between persisting and no persisting before
checkpointing the RDD.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
