Github user MrMathias commented on a diff in the pull request:
https://github.com/apache/spark/pull/20759#discussion_r173563217
--- Diff: docs/ml-collaborative-filtering.md ---
@@ -19,6 +19,7 @@ by a small set of latent factors that can be used to
predict missing entries.
algorithm to learn these latent factors. The implementation in `spark.ml`
has the
following parameters:
+* *checkpointInterval* helps with recovery when nodes fail and
StackOverflow exceptions caused by long lineage. **Will be silently ignored if
*SparkContext.CheckpointDir* is not set.** (defaults to 10).
--- End diff --
Checkpointing exists to better deal with node failure and decrease memory
consumption from lineage. This wording is taken from the parameter-comment in
the ALS implementation itself, so I think it is fitting.
This list of parameters is both a sub-set and unordered.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
