Joseph K. Bradley created SPARK-8733:
----------------------------------------
Summary: ML RDD.unpersist calls should use blocking = false
Key: SPARK-8733
URL: https://issues.apache.org/jira/browse/SPARK-8733
Project: Spark
Issue Type: Improvement
Components: ML, MLlib
Reporter: Joseph K. Bradley
MLlib uses unpersist in many places, but is not consistent about blocking vs
not. We should check through all of MLlib and change calls to use blocking =
false, unless there is a real need to block. I have run into issues with
futures timing out because of unpersist() calls, when there was no real need
for the ML method to fail.
See attached screenshot. Training succeeded, but the final unpersist during
cleanup failed.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
