srowen commented on a change in pull request #25789: [SPARK-28927][ML] Show
warning when input data to ALS is indeterminate
URL: https://github.com/apache/spark/pull/25789#discussion_r324476392
##########
File path: R/pkg/R/mllib_recommendation.R
##########
@@ -82,6 +82,10 @@ setClass("ALSModel", representation(jobj = "jobj"))
#' statsS <- summary(modelS)
#' }
#' @note spark.als since 2.1.0
+#' @note the input rating dataframe to the ALS implementation should not be
indeterminate.
Review comment:
But what if you lose the checkpoint? isn't this the same issue? At some
level the answer is "you can't really fix this, anywhere", right? in practice,
the fairly well understood caching/checkpoint mechanism works, everywhere. It
seems inconsistent to just address this for ALS, as if it's not the same issue
everywhere. It also seems hard to warn without providing any pointer to the
solution, if there is one, but I can see that a warning is better than nothing.
If you're trying to fix a specific problem, maybe indeed detect the problem
in question (a specific AIOOBE) and rewrap it, sure.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
