Github user datumbox commented on a diff in the pull request:
https://github.com/apache/spark/pull/17059#discussion_r103055703
--- Diff: mllib/src/main/scala/org/apache/spark/ml/recommendation/ALS.scala
---
@@ -82,12 +82,20 @@ private[recommendation] trait ALSModelParams extends
Params with HasPredictionCo
* Attempts to safely cast a user/item id to an Int. Throws an exception
if the value is
* out of integer range.
*/
- protected val checkedCast = udf { (n: Double) =>
- if (n > Int.MaxValue || n < Int.MinValue) {
- throw new IllegalArgumentException(s"ALS only supports values in
Integer range for columns " +
- s"${$(userCol)} and ${$(itemCol)}. Value $n was out of Integer
range.")
- } else {
- n.toInt
+ protected val checkedCast = udf { (n: Any) =>
+ n match {
+ case v: Int => v // Avoid unnecessary casting
+ case v: Number =>
+ val intV = v.intValue()
+ if (v == intV) { // True for Byte/Short, Long within the Int range
and Double/Float with no fractional part.
--- End diff --
One could write this differently and explicitly handle all the permitted
types. Unfortunately this would lead to duplicate code. Instead what I do here
is convert the number into Integer and compare it with the original Number. If
the values are identical one of the following is true:
- The value is Byte or Short.
- The value is Long but within the Integer range.
- The value is Double or Float but without any fractional part.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
