cloud-fan commented on code in PR #57746:
URL: https://github.com/apache/spark/pull/57746#discussion_r3728227504
##########
sql/catalyst/src/main/java/org/apache/spark/sql/catalyst/expressions/VectorFunctionImplUtils.java:
##########
@@ -448,6 +460,21 @@ public static Float vectorNorm(ArrayData vec, float
degree, UTF8String funcName)
}
}
+ /**
+ * Computes the Lp norm of a float vector using the specified degree.
+ * Supported degrees: 1.0 (L1), 2.0 (L2), Float.POSITIVE_INFINITY (L∞).
Review Comment:
Spark's repository guidance requires ASCII in code comments.
```suggestion
* Supported degrees: 1.0 (L1), 2.0 (L2), Float.POSITIVE_INFINITY
(infinity norm).
```
##########
sql/catalyst/src/main/java/org/apache/spark/sql/catalyst/expressions/VectorFunctionImplUtils.java:
##########
@@ -90,19 +93,22 @@ public static Float vectorCosineSimilarity(ArrayData left,
ArrayData right, UTF8
if (left.isNullAt(i) || right.isNullAt(i)) {
return null;
}
- float a = left.getFloat(i);
- float b = right.getFloat(i);
+ double a = left.getFloat(i);
+ double b = right.getFloat(i);
dotProduct += a * b;
norm1Sq += a * a;
norm2Sq += b * b;
i++;
}
- float normProduct = (float) Math.sqrt(norm1Sq * norm2Sq);
- if (normProduct < Float.MIN_NORMAL) {
+ // `norm1Sq * norm2Sq` cannot overflow in double precision: both factors
are bounded by
Review Comment:
Please narrow this bound to finite float elements. `ARRAY<FLOAT>` also
admits infinities, which make an accumulated squared norm infinite, so the
factors are not unconditionally bounded.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
