Copilot commented on code in PR #13214:
URL: https://github.com/apache/gluten/pull/13214#discussion_r4425687120
##########
docs/backends/velox/velox-backend-scalar-function-support.md:
##########
@@ -1,6 +1,6 @@
# Scalar Functions Support Status
-**Out of 433 scalar functions in Spark 4.1, Gluten currently fully supports
261 functions and partially supports 31 functions.**
+**Out of 433 scalar functions in Spark 4.1, Gluten currently fully supports
262 functions and partially supports 31 functions.**
Review Comment:
The headline count was incremented by 1, but this change marks two table
entries as supported (`>>>` and `shiftrightunsigned`). If the headline count is
intended to track unique Spark scalar *functions* (excluding operators /
aliases), consider clarifying that in the doc text; otherwise, update the
totals to match how the table is counted to avoid inconsistency.
##########
docs/backends/velox/velox-backend-scalar-function-support.md:
##########
@@ -42,14 +42,14 @@ The status applies to `spark.sql.ansi.enabled=false`. When
ANSI mode is enabled,
| & | BitwiseAnd | S | |
| << | ShiftLeft | S | |
| >> | ShiftRight | S | |
-| >>> | ShiftRightUnsigned | | |
+| >>> | ShiftRightUnsigned | S | |
Review Comment:
The headline count was incremented by 1, but this change marks two table
entries as supported (`>>>` and `shiftrightunsigned`). If the headline count is
intended to track unique Spark scalar *functions* (excluding operators /
aliases), consider clarifying that in the doc text; otherwise, update the
totals to match how the table is counted to avoid inconsistency.
##########
backends-velox/src/test/scala/org/apache/gluten/functions/ScalarFunctionsValidateSuite.scala:
##########
@@ -197,6 +197,54 @@ class ScalarFunctionsValidateSuite extends
FunctionsValidateSuite {
}
}
+ test("shiftrightunsigned") {
+ withTempPath {
+ path =>
+ withTempView("shift_input") {
+ Seq[(Integer, java.lang.Long, Integer)](
+ (8, 8L, 1),
+ (-1, -1L, 1),
+ (Int.MinValue, Long.MinValue, 1),
+ (Int.MinValue, Long.MinValue, 31),
+ (-1, -1L, 32),
+ (-1, -1L, 33),
+ (-1, -1L, 63),
+ (-1, -1L, 64),
+ (-1, -1L, 65),
+ (-1, -1L, -1),
+ (Int.MinValue, Long.MinValue, Int.MaxValue),
+ (null, null, 1),
+ (1, 1L, null)
+ )
+ .toDF("int_value", "long_value", "shift")
+ .write
+ .parquet(path.getCanonicalPath)
+
+
spark.read.parquet(path.getCanonicalPath).createOrReplaceTempView("shift_input")
+ runQueryAndCompare(
+ """
+ |SELECT
+ | shiftrightunsigned(int_value, shift),
+ | shiftrightunsigned(long_value, shift)
+ |FROM shift_input
+ |""".stripMargin) {
+ checkGlutenPlan[ProjectExecTransformer]
+ }
+ if (isSparkVersionGE("4.0")) {
+ runQueryAndCompare(
+ """
+ |SELECT
+ | int_value >>> shift,
+ | long_value >>> shift
+ |FROM shift_input
+ |""".stripMargin) {
+ checkGlutenPlan[ProjectExecTransformer]
+ }
Review Comment:
Both queries omit an `ORDER BY`, so row ordering is undefined. If
`runQueryAndCompare` performs order-sensitive comparison, this can make the
test flaky. Consider adding a deterministic `ORDER BY` (e.g., by `int_value`,
`long_value`, `shift`) or ensuring the helper compares results as unordered
sets.
##########
docs/backends/velox/velox-backend-scalar-function-support.md:
##########
@@ -42,14 +42,14 @@ The status applies to `spark.sql.ansi.enabled=false`. When
ANSI mode is enabled,
| & | BitwiseAnd | S | |
| << | ShiftLeft | S | |
| >> | ShiftRight | S | |
-| >>> | ShiftRightUnsigned | | |
+| >>> | ShiftRightUnsigned | S | |
| ^ | BitwiseXor | S | |
| bit_count | BitwiseCount | S | |
| bit_get | BitwiseGet | S | |
| getbit | BitwiseGet | S | |
| shiftleft | ShiftLeft | S | |
| shiftright | ShiftRight | S | |
-| shiftrightunsigned | ShiftRightUnsigned | | |
+| shiftrightunsigned | ShiftRightUnsigned | S | |
Review Comment:
The headline count was incremented by 1, but this change marks two table
entries as supported (`>>>` and `shiftrightunsigned`). If the headline count is
intended to track unique Spark scalar *functions* (excluding operators /
aliases), consider clarifying that in the doc text; otherwise, update the
totals to match how the table is counted to avoid inconsistency.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
