This is an automated email from the ASF dual-hosted git repository.
yao pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/master by this push:
new b52645652eff [SPARK-48168][SQL][FOLLOWUP] Fix bitwise shifting
operator's precedence
b52645652eff is described below
commit b52645652eff35345c868dc47e50b3970f3a7002
Author: Kent Yao <[email protected]>
AuthorDate: Mon May 27 17:55:17 2024 +0800
[SPARK-48168][SQL][FOLLOWUP] Fix bitwise shifting operator's precedence
### What changes were proposed in this pull request?
After referencing both `C`, `MySQL`'s doc,
https://en.cppreference.com/w/c/language/operator_precedence
https://dev.mysql.com/doc/refman/8.0/en/operator-precedence.html
And doing some experiments on scala-shell
```scala
scala> 1 & 2 >> 1
val res0: Int = 1
scala> 2 >> 1 << 1
val res1: Int = 2
scala> 1 << 1 + 2
val res2: Int = 8
```
The suitable precedence for `<< >> >>>` is between '+/-' and '&' with a
left-to-right associativity.
### Why are the changes needed?
bugfix
### Does this PR introduce _any_ user-facing change?
now, unreleased yet
### How was this patch tested?
new tests
### Was this patch authored or co-authored using generative AI tooling?
no
Closes #46753 from yaooqinn/SPARK-48168-F.
Authored-by: Kent Yao <[email protected]>
Signed-off-by: Kent Yao <[email protected]>
---
.../spark/sql/catalyst/parser/SqlBaseParser.g4 | 2 +-
.../sql-tests/analyzer-results/bitwise.sql.out | 21 +++++++++++++++++++
.../test/resources/sql-tests/inputs/bitwise.sql | 6 +++++-
.../resources/sql-tests/results/bitwise.sql.out | 24 ++++++++++++++++++++++
4 files changed, 51 insertions(+), 2 deletions(-)
diff --git
a/sql/api/src/main/antlr4/org/apache/spark/sql/catalyst/parser/SqlBaseParser.g4
b/sql/api/src/main/antlr4/org/apache/spark/sql/catalyst/parser/SqlBaseParser.g4
index f0c0adb88121..4552c17e0cf1 100644
---
a/sql/api/src/main/antlr4/org/apache/spark/sql/catalyst/parser/SqlBaseParser.g4
+++
b/sql/api/src/main/antlr4/org/apache/spark/sql/catalyst/parser/SqlBaseParser.g4
@@ -986,11 +986,11 @@ valueExpression
| operator=(MINUS | PLUS | TILDE) valueExpression
#arithmeticUnary
| left=valueExpression operator=(ASTERISK | SLASH | PERCENT | DIV)
right=valueExpression #arithmeticBinary
| left=valueExpression operator=(PLUS | MINUS | CONCAT_PIPE)
right=valueExpression #arithmeticBinary
+ | left=valueExpression shiftOperator right=valueExpression
#shiftExpression
| left=valueExpression operator=AMPERSAND right=valueExpression
#arithmeticBinary
| left=valueExpression operator=HAT right=valueExpression
#arithmeticBinary
| left=valueExpression operator=PIPE right=valueExpression
#arithmeticBinary
| left=valueExpression comparisonOperator right=valueExpression
#comparison
- | left=valueExpression shiftOperator right=valueExpression
#shiftExpression
;
shiftOperator
diff --git
a/sql/core/src/test/resources/sql-tests/analyzer-results/bitwise.sql.out
b/sql/core/src/test/resources/sql-tests/analyzer-results/bitwise.sql.out
index fee226c0c341..1267a984565a 100644
--- a/sql/core/src/test/resources/sql-tests/analyzer-results/bitwise.sql.out
+++ b/sql/core/src/test/resources/sql-tests/analyzer-results/bitwise.sql.out
@@ -418,3 +418,24 @@ select cast(null as map<int, array<int>>), 20181117 >> 2
-- !query analysis
Project [cast(null as map<int,array<int>>) AS NULL#x, (20181117 >> 2) AS
(20181117 >> 2)#x]
+- OneRowRelation
+
+
+-- !query
+select 1 << 1 + 2 as plus_over_shift
+-- !query analysis
+Project [(1 << (1 + 2)) AS plus_over_shift#x]
++- OneRowRelation
+
+
+-- !query
+select 2 >> 1 << 1 as left_to_right
+-- !query analysis
+Project [((2 >> 1) << 1) AS left_to_right#x]
++- OneRowRelation
+
+
+-- !query
+select 1 & 2 >> 1 as shift_over_ampersand
+-- !query analysis
+Project [(1 & (2 >> 1)) AS shift_over_ampersand#x]
++- OneRowRelation
diff --git a/sql/core/src/test/resources/sql-tests/inputs/bitwise.sql
b/sql/core/src/test/resources/sql-tests/inputs/bitwise.sql
index 5823b22ef645..e080fdd32a4a 100644
--- a/sql/core/src/test/resources/sql-tests/inputs/bitwise.sql
+++ b/sql/core/src/test/resources/sql-tests/inputs/bitwise.sql
@@ -86,4 +86,8 @@ SELECT 20181117 <<< 2;
SELECT 20181117 >>>> 2;
select cast(null as array<array<int>>), 20181117 >> 2;
select cast(null as array<array<int>>), 20181117 >>> 2;
-select cast(null as map<int, array<int>>), 20181117 >> 2;
\ No newline at end of file
+select cast(null as map<int, array<int>>), 20181117 >> 2;
+
+select 1 << 1 + 2 as plus_over_shift; -- if correct, the result is 8.
otherwise, 4
+select 2 >> 1 << 1 as left_to_right; -- if correct, the result is 2.
otherwise, 0
+select 1 & 2 >> 1 as shift_over_ampersand; -- if correct, the result is 1.
otherwise, 0
diff --git a/sql/core/src/test/resources/sql-tests/results/bitwise.sql.out
b/sql/core/src/test/resources/sql-tests/results/bitwise.sql.out
index a7ebaea293bf..7233b0d0ae49 100644
--- a/sql/core/src/test/resources/sql-tests/results/bitwise.sql.out
+++ b/sql/core/src/test/resources/sql-tests/results/bitwise.sql.out
@@ -450,3 +450,27 @@ select cast(null as map<int, array<int>>), 20181117 >> 2
struct<NULL:map<int,array<int>>,(20181117 >> 2):int>
-- !query output
NULL 5045279
+
+
+-- !query
+select 1 << 1 + 2 as plus_over_shift
+-- !query schema
+struct<plus_over_shift:int>
+-- !query output
+8
+
+
+-- !query
+select 2 >> 1 << 1 as left_to_right
+-- !query schema
+struct<left_to_right:int>
+-- !query output
+2
+
+
+-- !query
+select 1 & 2 >> 1 as shift_over_ampersand
+-- !query schema
+struct<shift_over_ampersand:int>
+-- !query output
+1
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
