This is an automated email from the ASF dual-hosted git repository.
dongjoon pushed a commit to branch branch-3.1
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/branch-3.1 by this push:
new 9dba27c [SPARK-35431][SQL][TESTS] Sort elements generated by
collect_set in SQLQueryTestSuite
9dba27c is described below
commit 9dba27c61a5451124afc7a4293986457e5d95177
Author: Takeshi Yamamuro <[email protected]>
AuthorDate: Mon May 17 22:51:32 2021 -0700
[SPARK-35431][SQL][TESTS] Sort elements generated by collect_set in
SQLQueryTestSuite
### What changes were proposed in this pull request?
To pass `subquery/scalar-subquery/scalar-subquery-select.sql`
(`SQLQueryTestSuite`) in Scala v2.13, this PR proposes to change the aggregate
expr of a test query in the file from `collect_set(...)` to
`sort_array(collect_set(...))` because `collect_set` depends on the
`mutable.HashSet` implementation and elements in the set are printed in a
different order in Scala v2.12/v2.13.
### Why are the changes needed?
To pass the test in Scala v2.13.
### Does this PR introduce _any_ user-facing change?
No, dev-only.
### How was this patch tested?
Manually checked.
Closes #32578 from maropu/FixSQLTestIssueInScala213.
Authored-by: Takeshi Yamamuro <[email protected]>
Signed-off-by: Dongjoon Hyun <[email protected]>
(cherry picked from commit 3b859a16c03fe0caaf8683d9cbc1d1c65551105a)
Signed-off-by: Dongjoon Hyun <[email protected]>
---
.../inputs/subquery/scalar-subquery/scalar-subquery-select.sql | 2 +-
.../results/subquery/scalar-subquery/scalar-subquery-select.sql.out | 4 ++--
2 files changed, 3 insertions(+), 3 deletions(-)
diff --git
a/sql/core/src/test/resources/sql-tests/inputs/subquery/scalar-subquery/scalar-subquery-select.sql
b/sql/core/src/test/resources/sql-tests/inputs/subquery/scalar-subquery/scalar-subquery-select.sql
index 81712bf..936da959 100644
---
a/sql/core/src/test/resources/sql-tests/inputs/subquery/scalar-subquery/scalar-subquery-select.sql
+++
b/sql/core/src/test/resources/sql-tests/inputs/subquery/scalar-subquery/scalar-subquery-select.sql
@@ -135,6 +135,6 @@ SELECT t1a,
(SELECT count_if(t2d > 0) FROM t2 WHERE t2a = t1a) count_if_t2,
(SELECT approx_count_distinct(t2d) FROM t2 WHERE t2a = t1a)
approx_count_distinct_t2,
(SELECT collect_list(t2d) FROM t2 WHERE t2a = t1a) collect_list_t2,
- (SELECT collect_set(t2d) FROM t2 WHERE t2a = t1a) collect_set_t2,
+ (SELECT sort_array(collect_set(t2d)) FROM t2 WHERE t2a = t1a)
collect_set_t2,
(SELECT hex(count_min_sketch(t2d, 0.5d, 0.5d, 1)) FROM t2 WHERE t2a = t1a)
collect_set_t2
FROM t1;
\ No newline at end of file
diff --git
a/sql/core/src/test/resources/sql-tests/results/subquery/scalar-subquery/scalar-subquery-select.sql.out
b/sql/core/src/test/resources/sql-tests/results/subquery/scalar-subquery/scalar-subquery-select.sql.out
index 16570c6..68aad89 100644
---
a/sql/core/src/test/resources/sql-tests/results/subquery/scalar-subquery/scalar-subquery-select.sql.out
+++
b/sql/core/src/test/resources/sql-tests/results/subquery/scalar-subquery/scalar-subquery-select.sql.out
@@ -204,7 +204,7 @@ SELECT t1a,
(SELECT count_if(t2d > 0) FROM t2 WHERE t2a = t1a) count_if_t2,
(SELECT approx_count_distinct(t2d) FROM t2 WHERE t2a = t1a)
approx_count_distinct_t2,
(SELECT collect_list(t2d) FROM t2 WHERE t2a = t1a) collect_list_t2,
- (SELECT collect_set(t2d) FROM t2 WHERE t2a = t1a) collect_set_t2,
+ (SELECT sort_array(collect_set(t2d)) FROM t2 WHERE t2a = t1a)
collect_set_t2,
(SELECT hex(count_min_sketch(t2d, 0.5d, 0.5d, 1)) FROM t2 WHERE t2a = t1a)
collect_set_t2
FROM t1
-- !query schema
@@ -215,7 +215,7 @@ val1a 0 0 0 [] []
0000000100000000000000000000000100000004000000005D8D6AB9000000
val1a 0 0 0 [] []
0000000100000000000000000000000100000004000000005D8D6AB90000000000000000000000000000000000000000000000000000000000000000
val1a 0 0 0 [] []
0000000100000000000000000000000100000004000000005D8D6AB90000000000000000000000000000000000000000000000000000000000000000
val1b 6 6 3 [19,119,319,19,19,19] [19,119,319]
0000000100000000000000060000000100000004000000005D8D6AB90000000000000000000000000000000400000000000000010000000000000001
-val1c 2 2 2 [219,19] [219,19]
0000000100000000000000020000000100000004000000005D8D6AB90000000000000000000000000000000100000000000000000000000000000001
+val1c 2 2 2 [219,19] [19,219]
0000000100000000000000020000000100000004000000005D8D6AB90000000000000000000000000000000100000000000000000000000000000001
val1d 0 0 0 [] []
0000000100000000000000000000000100000004000000005D8D6AB90000000000000000000000000000000000000000000000000000000000000000
val1d 0 0 0 [] []
0000000100000000000000000000000100000004000000005D8D6AB90000000000000000000000000000000000000000000000000000000000000000
val1d 0 0 0 [] []
0000000100000000000000000000000100000004000000005D8D6AB90000000000000000000000000000000000000000000000000000000000000000
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
