[GitHub] spark pull request #23138: [SPARK-23356][SQL][TEST] add new test cases for a...
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/23138 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request #23138: [SPARK-23356][SQL][TEST] add new test cases for a...
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/23138#discussion_r236334056
--- Diff:
sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/optimizer/SetOperationSuite.scala
---
@@ -196,4 +196,31 @@ class SetOperationSuite extends PlanTest {
))
comparePlans(expectedPlan, rewrittenPlan)
}
+
+ test("SPARK-23356 union: expressions with number in project list are
pushed down") {
--- End diff --
`expressions with number` -> `expressions with literal`
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request #23138: [SPARK-23356][SQL][TEST] add new test cases for a...
Github user gatorsmile commented on a diff in the pull request:
https://github.com/apache/spark/pull/23138#discussion_r236136273
--- Diff:
sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/optimizer/SetOperationSuite.scala
---
@@ -196,4 +196,31 @@ class SetOperationSuite extends PlanTest {
))
comparePlans(expectedPlan, rewrittenPlan)
}
+
+ test("SPARK-23356 union: expressions in project list are addition to
each side") {
+val unionQuery = testUnion.select(('a + 1).as("aa"))
+val unionOptimized = Optimize.execute(unionQuery.analyze)
+val unionCorrectAnswer =
+ Union(testRelation.select(('a + 1).as("aa")) ::
+testRelation2.select(('d + 1).as("aa")) ::
+testRelation3.select(('g + 1).as("aa")) :: Nil).analyze
+comparePlans(unionOptimized, unionCorrectAnswer)
+ }
+
+ test("SPARK-23356 union: expressions in project list are attribute
addition to each side") {
+val unionQuery = testUnion.select(('a + 'b).as("ab"))
+val unionOptimized = Optimize.execute(unionQuery.analyze)
+val unionCorrectAnswer =
+ Union(testRelation.select(('a + 'b).as("ab")) ::
+testRelation2.select(('d + 'e).as("ab")) ::
+testRelation3.select(('g + 'h).as("ab")) :: Nil).analyze
+comparePlans(unionOptimized, unionCorrectAnswer)
+ }
+
+ test("SPARK-23356 union: project don't each side with non-deterministic
expression") {
--- End diff --
no pushdown for non-deterministic expression
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request #23138: [SPARK-23356][SQL][TEST] add new test cases for a...
Github user gatorsmile commented on a diff in the pull request:
https://github.com/apache/spark/pull/23138#discussion_r236136228
--- Diff:
sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/optimizer/SetOperationSuite.scala
---
@@ -196,4 +196,31 @@ class SetOperationSuite extends PlanTest {
))
comparePlans(expectedPlan, rewrittenPlan)
}
+
+ test("SPARK-23356 union: expressions in project list are addition to
each side") {
+val unionQuery = testUnion.select(('a + 1).as("aa"))
+val unionOptimized = Optimize.execute(unionQuery.analyze)
+val unionCorrectAnswer =
+ Union(testRelation.select(('a + 1).as("aa")) ::
+testRelation2.select(('d + 1).as("aa")) ::
+testRelation3.select(('g + 1).as("aa")) :: Nil).analyze
+comparePlans(unionOptimized, unionCorrectAnswer)
+ }
+
+ test("SPARK-23356 union: expressions in project list are attribute
addition to each side") {
--- End diff --
the same here
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request #23138: [SPARK-23356][SQL][TEST] add new test cases for a...
Github user gatorsmile commented on a diff in the pull request:
https://github.com/apache/spark/pull/23138#discussion_r236136178
--- Diff:
sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/optimizer/SetOperationSuite.scala
---
@@ -196,4 +196,31 @@ class SetOperationSuite extends PlanTest {
))
comparePlans(expectedPlan, rewrittenPlan)
}
+
+ test("SPARK-23356 union: expressions in project list are addition to
each side") {
--- End diff --
`are addition to each side` -> `are pushed down`
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
