[GitHub] spark pull request #14930: [SPARK-16926] [SQL] Add unit test to compare tabl...
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/14930 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request #14930: [SPARK-16926] [SQL] Add unit test to compare tabl...
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/14930#discussion_r77287554
--- Diff:
sql/hive/src/test/scala/org/apache/spark/sql/hive/execution/HiveTableScanSuite.scala
---
@@ -143,4 +144,38 @@ class HiveTableScanSuite extends HiveComparisonTest
with SQLTestUtils with TestH
}
}
}
+
+ test("SPARK-16926: number of table and partition columns match for new
partitioned table") {
+val view = "src"
+withTempView(view) {
+ spark.range(1, 5).createOrReplaceTempView(view)
+ val table = "table_with_partition"
+ withTable(table) {
+sql(
+ s"""
+ |CREATE TABLE $table(id string)
+ |PARTITIONED BY (p1 string,p2 string,p3 string,p4 string,p5
string)
+ """.stripMargin)
+sql(
+ s"""
+ |FROM $view v
+ |INSERT INTO TABLE $table
+ |PARTITION (p1='a',p2='b',p3='c',p4='d',p5='e')
+ |SELECT v.id
+ |INSERT INTO TABLE $table
--- End diff --
why do we insert the same value to the table 2 times?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request #14930: [SPARK-16926] [SQL] Add unit test to compare tabl...
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/14930#discussion_r77285020
--- Diff:
sql/hive/src/test/scala/org/apache/spark/sql/hive/execution/HiveTableScanSuite.scala
---
@@ -143,4 +144,38 @@ class HiveTableScanSuite extends HiveComparisonTest
with SQLTestUtils with TestH
}
}
}
+
+ test("SPARK-16926: number of table and partition columns match for new
partitioned table") {
+val view = "src"
+withTempView(view) {
+ spark.range(1, 5).createOrReplaceTempView(view)
--- End diff --
oh sorry I read the code wrong...
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request #14930: [SPARK-16926] [SQL] Add unit test to compare tabl...
Github user dafrista commented on a diff in the pull request:
https://github.com/apache/spark/pull/14930#discussion_r77284176
--- Diff:
sql/hive/src/test/scala/org/apache/spark/sql/hive/execution/HiveTableScanSuite.scala
---
@@ -143,4 +144,38 @@ class HiveTableScanSuite extends HiveComparisonTest
with SQLTestUtils with TestH
}
}
}
+
+ test("SPARK-16926: number of table and partition columns match for new
partitioned table") {
+val view = "src"
+withTempView(view) {
+ spark.range(1, 5).createOrReplaceTempView(view)
--- End diff --
My understanding is that `createOrReplaceTempView` should ensure the same
thing because any existing view would be replaced. I'm I missing something?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request #14930: [SPARK-16926] [SQL] Add unit test to compare tabl...
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/14930#discussion_r77281400
--- Diff:
sql/hive/src/test/scala/org/apache/spark/sql/hive/execution/HiveTableScanSuite.scala
---
@@ -143,4 +144,38 @@ class HiveTableScanSuite extends HiveComparisonTest
with SQLTestUtils with TestH
}
}
}
+
+ test("SPARK-16926: number of table and partition columns match for new
partitioned table") {
+val view = "src"
+withTempView(view) {
+ spark.range(1, 5).createOrReplaceTempView(view)
--- End diff --
nit: we should alway use `createTempView` in test, to make sure the view we
created is what we want, rather than some existing view.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request #14930: [SPARK-16926] [SQL] Add unit test to compare tabl...
GitHub user dafrista opened a pull request: https://github.com/apache/spark/pull/14930 [SPARK-16926] [SQL] Add unit test to compare table and partition column metadata. ## What changes were proposed in this pull request? Add unit test for changes made in PR #14515. It makes sure that a newly created table has the same number of columns in table and partition metadata. This test fails before the changes introduced in #14515. ## How was this patch tested? Run new unit test. You can merge this pull request into a Git repository by running: $ git pull https://github.com/dafrista/spark partition-metadata-unit-test Alternatively you can review and apply these changes as the patch at: https://github.com/apache/spark/pull/14930.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #14930 commit 7ac4d63cafc854aa4a5c3622b668474da1255022 Author: Brian Cho Date: 2016-09-01T21:48:37Z Add unit test to compare table and partition columns. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
