This is an automated email from the ASF dual-hosted git repository.
gengliang pushed a commit to branch branch-3.0
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/branch-3.0 by this push:
new 95df63c [SPARK-30906][SQL][TESTS][FOLLOW-UP] Set the configuration
against TestHive explicitly in HiveSerDeSuite
95df63c is described below
commit 95df63c45fd9c8159ddee00014023eda10df93dc
Author: HyukjinKwon <gurwls...@apache.org>
AuthorDate: Wed Feb 26 18:01:26 2020 -0800
[SPARK-30906][SQL][TESTS][FOLLOW-UP] Set the configuration against TestHive
explicitly in HiveSerDeSuite
### What changes were proposed in this pull request?
After https://github.com/apache/spark/pull/27659 (see
https://amplab.cs.berkeley.edu/jenkins/job/spark-master-test-sbt-hadoop-2.7-hive-2.3/253/),
the tests below fail consistently, specifically in one job
https://amplab.cs.berkeley.edu/jenkins/job/spark-master-test-sbt-hadoop-2.7-hive-2.3/
in Jenkins
```
org.apache.spark.sql.hive.execution.HiveSerDeSuite.Test the default
fileformat for Hive-serde tables
```
The profile is same as PR builder but seems it fails specifically in this
machine.
Several configurations used in `HiveSerDeSuite` are not being set
presumably due to the inconsistency between `SQLConf.get` and the active Spark
session described in the https://github.com/apache/spark/pull/27387, and as a
side effect of the cloned session at https://github.com/apache/spark/pull/27659.
This PR proposes to explicitly set the configuration against `TestHive` by
using `withExistingConf` at `withSQLConf`
### Why are the changes needed?
To make `spark-master-test-sbt-hadoop-2.7-hive-2.3` job pass.
### Does this PR introduce any user-facing change?
No.
### How was this patch tested?
Cannot reproduce in my local. Presumably it cannot be reproduced in the PR
builder. We should see if the tests pass at
`spark-master-test-sbt-hadoop-2.7-hive-2.3` job after this PR is merged.
Closes #27705 from HyukjinKwon/SPARK-30906.
Authored-by: HyukjinKwon <gurwls...@apache.org>
Signed-off-by: Gengliang Wang <gengliang.w...@databricks.com>
(cherry picked from commit cd3ef2249fc6bd60e4ba62ee046dcf4115864206)
Signed-off-by: Gengliang Wang <gengliang.w...@databricks.com>
---
.../scala/org/apache/spark/sql/hive/execution/HiveSerDeSuite.scala | 4 ++++
1 file changed, 4 insertions(+)
diff --git
a/sql/hive/src/test/scala/org/apache/spark/sql/hive/execution/HiveSerDeSuite.scala
b/sql/hive/src/test/scala/org/apache/spark/sql/hive/execution/HiveSerDeSuite.scala
index 9a1190a..d2d3502 100644
---
a/sql/hive/src/test/scala/org/apache/spark/sql/hive/execution/HiveSerDeSuite.scala
+++
b/sql/hive/src/test/scala/org/apache/spark/sql/hive/execution/HiveSerDeSuite.scala
@@ -82,6 +82,10 @@ class HiveSerDeSuite extends HiveComparisonTest with
PlanTest with BeforeAndAfte
}.head
}
+ // Make sure we set the config values to TestHive.conf.
+ override def withSQLConf(pairs: (String, String)*)(f: => Unit): Unit =
+ SQLConf.withExistingConf(TestHive.conf)(super.withSQLConf(pairs: _*)(f))
+
test("Test the default fileformat for Hive-serde tables") {
withSQLConf("hive.default.fileformat" -> "orc",
SQLConf.LEGACY_CREATE_HIVE_TABLE_BY_DEFAULT_ENABLED.key -> "true") {
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org
For additional commands, e-mail: commits-h...@spark.apache.org
