yaooqinn commented on a change in pull request #27933: [SPARK-31170][SQL] Spark
SQL Cli should respect hive-site.xml and spark.sql.warehouse.dir
URL: https://github.com/apache/spark/pull/27933#discussion_r393525824
##########
File path:
sql/hive-thriftserver/src/test/scala/org/apache/spark/sql/hive/thriftserver/CliSuite.scala
##########
@@ -159,6 +160,17 @@ class CliSuite extends SparkFunSuite with
BeforeAndAfterAll with Logging {
}
}
+ test("Pick spark.sql.warehouse.dir first for Spark Cli if set") {
+ val sparkWareHouseDir = Utils.createTempDir()
+ new File(warehousePath, "metastore_db").delete()
+ runCliWithin(
+ 1.minute,
+ Seq("--conf",
s"""${StaticSQLConf.WAREHOUSE_PATH.key}=$sparkWareHouseDir"""))(
+ """desc database default;""" -> sparkWareHouseDir.getAbsolutePath
+ )
+ sparkWareHouseDir.delete()
Review comment:
In this test, we will run a spark-sql job to check the output
```
[info] - Pick spark.sql.warehouse.dir first for Spark Cli if set *** FAILED
*** (1 minute)
[info] =======================
[info] CliSuite failure output
[info] =======================
[info] Spark SQL CLI command line: ../../bin/spark-sql --master local
--driver-java-options -Dderby.system.durability=test --conf
spark.ui.enabled=false --hiveconf
javax.jdo.option.ConnectionURL=jdbc:derby:;databaseName=/Users/kentyao/spark/target/tmp/spark-3570629a-23a7-4e2a-b21a-9bce68025228;create=true
--hiveconf
hive.metastore.warehouse.dir=/Users/kentyao/spark/target/tmp/spark-dbe4dd6a-1129-43f5-83bc-62dd2072b410
--hiveconf
hive.exec.scratchdir=/Users/kentyao/spark/target/tmp/spark-9422968c-94d0-4251-b591-30cbcc061845
--hiveconf conf1=conftest --hiveconf conf2=1 --conf
spark.sql.warehouse.dir=/Users/kentyao/spark/target/tmp/spark-1219faf2-918f-4ac4-8d28-fa4dd2d24c7e
[info] Exception: java.util.concurrent.TimeoutException: Futures timed out
after [1 minute]
[info] Executed query 0 "desc database default;",
[info] But failed to capture expected output
"/Users/kentyao/spark/target/tmp/spark-1219faf2-918f-4ac4-8d28-fa4dd2d24c7e"
within 1 minute.
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
