[spark] branch master updated: [SPARK-44978][SQL][TEST] Fix SQLQueryTestSuite unable create table normally

[gurwls223]

This is an automated email from the ASF dual-hosted git repository.

gurwls223 pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/spark.git


The following commit(s) were added to refs/heads/master by this push:
     new fb3b0684e9c [SPARK-44978][SQL][TEST] Fix SQLQueryTestSuite unable 
create table normally
fb3b0684e9c is described below

commit fb3b0684e9cb5768d4a090ab53457957dac4a6d9
Author: Jia Fan <fanjiaemi...@qq.com>
AuthorDate: Mon Aug 28 10:22:19 2023 +0900

    [SPARK-44978][SQL][TEST] Fix SQLQueryTestSuite unable create table normally
    
    ### What changes were proposed in this pull request?
    When we repeatedly execute `SQLQueryTestSuite` to generate the golden file, 
the warehouse file executed last time is not cleaned up (maybe killed when test 
not finish), resulting in an error result.
    
![image](https://github.com/apache/spark/assets/32387433/407c0d2f-297a-45c3-9e4a-fbc974b83cba)
    
    So this PR clean all warehouse file in `SQLQueryTestSuite` before run test 
case.
    
    ### Why are the changes needed?
    Fix annoying error generating golden files
    
    ### Does this PR introduce _any_ user-facing change?
    No
    
    ### How was this patch tested?
    exist test. Also test in local.
    
    ### Was this patch authored or co-authored using generative AI tooling?
    No
    
    Closes #42691 from Hisoka-X/SPARK-44978_clean_all_warehouse.
    
    Authored-by: Jia Fan <fanjiaemi...@qq.com>
    Signed-off-by: Hyukjin Kwon <gurwls...@apache.org>
---
 .../test/scala/org/apache/spark/sql/SQLQueryTestSuite.scala    | 10 ++++------
 1 file changed, 4 insertions(+), 6 deletions(-)

diff --git 
a/sql/core/src/test/scala/org/apache/spark/sql/SQLQueryTestSuite.scala 
b/sql/core/src/test/scala/org/apache/spark/sql/SQLQueryTestSuite.scala
index 71af1fd69c3..0b27f80a5d0 100644
--- a/sql/core/src/test/scala/org/apache/spark/sql/SQLQueryTestSuite.scala
+++ b/sql/core/src/test/scala/org/apache/spark/sql/SQLQueryTestSuite.scala
@@ -689,12 +689,10 @@ class SQLQueryTestSuite extends QueryTest with 
SharedSparkSession with SQLHelper
   protected def createTestTables(session: SparkSession): Unit = {
     import session.implicits._
 
-    // Before creating test tables, deletes orphan directories in warehouse dir
-    Seq("testdata", "arraydata", "mapdata", "aggtest", "onek", 
"tenk1").foreach { dirName =>
-      val f = new File(new URI(s"${conf.warehousePath}/$dirName"))
-      if (f.exists()) {
-        Utils.deleteRecursively(f)
-      }
+    // Before creating test tables, deletes warehouse dir
+    val f = new File(new URI(conf.warehousePath))
+    if (f.exists()) {
+      Utils.deleteRecursively(f)
     }
 
     (1 to 100).map(i => (i, i.toString)).toDF("key", "value")


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org
For additional commands, e-mail: commits-h...@spark.apache.org