voonhous commented on code in PR #19926:
URL: https://github.com/apache/hudi/pull/19926#discussion_r4615565766


##########
.github/workflows/bot.yml:
##########
@@ -1222,14 +1224,14 @@ jobs:
           SPARK_PROFILE: ${{ matrix.sparkProfile }}
           SPARK_MODULES: ${{ matrix.sparkModules }}
         run:
-          mvn test -Punit-tests -Pjava17 -D"$SCALA_PROFILE" -D"$SPARK_PROFILE" 
-Dtest=skipJavaTests $SCALA_TEST_DML2_FILTER 
-Dsurefire.failIfNoSpecifiedTests=false -pl 
"hudi-common,$SPARK_COMMON_MODULES,$SPARK_MODULES" $MVN_ARGS -Djacoco.skip=false
+          mvn test -Punit-tests -Pjava17 -D"$SCALA_PROFILE" -D"$SPARK_PROFILE" 
-Dtest=skipJavaTests $SCALA_TEST_DML2_FILTER 
-Dsurefire.failIfNoSpecifiedTests=false -pl 
"hudi-common,$SPARK_COMMON_MODULES,$SPARK_MODULES" $MVN_ARGS 
-Djacoco.skip=false -Dhudi.spark.test.sharedSession=true

Review Comment:
   Fixed: both partition-pruning tests now keep the `StageParallelismListener` 
they add and remove it in `finally`, so it does not outlive the test when the 
`SparkContext` is shared.
   



##########
hudi-spark-datasource/hudi-spark/src/test/scala/org/apache/spark/sql/hudi/common/HoodieSparkSqlTestBase.scala:
##########
@@ -118,23 +157,68 @@ class HoodieSparkSqlTestBase extends FunSuite with 
BeforeAndAfterAll {
         // it, so a throwing or non-withRecordType INMEMORY test would 
otherwise leak state here.
         // Runs before the catalog cleanup so it holds even if a drop throws.
         HoodieInMemoryHashIndex.clear()
-        val catalog = spark.sessionState.catalog
-        catalog.listDatabases().foreach { db =>
-          catalog.listTables(db).foreach { table =>
-            catalog.dropTable(table, true, true)
-          }
-        }
+        dropSuiteTables()
       }
     )
   }
 
+  /**
+   * Shared mode: scalatest runs each suite on its own thread and Hudi reads 
SparkSession.active in a few
+   * places (HoodieCatalog captures it when the session's catalog is first 
built, BaseProcedure per CALL),
+   * so pin this suite's child session to the test thread and fail fast if the 
session's HoodieCatalog was
+   * built against another session.
+   */
+  private def bindSuiteSession(): Unit = {
+    SparkSession.setActiveSession(spark)
+    
spark.sessionState.catalogManager.catalog(CatalogManager.SESSION_CATALOG_NAME) 
match {
+      case hoodieCatalog: HoodieCatalog =>
+        assert(hoodieCatalog.spark eq spark,
+          s"HoodieCatalog of ${getClass.getSimpleName} is bound to another 
SparkSession")
+      case _ =>
+    }
+  }
+
+  /**
+   * Drops the tables a test left behind. Per-suite mode owns the whole 
catalog. Shared mode shares the
+   * external catalog with every other suite in the JVM, so it drops only this 
suite's generateTableName
+   * tables plus any table whose name is not of that form (fixed names and 
temp views); under serial
+   * execution those can only come from the test that just ran. Fixed names 
are renamed in a later step.
+   */
+  private def dropSuiteTables(): Unit = {
+    val catalog = spark.sessionState.catalog
+    catalog.listDatabases().foreach { db =>

Review Comment:
   Fixed in `dropSuiteTables`: in shared mode it now also lists `global_temp` 
and drops the global views whose name carries the suite's `generateTableName` 
prefix. Every global view in the suites (`TestCopyToTempViewProcedure`, 
`TestDropTable`) is named that way, and matching on the prefix only keeps it 
safe once suites run concurrently.
   



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to