This is an automated email from the ASF dual-hosted git repository.

sarutak pushed a commit to branch branch-4.x
in repository https://gitbox.apache.org/repos/asf/spark.git


The following commit(s) were added to refs/heads/branch-4.x by this push:
     new 8fc2bc805880 [SPARK-60328][SQL][TEST] Use a unique jar copy in the 
HiveDDLSuite SPARK-34261 test
8fc2bc805880 is described below

commit 8fc2bc805880a7d0a142ac924aae90b1f3a69108
Author: John Zhuge <[email protected]>
AuthorDate: Mon Nov 2 14:17:06 2026 +0900

    [SPARK-60328][SQL][TEST] Use a unique jar copy in the HiveDDLSuite 
SPARK-34261 test
    
    ### What changes were proposed in this pull request?
    
    The HiveDDLSuite test SPARK-34261: Avoid side effect if create exists 
temporary function now uses its own copy of TestUDTFJar.jar, with a name unique 
to this test. The copy cannot already be in `listJars()`, so the cleanup that 
removed the shared jar from `SparkContext.addedJars` is gone.
    
    ### Why are the changes needed?
    
    The old cleanup depends on how map `addedJars` is keyed. It clears only the 
"default" bucket and only the first matching key, but `listJars()` flattens 
every bucket. If the shared jar is registered under another bucket or a second 
key, the first assertion fails depending on test order. Clearing more buckets 
would also drop the jar from other live sessions. This line already needed a 
fix once when jar tracking changed (SPARK-46237).
    
    With a unique jar, the test only checks what it is meant to check: a 
failing `CREATE TEMPORARY FUNCTION ... USING JAR` does not add the jar, and 
`CREATE OR REPLACE` does.
    
    ### Does this PR introduce _any_ user-facing change?
    
    No. Test-only change.
    
    ### How was this patch tested?
    
    ```
    build/sbt -Phive 'hive/testOnly *HiveDDLSuite -- -z "SPARK-34261"'
    [info] - SPARK-34261: Avoid side effect if create exists temporary function
    [info] Tests: succeeded 1, failed 0, canceled 0, ignored 0, pending 0
    ```
    
    It also passes when the shared `TestUDTFJar.jar` is added with `ADD JAR` 
before the test body, which shows that leftover state from the shared jar no 
longer affects it.
    
    ### Was this patch authored or co-authored using generative AI tooling?
    
    Generated-by: Claude Code 2.1.301 (Claude Opus 5.5)
    
    Closes #59587 from jzhuge/SPARK-34261-followup-clear-all-buckets.
    
    Authored-by: John Zhuge <[email protected]>
    Signed-off-by: Kousuke Saruta <[email protected]>
    (cherry picked from commit 9e771a7bef9a5f5bba407dec3f5aebd5ec784adc)
    Signed-off-by: Kousuke Saruta <[email protected]>
---
 .../scala/org/apache/spark/sql/hive/execution/HiveDDLSuite.scala   | 7 ++++---
 1 file changed, 4 insertions(+), 3 deletions(-)

diff --git 
a/sql/hive/src/test/scala/org/apache/spark/sql/hive/execution/HiveDDLSuite.scala
 
b/sql/hive/src/test/scala/org/apache/spark/sql/hive/execution/HiveDDLSuite.scala
index 54d5cc734fd2..0f2cb6ab33e3 100644
--- 
a/sql/hive/src/test/scala/org/apache/spark/sql/hive/execution/HiveDDLSuite.scala
+++ 
b/sql/hive/src/test/scala/org/apache/spark/sql/hive/execution/HiveDDLSuite.scala
@@ -19,6 +19,7 @@ package org.apache.spark.sql.hive.execution
 
 import java.io.File
 import java.net.URI
+import java.nio.file.{Files, StandardCopyOption}
 import java.time.{LocalDateTime, LocalTime}
 import java.util.Locale
 
@@ -3318,11 +3319,11 @@ class HiveDDLSuite
     withUserDefinedFunction("f1" -> true) {
       sql("CREATE TEMPORARY FUNCTION f1 AS 
'org.apache.hadoop.hive.ql.udf.UDFUUID'")
 
-      val udtfJar = TestUDTFJar.jar
+      // A copy unique to this test cannot be registered by an earlier test.
+      val udtfJar = File.createTempFile("SPARK-34261-", ".jar", 
Utils.createTempDir())
+      Files.copy(TestUDTFJar.jar.toPath, udtfJar.toPath, 
StandardCopyOption.REPLACE_EXISTING)
       val jarName = udtfJar.getName
       val jar = udtfJar.toURI.toString
-      spark.sparkContext.allAddedJars.keys.find(_.contains(jarName))
-        .foreach(k => 
spark.sparkContext.addedJars.get("default").foreach(_.remove(k)))
       assert(!spark.sparkContext.listJars().exists(_.contains(jarName)))
       val e = intercept[AnalysisException] {
         sql("CREATE TEMPORARY FUNCTION f1 AS " +


---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to