John Zhuge created SPARK-60328:
----------------------------------
Summary: Clear test jar from every artifact bucket in HiveDDLSuite
SPARK-34261 test
Key: SPARK-60328
URL: https://issues.apache.org/jira/browse/SPARK-60328
Project: Spark
Issue Type: Test
Components: SQL
Affects Versions: 4.2.0
Reporter: John Zhuge
The cleanup in the HiveDDLSuite test "SPARK-34261: Avoid side effect if create
exists temporary function" removes the test jar only from the "default" bucket
of SparkContext.addedJars:
{code:scala}
.foreach(k => spark.sparkContext.addedJars.get("default").foreach(_.remove(k)))
{code}
But SQL commands run inside artifactManager.withResources, and
*spark.sql.artifact.isolation.enabled *defaults to true, so jars added through
SQL are stored under the session UUID. listJars() flattens all buckets, so a
jar left behind by an earlier test survives this cleanup and the next assertion
fails, depending on test order.
Fix: remove the jar from every bucket. Test-only change.
{code:scala}
.foreach(k => spark.sparkContext.addedJars.values.foreach(_.remove(k)))
{code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
