This is an automated email from the ASF dual-hosted git repository.
yao pushed a commit to branch branch-3.4
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/branch-3.4 by this push:
new 61868525785 [SPARK-45878][SQL][TESTS] Fix
ConcurrentModificationException in CliSuite
61868525785 is described below
commit 618685257859d5a34b50e22905c73f639bd2cc30
Author: Kent Yao <[email protected]>
AuthorDate: Fri Nov 10 21:09:43 2023 +0800
[SPARK-45878][SQL][TESTS] Fix ConcurrentModificationException in CliSuite
### What changes were proposed in this pull request?
This PR changes the ArrayBuffer for logs to immutable for reading to
prevent ConcurrentModificationException which hides the actual cause of failure
### Why are the changes needed?
```scala
[info] - SPARK-29022 Commands using SerDe provided in ADD JAR sql ***
FAILED *** (11 seconds, 105 milliseconds)
[info] java.util.ConcurrentModificationException: mutation occurred
during iteration
[info] at
scala.collection.mutable.MutationTracker$.checkMutations(MutationTracker.scala:43)
[info] at
scala.collection.mutable.CheckedIndexedSeqView$CheckedIterator.hasNext(CheckedIndexedSeqView.scala:47)
[info] at
scala.collection.IterableOnceOps.addString(IterableOnce.scala:1247)
[info] at
scala.collection.IterableOnceOps.addString$(IterableOnce.scala:1241)
[info] at scala.collection.AbstractIterable.addString(Iterable.scala:933)
[info] at
org.apache.spark.sql.hive.thriftserver.CliSuite.runCliWithin(CliSuite.scala:205)
[info] at
org.apache.spark.sql.hive.thriftserver.CliSuite.$anonfun$new$20(CliSuite.scala:501)
```
### Does this PR introduce _any_ user-facing change?
no
### How was this patch tested?
existing tests
### Was this patch authored or co-authored using generative AI tooling?
no
Closes #43749 from yaooqinn/SPARK-45878.
Authored-by: Kent Yao <[email protected]>
Signed-off-by: Kent Yao <[email protected]>
(cherry picked from commit b347237735094e9092f4100583ed1d6f3eacf1f6)
Signed-off-by: Kent Yao <[email protected]>
---
.../test/scala/org/apache/spark/sql/hive/thriftserver/CliSuite.scala | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)
diff --git
a/sql/hive-thriftserver/src/test/scala/org/apache/spark/sql/hive/thriftserver/CliSuite.scala
b/sql/hive-thriftserver/src/test/scala/org/apache/spark/sql/hive/thriftserver/CliSuite.scala
index 1b91442c228..d3721cf68ab 100644
---
a/sql/hive-thriftserver/src/test/scala/org/apache/spark/sql/hive/thriftserver/CliSuite.scala
+++
b/sql/hive-thriftserver/src/test/scala/org/apache/spark/sql/hive/thriftserver/CliSuite.scala
@@ -198,7 +198,7 @@ class CliSuite extends SparkFunSuite {
ThreadUtils.awaitResult(foundAllExpectedAnswers.future, timeoutForQuery)
log.info("Found all expected output.")
} catch { case cause: Throwable =>
- val message =
+ val message = lock.synchronized {
s"""
|=======================
|CliSuite failure output
@@ -212,6 +212,7 @@ class CliSuite extends SparkFunSuite {
|End CliSuite failure output
|===========================
""".stripMargin
+ }
logError(message, cause)
fail(message, cause)
} finally {
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
