[
https://issues.apache.org/jira/browse/SPARK-49228?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17873390#comment-17873390
]
Yang Jie commented on SPARK-49228:
----------------------------------
I can also reproduce this issue locally. The stack trace for the long-running
Runnable as follows:
{code:java}
"main" #1 prio=5 os_prio=31 cpu=177053.75ms elapsed=179.28s
tid=0x000000013000d200 nid=0x2803 runnable [0x000000016db1d000]
java.lang.Thread.State: RUNNABLE
at
java.util.stream.ReferencePipeline$2$1.accept([email protected]/ReferencePipeline.java:178)
at
java.util.HashMap$KeySpliterator.forEachRemaining([email protected]/HashMap.java:1707)
at
java.util.stream.AbstractPipeline.copyInto([email protected]/AbstractPipeline.java:509)
at
java.util.stream.AbstractPipeline.wrapAndCopyInto([email protected]/AbstractPipeline.java:499)
at
java.util.stream.ReduceOps$ReduceOp.evaluateSequential([email protected]/ReduceOps.java:921)
at
java.util.stream.AbstractPipeline.evaluate([email protected]/AbstractPipeline.java:234)
at
java.util.stream.LongPipeline.reduce([email protected]/LongPipeline.java:498)
at
java.util.stream.LongPipeline.sum([email protected]/LongPipeline.java:456)
at
org.apache.spark.memory.TaskMemoryManager.acquireExecutionMemory(TaskMemoryManager.java:219)
- locked <0x00000007084449e8> (a org.apache.spark.memory.TaskMemoryManager)
at
org.apache.spark.memory.TaskMemoryManager.allocatePage(TaskMemoryManager.java:348)
at
org.apache.spark.memory.MemoryConsumer.allocatePage(MemoryConsumer.java:117)
at
org.apache.spark.util.collection.unsafe.sort.UnsafeExternalSorter.acquireNewPageIfNecessary(UnsafeExternalSorter.java:437)
at
org.apache.spark.util.collection.unsafe.sort.UnsafeExternalSorter.allocateMemoryForRecordIfNecessary(UnsafeExternalSorter.java:456)
at
org.apache.spark.util.collection.unsafe.sort.UnsafeExternalSorter.insertRecord(UnsafeExternalSorter.java:491)
at
org.apache.spark.sql.execution.ExternalAppendOnlyUnsafeRowArray.$anonfun$add$2(ExternalAppendOnlyUnsafeRowArray.scala:151)
at
org.apache.spark.sql.execution.ExternalAppendOnlyUnsafeRowArray.$anonfun$add$2$adapted(ExternalAppendOnlyUnsafeRowArray.scala:145)
at
org.apache.spark.sql.execution.ExternalAppendOnlyUnsafeRowArray$$Lambda$888/0x000000b0015a3918.apply(Unknown
Source)
at scala.collection.IterableOnceOps.foreach(IterableOnce.scala:619)
at scala.collection.IterableOnceOps.foreach$(IterableOnce.scala:617)
at scala.collection.AbstractIterable.foreach(Iterable.scala:935)
at
org.apache.spark.sql.execution.ExternalAppendOnlyUnsafeRowArray.add(ExternalAppendOnlyUnsafeRowArray.scala:145)
at
org.apache.spark.sql.execution.ExternalAppendOnlyUnsafeRowArrayBenchmark$.$anonfun$testAgainstRawArrayBuffer$6(ExternalAppendOnlyUnsafeRowArrayBenchmark.scala:112)
at
org.apache.spark.sql.execution.ExternalAppendOnlyUnsafeRowArrayBenchmark$.$anonfun$testAgainstRawArrayBuffer$6$adapted(ExternalAppendOnlyUnsafeRowArrayBenchmark.scala:112)
at
org.apache.spark.sql.execution.ExternalAppendOnlyUnsafeRowArrayBenchmark$$$Lambda$879/0x000000b00159bbf8.apply(Unknown
Source)
at scala.collection.immutable.VectorStatics$.foreachRec(Vector.scala:2124)
at scala.collection.immutable.Vector.foreach(Vector.scala:2130)
at
org.apache.spark.sql.execution.ExternalAppendOnlyUnsafeRowArrayBenchmark$.$anonfun$testAgainstRawArrayBuffer$5(ExternalAppendOnlyUnsafeRowArrayBenchmark.scala:112)
at
org.apache.spark.sql.execution.ExternalAppendOnlyUnsafeRowArrayBenchmark$$$Lambda$878/0x000000b00159b800.apply$mcVJ$sp(Unknown
Source)
at scala.runtime.java8.JFunction1$mcVJ$sp.apply(JFunction1$mcVJ$sp.scala:18)
at
scala.collection.immutable.NumericRange.foreach$mVc$sp(NumericRange.scala:115)
at
org.apache.spark.sql.execution.ExternalAppendOnlyUnsafeRowArrayBenchmark$.$anonfun$testAgainstRawArrayBuffer$4(ExternalAppendOnlyUnsafeRowArrayBenchmark.scala:107)
at
org.apache.spark.sql.execution.ExternalAppendOnlyUnsafeRowArrayBenchmark$$$Lambda$301/0x000000b00127c000.apply$mcVI$sp(Unknown
Source)
at
org.apache.spark.benchmark.Benchmark.$anonfun$addCase$1(Benchmark.scala:77)
at
org.apache.spark.benchmark.Benchmark.$anonfun$addCase$1$adapted(Benchmark.scala:75)
at
org.apache.spark.benchmark.Benchmark$$Lambda$300/0x000000b0012778b0.apply(Unknown
Source)
at org.apache.spark.benchmark.Benchmark.measure(Benchmark.scala:140)
at org.apache.spark.benchmark.Benchmark.$anonfun$run$1(Benchmark.scala:106)
at
org.apache.spark.benchmark.Benchmark$$Lambda$865/0x000000b00158c610.apply(Unknown
Source)
at
scala.collection.StrictOptimizedIterableOps.map(StrictOptimizedIterableOps.scala:100)
at
scala.collection.StrictOptimizedIterableOps.map$(StrictOptimizedIterableOps.scala:87)
at scala.collection.mutable.ArrayBuffer.map(ArrayBuffer.scala:42)
at org.apache.spark.benchmark.Benchmark.run(Benchmark.scala:104)
at
org.apache.spark.sql.execution.ExternalAppendOnlyUnsafeRowArrayBenchmark$.$anonfun$testAgainstRawArrayBuffer$7(ExternalAppendOnlyUnsafeRowArrayBenchmark.scala:123)
at
org.apache.spark.sql.execution.ExternalAppendOnlyUnsafeRowArrayBenchmark$$$Lambda$302/0x000000b00127c3f8.apply$mcV$sp(Unknown
Source)
at
org.apache.spark.sql.execution.ExternalAppendOnlyUnsafeRowArrayBenchmark$.withFakeTaskContext(ExternalAppendOnlyUnsafeRowArrayBenchmark.scala:58)
at
org.apache.spark.sql.execution.ExternalAppendOnlyUnsafeRowArrayBenchmark$.testAgainstRawArrayBuffer(ExternalAppendOnlyUnsafeRowArrayBenchmark.scala:123)
at
org.apache.spark.sql.execution.ExternalAppendOnlyUnsafeRowArrayBenchmark$.$anonfun$runBenchmarkSuite$1(ExternalAppendOnlyUnsafeRowArrayBenchmark.scala:193)
at
org.apache.spark.sql.execution.ExternalAppendOnlyUnsafeRowArrayBenchmark$$$Lambda$294/0x000000b00125bec8.apply$mcV$sp(Unknown
Source)
at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.scala:18)
at
org.apache.spark.benchmark.BenchmarkBase.runBenchmark(BenchmarkBase.scala:42)
at
org.apache.spark.sql.execution.ExternalAppendOnlyUnsafeRowArrayBenchmark$.runBenchmarkSuite(ExternalAppendOnlyUnsafeRowArrayBenchmark.scala:190)
at org.apache.spark.benchmark.BenchmarkBase.main(BenchmarkBase.scala:72)
at
org.apache.spark.sql.execution.ExternalAppendOnlyUnsafeRowArrayBenchmark.main(ExternalAppendOnlyUnsafeRowArrayBenchmark.scala)
{code}
It seems that no other threads are BLOCKED, only this thread is in a
long-running RUNNABLE state, and there appears to be no change in its status. I
tried revert SPARK-48628 or reset the code base to a commit before
SPARK-48628, then
ExternalAppendOnlyUnsafeRowArrayBenchmark passed.
> Investigate ExternalAppendOnlyUnsafeRowArrayBenchmark
> -----------------------------------------------------
>
> Key: SPARK-49228
> URL: https://issues.apache.org/jira/browse/SPARK-49228
> Project: Spark
> Issue Type: Sub-task
> Components: Tests
> Affects Versions: 4.0.0
> Reporter: Dongjoon Hyun
> Priority: Major
>
> `ExternalAppendOnlyUnsafeRowArrayBenchmark` hangs in CI and is killed.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
