[jira] [Resolved] (SPARK-54386) JavaEncoderSuite testSimpleEncoders has potential to deadlock

[Asif (Jira)]

     [ 
https://issues.apache.org/jira/browse/SPARK-54386?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Asif resolved SPARK-54386.
--------------------------
    Resolution: Duplicate

> JavaEncoderSuite testSimpleEncoders has potential to deadlock
> -------------------------------------------------------------
>
>                 Key: SPARK-54386
>                 URL: https://issues.apache.org/jira/browse/SPARK-54386
>             Project: Spark
>          Issue Type: Bug
>          Components: Connect, SQL
>    Affects Versions: 4.1.0, 4.2.0
>            Reporter: Asif
>            Priority: Minor
>              Labels: connect, pull-request-available, sql
>
> The test JavaEncoderSuite.testSimpleEncoders has potential to cause deadlock 
> to the fact that Connect Server process spawned from the RemoteSparkSession, 
> does not consume from the server process 's input stream, causing the server 
> process's output streams ( output and error) to block while logging , in the 
> buffers  which get exhausted . As a result shuffle stage does not proceed, 
> with the deadlock in DagSchedulerEvent logging ...
> I have changes in my master ( but those are valid changes specific to my PRs) 
> and may be that has increased the logging... not sure.. need to look into 
> that ..
> The deadlock is not showing up in stock spark master..
> But after debugging and fixing the issue in my branch, the cause definitively 
> appears to be unconsumed process outout...
> Below is the stacktrace of the Connect Spark Server .. showing the 
> deadlock....caused by  the logging thread acquiring the Log4J 's class lock, 
> and then getting blocked on writing.. 
> please note that even though the thread is shown in the Runnable state, it is 
> actually blocked... The. reason for this jvm behaviour is available on web...
>  
> 2025-11-17 11:58:36
> Full thread dump OpenJDK 64-Bit Server VM (21.0.6 mixed mode, sharing):
>  
> Threads class SMR info:
> _java_thread_list=0x0000600003202280, length=76, elements=
> { 0x000000011f80d400, 0x000000013000c200, 0x000000010f8a9400, 
> 0x000000013000ca00, 0x000000013000d200, 0x0000000130012c00, 
> 0x0000000130013400, 0x0000000130013c00, 0x000000012f811400, 
> 0x000000012f82ac00, 0x000000011f998c00, 0x000000011fadd000, 
> 0x000000011faf4e00, 0x000000011fae1400, 0x000000011faf3800, 
> 0x000000011faf0c00, 0x000000011faf1400, 0x000000011faf1c00, 
> 0x000000011fade600, 0x000000011fadee00, 0x000000011fadf600, 
> 0x000000011fadfe00, 0x000000010fd39800, 0x000000010fd3a000, 
> 0x000000010fd54e00, 0x000000010fd55600, 0x000000010fd55e00, 
> 0x000000010fd56600, 0x000000010fd5a200, 0x000000010fd5aa00, 
> 0x0000000130706400, 0x000000010fd6f800, 0x000000010fd7e000, 
> 0x000000010fd7a000, 0x000000010fd7f600, 0x000000010fd84c00, 
> 0x000000012aba5400, 0x000000012abd6a00, 0x000000012abfd200, 
> 0x0000000130063600, 0x000000011fb12800, 0x000000011fb11e00, 
> 0x000000011fb17400, 0x000000013070b400, 0x000000011fb19c00, 
> 0x000000012ac0c400, 0x000000012ac15a00, 0x000000013071b400, 
> 0x000000011fb24400, 0x000000011fb24c00, 0x000000012ac12800, 
> 0x000000011fb48400, 0x000000012a01e000, 0x000000012b89c200, 
> 0x000000011fc61c00, 0x000000010f91be00, 0x000000010f91cc00, 
> 0x000000012b968000, 0x000000010f946800, 0x000000010f947000, 
> 0x000000010f94c400, 0x000000010f94cc00, 0x000000010fc79a00, 
> 0x000000010fc7a200, 0x000000010fc7aa00, 0x000000010fc7b200, 
> 0x000000010fc7ba00, 0x000000010fc7c200, 0x000000010e858000, 
> 0x00000001305aa400, 0x000000010e857600, 0x000000012a01d600, 
> 0x000000012bd71c00, 0x000000012bd1c400, 0x000000012c14f600, 
> 0x000000010e873a00 }
>  
> "main" #1 [6147] prio=5 os_prio=31 cpu=1553.09ms elapsed=45.28s 
> tid=0x000000011f80d400 nid=6147 runnable  [0x000000016dca9000]
>    java.lang.Thread.State: RUNNABLE
> at java.io.FileInputStream.readBytes(java.base@21.0.6/Native Method)
> at java.io.FileInputStream.read(java.base@21.0.6/FileInputStream.java:287)
> at 
> java.io.BufferedInputStream.read1(java.base@21.0.6/BufferedInputStream.java:345)
> at 
> java.io.BufferedInputStream.implRead(java.base@21.0.6/BufferedInputStream.java:420)
> at 
> java.io.BufferedInputStream.read(java.base@21.0.6/BufferedInputStream.java:399)
> at sun.nio.cs.StreamDecoder.readBytes(java.base@21.0.6/StreamDecoder.java:350)
> at sun.nio.cs.StreamDecoder.implRead(java.base@21.0.6/StreamDecoder.java:393)
> at 
> sun.nio.cs.StreamDecoder.lockedRead(java.base@21.0.6/StreamDecoder.java:217)
> at sun.nio.cs.StreamDecoder.read(java.base@21.0.6/StreamDecoder.java:171)
> at java.io.InputStreamReader.read(java.base@21.0.6/InputStreamReader.java:188)
> at java.io.BufferedReader.fill(java.base@21.0.6/BufferedReader.java:160)
> at 
> java.io.BufferedReader.implReadLine(java.base@21.0.6/BufferedReader.java:370)
> at java.io.BufferedReader.readLine(java.base@21.0.6/BufferedReader.java:347)
> at java.io.BufferedReader.readLine(java.base@21.0.6/BufferedReader.java:436)
> at scala.io.StdIn.readLine(StdIn.scala:30)
> at scala.io.StdIn.readLine$(StdIn.scala:30)
> at scala.io.StdIn$.readLine(StdIn.scala:241)
> at 
> org.apache.spark.sql.connect.SimpleSparkConnectService$.main(SimpleSparkConnectService.scala:52)
> at 
> org.apache.spark.sql.connect.SimpleSparkConnectService.main(SimpleSparkConnectService.scala)
> at 
> jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(java.base@21.0.6/Native 
> Method)
> at 
> jdk.internal.reflect.NativeMethodAccessorImpl.invoke(java.base@21.0.6/NativeMethodAccessorImpl.java:75)
> at 
> jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(java.base@21.0.6/DelegatingMethodAccessorImpl.java:52)
> at java.lang.reflect.Method.invoke(java.base@21.0.6/Method.java:580)
> at 
> org.apache.spark.deploy.JavaMainApplication.start(SparkApplication.scala:52)
> at 
> org.apache.spark.deploy.SparkSubmit.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:1028)
> at org.apache.spark.deploy.SparkSubmit.doRunMain$1(SparkSubmit.scala:203)
> at org.apache.spark.deploy.SparkSubmit.submit(SparkSubmit.scala:226)
> at org.apache.spark.deploy.SparkSubmit.doSubmit(SparkSubmit.scala:95)
> at 
> org.apache.spark.deploy.SparkSubmit$$anon$2.doSubmit(SparkSubmit.scala:1166)
> at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:1175)
> at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
>  
>    Locked ownable synchronizers:
>  - <0x00000007c17c13c0> (a 
> java.util.concurrent.locks.ReentrantLock$NonfairSync)
>  - <0x00000007c329dfa0> (a 
> java.util.concurrent.locks.ReentrantLock$NonfairSync)
>  - <0x00000007c329dfd0> (a 
> java.util.concurrent.locks.ReentrantLock$NonfairSync)
>  
> "Reference Handler" #9 [29443] daemon prio=10 os_prio=31 cpu=2.22ms 
> elapsed=45.27s tid=0x000000013000c200 nid=29443 waiting on condition  
> [0x000000016ee22000]
>    java.lang.Thread.State: RUNNABLE
> at 
> java.lang.ref.Reference.waitForReferencePendingList(java.base@21.0.6/Native 
> Method)
> at 
> java.lang.ref.Reference.processPendingReferences(java.base@21.0.6/Reference.java:246)
> at 
> java.lang.ref.Reference$ReferenceHandler.run(java.base@21.0.6/Reference.java:208)
>  
>    Locked ownable synchronizers:
>  - None
>  
> "Finalizer" #10 [29187] daemon prio=8 os_prio=31 cpu=0.14ms elapsed=45.27s 
> tid=0x000000010f8a9400 nid=29187 in Object.wait()  [0x000000016f02e000]
>    java.lang.Thread.State: WAITING (on object monitor)
> at java.lang.Object.wait0(java.base@21.0.6/Native Method)
>  - waiting on <0x00000007c131e0f0> (a java.lang.ref.NativeReferenceQueue$Lock)
> at java.lang.Object.wait(java.base@21.0.6/Object.java:366)
> at java.lang.Object.wait(java.base@21.0.6/Object.java:339)
> at 
> java.lang.ref.NativeReferenceQueue.await(java.base@21.0.6/NativeReferenceQueue.java:48)
> at 
> java.lang.ref.ReferenceQueue.remove0(java.base@21.0.6/ReferenceQueue.java:158)
> at 
> java.lang.ref.NativeReferenceQueue.remove(java.base@21.0.6/NativeReferenceQueue.java:89)
>  - locked <0x00000007c131e0f0> (a java.lang.ref.NativeReferenceQueue$Lock)
> at 
> java.lang.ref.Finalizer$FinalizerThread.run(java.base@21.0.6/Finalizer.java:173)
>  
>    Locked ownable synchronizers:
>  - None
>  
> "Signal Dispatcher" #11 [28675] daemon prio=9 os_prio=31 cpu=0.21ms 
> elapsed=45.27s tid=0x000000013000ca00 nid=28675 waiting on condition  
> [0x0000000000000000]
>    java.lang.Thread.State: RUNNABLE
>  
>    Locked ownable synchronizers:
>  - None
>  
> "Service Thread" #12 [24579] daemon prio=9 os_prio=31 cpu=46.92ms 
> elapsed=45.27s tid=0x000000013000d200 nid=24579 runnable  [0x0000000000000000]
>    java.lang.Thread.State: RUNNABLE
>  
>    Locked ownable synchronizers:
>  - None
>  
> "Monitor Deflation Thread" #13 [28419] daemon prio=9 os_prio=31 cpu=1.99ms 
> elapsed=45.27s tid=0x0000000130012c00 nid=28419 runnable  [0x0000000000000000]
>    java.lang.Thread.State: RUNNABLE
>  
>    Locked ownable synchronizers:
>  - None
>  
> "C2 CompilerThread0" #14 [27907] daemon prio=9 os_prio=31 cpu=2910.70ms 
> elapsed=45.27s tid=0x0000000130013400 nid=27907 waiting on condition  
> [0x0000000000000000]
>    java.lang.Thread.State: RUNNABLE
>    No compile task
>  
>    Locked ownable synchronizers:
>  - None
>  
> "C1 CompilerThread0" #17 [25347] daemon prio=9 os_prio=31 cpu=909.94ms 
> elapsed=45.27s tid=0x0000000130013c00 nid=25347 waiting on condition  
> [0x0000000000000000]
>    java.lang.Thread.State: RUNNABLE
>    No compile task
>  
>    Locked ownable synchronizers:
>  - None
>  
> "Common-Cleaner" #18 [27651] daemon prio=8 os_prio=31 cpu=2.50ms 
> elapsed=45.26s tid=0x000000012f811400 nid=27651 waiting on condition  
> [0x000000016fc76000]
>    java.lang.Thread.State: TIMED_WAITING (parking)
> at jdk.internal.misc.Unsafe.park(java.base@21.0.6/Native Method)
>  - parking to wait for  <0x00000007c131e258> (a 
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
> at 
> java.util.concurrent.locks.LockSupport.parkNanos(java.base@21.0.6/LockSupport.java:269)
> at 
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(java.base@21.0.6/AbstractQueuedSynchronizer.java:1852)
> at java.lang.ref.ReferenceQueue.await(java.base@21.0.6/ReferenceQueue.java:71)
> at 
> java.lang.ref.ReferenceQueue.remove0(java.base@21.0.6/ReferenceQueue.java:143)
> at 
> java.lang.ref.ReferenceQueue.remove(java.base@21.0.6/ReferenceQueue.java:218)
> at jdk.internal.ref.CleanerImpl.run(java.base@21.0.6/CleanerImpl.java:140)
> at java.lang.Thread.runWith(java.base@21.0.6/Thread.java:1596)
> at java.lang.Thread.run(java.base@21.0.6/Thread.java:1583)
> at 
> jdk.internal.misc.InnocuousThread.run(java.base@21.0.6/InnocuousThread.java:186)
>  
>    Locked ownable synchronizers:
>  - None
>  
> "Notification Thread" #19 [25859] daemon prio=9 os_prio=31 cpu=0.03ms 
> elapsed=45.25s tid=0x000000012f82ac00 nid=25859 runnable  [0x0000000000000000]
>    java.lang.Thread.State: RUNNABLE
>  
>    Locked ownable synchronizers:
>  - None
>  
> "org.apache.hadoop.fs.FileSystem$Statistics$StatisticsDataReferenceCleaner" 
> #31 [34307] daemon prio=5 os_prio=31 cpu=0.17ms elapsed=44.63s 
> tid=0x000000011f998c00 nid=34307 waiting on condition  [0x0000000179472000]
>    java.lang.Thread.State: WAITING (parking)
> at jdk.internal.misc.Unsafe.park(java.base@21.0.6/Native Method)
>  - parking to wait for  <0x00000007c1f84430> (a 
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
> at 
> java.util.concurrent.locks.LockSupport.park(java.base@21.0.6/LockSupport.java:371)
> at 
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionNode.block(java.base@21.0.6/AbstractQueuedSynchronizer.java:519)
> at 
> java.util.concurrent.ForkJoinPool.unmanagedBlock(java.base@21.0.6/ForkJoinPool.java:3780)
> at 
> java.util.concurrent.ForkJoinPool.managedBlock(java.base@21.0.6/ForkJoinPool.java:3725)
> at 
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(java.base@21.0.6/AbstractQueuedSynchronizer.java:1712)
> at java.lang.ref.ReferenceQueue.await(java.base@21.0.6/ReferenceQueue.java:67)
> at 
> java.lang.ref.ReferenceQueue.remove0(java.base@21.0.6/ReferenceQueue.java:158)
> at 
> java.lang.ref.ReferenceQueue.remove(java.base@21.0.6/ReferenceQueue.java:234)
> at 
> org.apache.hadoop.fs.FileSystem$Statistics$StatisticsDataReferenceCleaner.run(FileSystem.java:4171)
> at java.lang.Thread.runWith(java.base@21.0.6/Thread.java:1596)
> at java.lang.Thread.run(java.base@21.0.6/Thread.java:1583)
>  
>    Locked ownable synchronizers:
>  - None
>  
> "rpc-boss-3-1" #33 [41219] daemon prio=5 os_prio=31 cpu=3.54ms elapsed=43.95s 
> tid=0x000000011fadd000 nid=41219 runnable  [0x000000017967e000]
>    java.lang.Thread.State: RUNNABLE
> at io.netty.channel.kqueue.Native.keventWait(Native Method)
> at io.netty.channel.kqueue.Native.keventWait(Native.java:133)
> at 
> io.netty.channel.kqueue.KQueueIoHandler.kqueueWait(KQueueIoHandler.java:187)
> at 
> io.netty.channel.kqueue.KQueueIoHandler.kqueueWait(KQueueIoHandler.java:179)
> at io.netty.channel.kqueue.KQueueIoHandler.run(KQueueIoHandler.java:234)
> at 
> io.netty.channel.SingleThreadIoEventLoop.runIo(SingleThreadIoEventLoop.java:225)
> at 
> io.netty.channel.SingleThreadIoEventLoop.run(SingleThreadIoEventLoop.java:196)
> at 
> io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:1193)
> at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
> at 
> io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
> at java.lang.Thread.runWith(java.base@21.0.6/Thread.java:1596)
> at java.lang.Thread.run(java.base@21.0.6/Thread.java:1583)
>  
>    Locked ownable synchronizers:
>  - <0x00000007c2e5c5b8> (a 
> java.util.concurrent.locks.ReentrantLock$NonfairSync)
>  
> "dispatcher-event-loop-0" #34 [35075] daemon prio=5 os_prio=31 cpu=5.67ms 
> elapsed=43.94s tid=0x000000011faf4e00 nid=35075 waiting on condition  
> [0x000000017988a000]
>    java.lang.Thread.State: WAITING (parking)
> at jdk.internal.misc.Unsafe.park(java.base@21.0.6/Native Method)
>  - parking to wait for  <0x00000007c1f84a80> (a 
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
> at 
> java.util.concurrent.locks.LockSupport.park(java.base@21.0.6/LockSupport.java:371)
> at 
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionNode.block(java.base@21.0.6/AbstractQueuedSynchronizer.java:519)
> at 
> java.util.concurrent.ForkJoinPool.unmanagedBlock(java.base@21.0.6/ForkJoinPool.java:3780)
> at 
> java.util.concurrent.ForkJoinPool.managedBlock(java.base@21.0.6/ForkJoinPool.java:3725)
> at 
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(java.base@21.0.6/AbstractQueuedSynchronizer.java:1712)
> at 
> java.util.concurrent.LinkedBlockingQueue.take(java.base@21.0.6/LinkedBlockingQueue.java:435)
> at 
> org.apache.spark.rpc.netty.MessageLoop.org$apache$spark$rpc$netty$MessageLoop$$receiveLoop(MessageLoop.scala:70)
> at org.apache.spark.rpc.netty.MessageLoop$$anon$1.run(MessageLoop.scala:42)
> at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(java.base@21.0.6/ThreadPoolExecutor.java:1144)
> at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(java.base@21.0.6/ThreadPoolExecutor.java:642)
> at java.lang.Thread.runWith(java.base@21.0.6/Thread.java:1596)
> at java.lang.Thread.run(java.base@21.0.6/Thread.java:1583)
>  
>    Locked ownable synchronizers:
>  - <0x00000007c1f84c58> (a java.util.concurrent.ThreadPoolExecutor$Worker)
>  
> "dispatcher-event-loop-1" #35 [40963] daemon prio=5 os_prio=31 cpu=13.28ms 
> elapsed=43.94s tid=0x000000011fae1400 nid=40963 waiting on condition  
> [0x0000000179a96000]
>    java.lang.Thread.State: WAITING (parking)
> at jdk.internal.misc.Unsafe.park(java.base@21.0.6/Native Method)
>  - parking to wait for  <0x00000007c1f84a80> (a 
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
> at 
> java.util.concurrent.locks.LockSupport.park(java.base@21.0.6/LockSupport.java:371)
> at 
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionNode.block(java.base@21.0.6/AbstractQueuedSynchronizer.java:519)
> at 
> java.util.concurrent.ForkJoinPool.unmanagedBlock(java.base@21.0.6/ForkJoinPool.java:3780)
> at 
> java.util.concurrent.ForkJoinPool.managedBlock(java.base@21.0.6/ForkJoinPool.java:3725)
> at 
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(java.base@21.0.6/AbstractQueuedSynchronizer.java:1712)
> at 
> java.util.concurrent.LinkedBlockingQueue.take(java.base@21.0.6/LinkedBlockingQueue.java:435)
> at 
> org.apache.spark.rpc.netty.MessageLoop.org$apache$spark$rpc$netty$MessageLoop$$receiveLoop(MessageLoop.scala:70)
> at org.apache.spark.rpc.netty.MessageLoop$$anon$1.run(MessageLoop.scala:42)
> at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(java.base@21.0.6/ThreadPoolExecutor.java:1144)
> at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(java.base@21.0.6/ThreadPoolExecutor.java:642)
> at java.lang.Thread.runWith(java.base@21.0.6/Thread.java:1596)
> at java.lang.Thread.run(java.base@21.0.6/Thread.java:1583)
>  
>    Locked ownable synchronizers:
>  - <0x00000007c1f84d98> (a java.util.concurrent.ThreadPoolExecutor$Worker)
>  
> "dispatcher-event-loop-2" #36 [35587] daemon prio=5 os_prio=31 cpu=1.62ms 
> elapsed=43.94s tid=0x000000011faf3800 nid=35587 waiting on condition  
> [0x0000000179ca2000]
>    java.lang.Thread.State: WAITING (parking)
> at jdk.internal.misc.Unsafe.park(java.base@21.0.6/Native Method)
>  - parking to wait for  <0x00000007c1f84a80> (a 
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
> at 
> java.util.concurrent.locks.LockSupport.park(java.base@21.0.6/LockSupport.java:371)
> at 
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionNode.block(java.base@21.0.6/AbstractQueuedSynchronizer.java:519)
> at 
> java.util.concurrent.ForkJoinPool.unmanagedBlock(java.base@21.0.6/ForkJoinPool.java:3780)
> at 
> java.util.concurrent.ForkJoinPool.managedBlock(java.base@21.0.6/ForkJoinPool.java:3725)
> at 
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(java.base@21.0.6/AbstractQueuedSynchronizer.java:1712)
> at 
> java.util.concurrent.LinkedBlockingQueue.take(java.base@21.0.6/LinkedBlockingQueue.java:435)
> at 
> org.apache.spark.rpc.netty.MessageLoop.org$apache$spark$rpc$netty$MessageLoop$$receiveLoop(MessageLoop.scala:70)
> at org.apache.spark.rpc.netty.MessageLoop$$anon$1.run(MessageLoop.scala:42)
> at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(java.base@21.0.6/ThreadPoolExecutor.java:1144)
> at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(java.base@21.0.6/ThreadPoolExecutor.java:642)
> at java.lang.Thread.runWith(java.base@21.0.6/Thread.java:1596)
> at java.lang.Thread.run(java.base@21.0.6/Thread.java:1583)
>  
>    Locked ownable synchronizers:
>  - <0x00000007c1f8ed58> (a java.util.concurrent.ThreadPoolExecutor$Worker)
>  
> "dispatcher-event-loop-3" #37 [40195] daemon prio=5 os_prio=31 cpu=3.08ms 
> elapsed=43.94s tid=0x000000011faf0c00 nid=40195 waiting on condition  
> [0x0000000179eae000]
>    java.lang.Thread.State: WAITING (parking)
> at jdk.internal.misc.Unsafe.park(java.base@21.0.6/Native Method)
>  - parking to wait for  <0x00000007c1f84a80> (a 
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
> at 
> java.util.concurrent.locks.LockSupport.park(java.base@21.0.6/LockSupport.java:371)
> at 
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionNode.block(java.base@21.0.6/AbstractQueuedSynchronizer.java:519)
> at 
> java.util.concurrent.ForkJoinPool.unmanagedBlock(java.base@21.0.6/ForkJoinPool.java:3780)
> at 
> java.util.concurrent.ForkJoinPool.managedBlock(java.base@21.0.6/ForkJoinPool.java:3725)
> at 
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(java.base@21.0.6/AbstractQueuedSynchronizer.java:1712)
> at 
> java.util.concurrent.LinkedBlockingQueue.take(java.base@21.0.6/LinkedBlockingQueue.java:435)
> at 
> org.apache.spark.rpc.netty.MessageLoop.org$apache$spark$rpc$netty$MessageLoop$$receiveLoop(MessageLoop.scala:70)
> at org.apache.spark.rpc.netty.MessageLoop$$anon$1.run(MessageLoop.scala:42)
> at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(java.base@21.0.6/ThreadPoolExecutor.java:1144)
> at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(java.base@21.0.6/ThreadPoolExecutor.java:642)
> at java.lang.Thread.runWith(java.base@21.0.6/Thread.java:1596)
> at java.lang.Thread.run(java.base@21.0.6/Thread.java:1583)
>  
>    Locked ownable synchronizers:
>  - <0x00000007c1f8f000> (a java.util.concurrent.ThreadPoolExecutor$Worker)
>  
> "dispatcher-event-loop-4" #38 [35843] daemon prio=5 os_prio=31 cpu=0.26ms 
> elapsed=43.94s tid=0x000000011faf1400 nid=35843 waiting on condition  
> [0x000000017a0ba000]
>    java.lang.Thread.State: WAITING (parking)
> at jdk.internal.misc.Unsafe.park(java.base@21.0.6/Native Method)
>  - parking to wait for  <0x00000007c1f84a80> (a 
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
> at 
> java.util.concurrent.locks.LockSupport.park(java.base@21.0.6/LockSupport.java:371)
> at 
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionNode.block(java.base@21.0.6/AbstractQueuedSynchronizer.java:519)
> at 
> java.util.concurrent.ForkJoinPool.unmanagedBlock(java.base@21.0.6/ForkJoinPool.java:3780)
> at 
> java.util.concurrent.ForkJoinPool.managedBlock(java.base@21.0.6/ForkJoinPool.java:3725)
> at 
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(java.base@21.0.6/AbstractQueuedSynchronizer.java:1712)
> at 
> java.util.concurrent.LinkedBlockingQueue.take(java.base@21.0.6/LinkedBlockingQueue.java:435)
> at 
> org.apache.spark.rpc.netty.MessageLoop.org$apache$spark$rpc$netty$MessageLoop$$receiveLoop(MessageLoop.scala:70)
> at org.apache.spark.rpc.netty.MessageLoop$$anon$1.run(MessageLoop.scala:42)
> at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(java.base@21.0.6/ThreadPoolExecutor.java:1144)
> at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(java.base@21.0.6/ThreadPoolExecutor.java:642)
> at java.lang.Thread.runWith(java.base@21.0.6/Thread.java:1596)
> at java.lang.Thread.run(java.base@21.0.6/Thread.java:1583)
>  
>    Locked ownable synchronizers:
>  - <0x00000007c1f8f1f8> (a java.util.concurrent.ThreadPoolExecutor$Worker)
>  
> "dispatcher-event-loop-5" #39 [39427] daemon prio=5 os_prio=31 cpu=0.33ms 
> elapsed=43.94s tid=0x000000011faf1c00 nid=39427 waiting on condition  
> [0x000000017a2c6000]
>    java.lang.Thread.State: WAITING (parking)
> at jdk.internal.misc.Unsafe.park(java.base@21.0.6/Native Method)
>  - parking to wait for  <0x00000007c1f84a80> (a 
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
> at 
> java.util.concurrent.locks.LockSupport.park(java.base@21.0.6/LockSupport.java:371)
> at 
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionNode.block(java.base@21.0.6/AbstractQueuedSynchronizer.java:519)
> at 
> java.util.concurrent.ForkJoinPool.unmanagedBlock(java.base@21.0.6/ForkJoinPool.java:3780)
> at 
> java.util.concurrent.ForkJoinPool.managedBlock(java.base@21.0.6/ForkJoinPool.java:3725)
> at 
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(java.base@21.0.6/AbstractQueuedSynchronizer.java:1712)
> at 
> java.util.concurrent.LinkedBlockingQueue.take(java.base@21.0.6/LinkedBlockingQueue.java:435)
> at 
> org.apache.spark.rpc.netty.MessageLoop.org$apache$spark$rpc$netty$MessageLoop$$receiveLoop(MessageLoop.scala:70)
> at org.apache.spark.rpc.netty.MessageLoop$$anon$1.run(MessageLoop.scala:42)
> at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(java.base@21.0.6/ThreadPoolExecutor.java:1144)
> at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(java.base@21.0.6/ThreadPoolExecutor.java:642)
> at java.lang.Thread.runWith(java.base@21.0.6/Thread.java:1596)
> at java.lang.Thread.run(java.base@21.0.6/Thread.java:1583)
>  
>    Locked ownable synchronizers:
>  - <0x00000007c1f8f7e0> (a java.util.concurrent.ThreadPoolExecutor$Worker)
>  
> "dispatcher-event-loop-6" #40 [39171] daemon prio=5 os_prio=31 cpu=0.12ms 
> elapsed=43.94s tid=0x000000011fade600 nid=39171 waiting on condition  
> [0x000000017a4d2000]
>    java.lang.Thread.State: WAITING (parking)
> at jdk.internal.misc.Unsafe.park(java.base@21.0.6/Native Method)
>  - parking to wait for  <0x00000007c1f84a80> (a 
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
> at 
> java.util.concurrent.locks.LockSupport.park(java.base@21.0.6/LockSupport.java:371)
> at 
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionNode.block(java.base@21.0.6/AbstractQueuedSynchronizer.java:519)
> at 
> java.util.concurrent.ForkJoinPool.unmanagedBlock(java.base@21.0.6/ForkJoinPool.java:3780)
> at 
> java.util.concurrent.ForkJoinPool.managedBlock(java.base@21.0.6/ForkJoinPool.java:3725)
> at 
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(java.base@21.0.6/AbstractQueuedSynchronizer.java:1712)
> at 
> java.util.concurrent.LinkedBlockingQueue.take(java.base@21.0.6/LinkedBlockingQueue.java:435)
> at 
> org.apache.spark.rpc.netty.MessageLoop.org$apache$spark$rpc$netty$MessageLoop$$receiveLoop(MessageLoop.scala:70)
> at org.apache.spark.rpc.netty.MessageLoop$$anon$1.run(MessageLoop.scala:42)
> at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(java.base@21.0.6/ThreadPoolExecutor.java:1144)
> at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(java.base@21.0.6/ThreadPoolExecutor.java:642)
> at java.lang.Thread.runWith(java.base@21.0.6/Thread.java:1596)
> at java.lang.Thread.run(java.base@21.0.6/Thread.java:1583)
>  
>    Locked ownable synchronizers:
>  - <0x00000007c1f8f3f0> (a java.util.concurrent.ThreadPoolExecutor$Worker)
>  
> "dispatcher-event-loop-7" #41 [38659] daemon prio=5 os_prio=31 cpu=0.14ms 
> elapsed=43.94s tid=0x000000011fadee00 nid=38659 waiting on condition  
> [0x000000017a6de000]
>    java.lang.Thread.State: WAITING (parking)
> at jdk.internal.misc.Unsafe.park(java.base@21.0.6/Native Method)
>  - parking to wait for  <0x00000007c1f84a80> (a 
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
> at 
> java.util.concurrent.locks.LockSupport.park(java.base@21.0.6/LockSupport.java:371)
> at 
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionNode.block(java.base@21.0.6/AbstractQueuedSynchronizer.java:519)
> at 
> java.util.concurrent.ForkJoinPool.unmanagedBlock(java.base@21.0.6/ForkJoinPool.java:3780)
> at 
> java.util.concurrent.ForkJoinPool.managedBlock(java.base@21.0.6/ForkJoinPool.java:3725)
> at 
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(java.base@21.0.6/AbstractQueuedSynchronizer.java:1712)
> at 
> java.util.concurrent.LinkedBlockingQueue.take(java.base@21.0.6/LinkedBlockingQueue.java:435)
> at 
> org.apache.spark.rpc.netty.MessageLoop.org$apache$spark$rpc$netty$MessageLoop$$receiveLoop(MessageLoop.scala:70)
> at org.apache.spark.rpc.netty.MessageLoop$$anon$1.run(MessageLoop.scala:42)
> at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(java.base@21.0.6/ThreadPoolExecutor.java:1144)
> at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(java.base@21.0.6/ThreadPoolExecutor.java:642)
> at java.lang.Thread.runWith(java.base@21.0.6/Thread.java:1596)
> at java.lang.Thread.run(java.base@21.0.6/Thread.java:1583)
>  
>    Locked ownable synchronizers:
>  - <0x00000007c1f8f9d8> (a java.util.concurrent.ThreadPoolExecutor$Worker)
>  
> "dispatcher-event-loop-8" #42 [38147] daemon prio=5 os_prio=31 cpu=0.06ms 
> elapsed=43.94s tid=0x000000011fadf600 nid=38147 waiting on condition  
> [0x000000017a8ea000]
>    java.lang.Thread.State: WAITING (parking)
> at jdk.internal.misc.Unsafe.park(java.base@21.0.6/Native Method)
>  - parking to wait for  <0x00000007c1f84a80> (a 
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
> at 
> java.util.concurrent.locks.LockSupport.park(java.base@21.0.6/LockSupport.java:371)
> at 
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionNode.block(java.base@21.0.6/AbstractQueuedSynchronizer.java:519)
> at 
> java.util.concurrent.ForkJoinPool.unmanagedBlock(java.base@21.0.6/ForkJoinPool.java:3780)
> at 
> java.util.concurrent.ForkJoinPool.managedBlock(java.base@21.0.6/ForkJoinPool.java:3725)
> at 
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(java.base@21.0.6/AbstractQueuedSynchronizer.java:1712)
> at 
> java.util.concurrent.LinkedBlockingQueue.take(java.base@21.0.6/LinkedBlockingQueue.java:435)
> at 
> org.apache.spark.rpc.netty.MessageLoop.org$apache$spark$rpc$netty$MessageLoop$$receiveLoop(MessageLoop.scala:70)
> at org.apache.spark.rpc.netty.MessageLoop$$anon$1.run(MessageLoop.scala:42)
> at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(java.base@21.0.6/ThreadPoolExecutor.java:1144)
> at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(java.base@21.0.6/ThreadPoolExecutor.java:642)
> at java.lang.Thread.runWith(java.base@21.0.6/Thread.java:1596)
> at java.lang.Thread.run(java.base@21.0.6/Thread.java:1583)
>  
>    Locked ownable synchronizers:
>  - <0x00000007c1f8f5e8> (a java.util.concurrent.ThreadPoolExecutor$Worker)
>  
> "dispatcher-event-loop-9" #43 [37891] daemon prio=5 os_prio=31 cpu=0.07ms 
> elapsed=43.94s tid=0x000000011fadfe00 nid=37891 waiting on condition  
> [0x000000017aaf6000]
>    java.lang.Thread.State: WAITING (parking)
> at jdk.internal.misc.Unsafe.park(java.base@21.0.6/Native Method)
>  - parking to wait for  <0x00000007c1f84a80> (a 
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
> at 
> java.util.concurrent.locks.LockSupport.park(java.base@21.0.6/LockSupport.java:371)
> at 
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionNode.block(java.base@21.0.6/AbstractQueuedSynchronizer.java:519)
> at 
> java.util.concurrent.ForkJoinPool.unmanagedBlock(java.base@21.0.6/ForkJoinPool.java:3780)
> at 
> java.util.concurrent.ForkJoinPool.managedBlock(java.base@21.0.6/ForkJoinPool.java:3725)
> at 
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(java.base@21.0.6/AbstractQueuedSynchronizer.java:1712)
> at 
> java.util.concurrent.LinkedBlockingQueue.take(java.base@21.0.6/LinkedBlockingQueue.java:435)
> at 
> org.apache.spark.rpc.netty.MessageLoop.org$apache$spark$rpc$netty$MessageLoop$$receiveLoop(MessageLoop.scala:70)
> at org.apache.spark.rpc.netty.MessageLoop$$anon$1.run(MessageLoop.scala:42)
> at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(java.base@21.0.6/ThreadPoolExecutor.java:1144)
> at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(java.base@21.0.6/ThreadPoolExecutor.java:642)
> at java.lang.Thread.runWith(java.base@21.0.6/Thread.java:1596)
> at java.lang.Thread.run(java.base@21.0.6/Thread.java:1583)
>  
>    Locked ownable synchronizers:
>  - <0x00000007c1f8fbd0> (a java.util.concurrent.ThreadPoolExecutor$Worker)
>  
> "map-output-dispatcher-0" #44 [37379] daemon prio=5 os_prio=31 cpu=0.03ms 
> elapsed=43.94s tid=0x000000010fd39800 nid=37379 waiting on condition  
> [0x000000017ad02000]
>    java.lang.Thread.State: WAITING (parking)
> at jdk.internal.misc.Unsafe.park(java.base@21.0.6/Native Method)
>  - parking to wait for  <0x00000007c1f74ef0> (a 
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
> at 
> java.util.concurrent.locks.LockSupport.park(java.base@21.0.6/LockSupport.java:371)
> at 
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionNode.block(java.base@21.0.6/AbstractQueuedSynchronizer.java:519)
> at 
> java.util.concurrent.ForkJoinPool.unmanagedBlock(java.base@21.0.6/ForkJoinPool.java:3780)
> at 
> java.util.concurrent.ForkJoinPool.managedBlock(java.base@21.0.6/ForkJoinPool.java:3725)
> at 
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(java.base@21.0.6/AbstractQueuedSynchronizer.java:1712)
> at 
> java.util.concurrent.LinkedBlockingQueue.take(java.base@21.0.6/LinkedBlockingQueue.java:435)
> at 
> org.apache.spark.MapOutputTrackerMaster$MessageLoop.run(MapOutputTracker.scala:800)
> at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(java.base@21.0.6/ThreadPoolExecutor.java:1144)
> at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(java.base@21.0.6/ThreadPoolExecutor.java:642)
> at java.lang.Thread.runWith(java.base@21.0.6/Thread.java:1596)
> at java.lang.Thread.run(java.base@21.0.6/Thread.java:1583)
>  
>    Locked ownable synchronizers:
>  - <0x00000007c1f75058> (a java.util.concurrent.ThreadPoolExecutor$Worker)
>  
> "map-output-dispatcher-1" #45 [36867] daemon prio=5 os_prio=31 cpu=0.01ms 
> elapsed=43.94s tid=0x000000010fd3a000 nid=36867 waiting on condition  
> [0x000000017af0e000]
>    java.lang.Thread.State: WAITING (parking)
> at jdk.internal.misc.Unsafe.park(java.base@21.0.6/Native Method)
>  - parking to wait for  <0x00000007c1f74ef0> (a 
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
> at 
> java.util.concurrent.locks.LockSupport.park(java.base@21.0.6/LockSupport.java:371)
> at 
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionNode.block(java.base@21.0.6/AbstractQueuedSynchronizer.java:519)
> at 
> java.util.concurrent.ForkJoinPool.unmanagedBlock(java.base@21.0.6/ForkJoinPool.java:3780)
> at 
> java.util.concurrent.ForkJoinPool.managedBlock(java.base@21.0.6/ForkJoinPool.java:3725)
> at 
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(java.base@21.0.6/AbstractQueuedSynchronizer.java:1712)
> at 
> java.util.concurrent.LinkedBlockingQueue.take(java.base@21.0.6/LinkedBlockingQueue.java:435)
> at 
> org.apache.spark.MapOutputTrackerMaster$MessageLoop.run(MapOutputTracker.scala:800)
> at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(java.base@21.0.6/ThreadPoolExecutor.java:1144)
> at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(java.base@21.0.6/ThreadPoolExecutor.java:642)
> at java.lang.Thread.runWith(java.base@21.0.6/Thread.java:1596)
> at java.lang.Thread.run(java.base@21.0.6/Thread.java:1583)
>  
>    Locked ownable synchronizers:
>  - <0x00000007c1f75de8> (a java.util.concurrent.ThreadPoolExecutor$Worker)
>  
> "map-output-dispatcher-2" #46 [43523] daemon prio=5 os_prio=31 cpu=0.01ms 
> elapsed=43.94s tid=0x000000010fd54e00 nid=43523 waiting on condition  
> [0x000000017b11a000]
>    java.lang.Thread.State: WAITING (parking)
> at jdk.internal.misc.Unsafe.park(java.base@21.0.6/Native Method)
>  - parking to wait for  <0x00000007c1f74ef0> (a 
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
> at 
> java.util.concurrent.locks.LockSupport.park(java.base@21.0.6/LockSupport.java:371)
> at 
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionNode.block(java.base@21.0.6/AbstractQueuedSynchronizer.java:519)
> at 
> java.util.concurrent.ForkJoinPool.unmanagedBlock(java.base@21.0.6/ForkJoinPool.java:3780)
> at 
> java.util.concurrent.ForkJoinPool.managedBlock(java.base@21.0.6/ForkJoinPool.java:3725)
> at 
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(java.base@21.0.6/AbstractQueuedSynchronizer.java:1712)
> at 
> java.util.concurrent.LinkedBlockingQueue.take(java.base@21.0.6/LinkedBlockingQueue.java:435)
> at 
> org.apache.spark.MapOutputTrackerMaster$MessageLoop.run(MapOutputTracker.scala:800)
> at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(java.base@21.0.6/ThreadPoolExecutor.java:1144)
> at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(java.base@21.0.6/ThreadPoolExecutor.java:642)
> at java.lang.Thread.runWith(java.base@21.0.6/Thread.java:1596)
> at java.lang.Thread.run(java.base@21.0.6/Thread.java:1583)
>  
>    Locked ownable synchronizers:
>  - <0x00000007c1f757a0> (a java.util.concurrent.ThreadPoolExecutor$Worker)
>  
> "map-output-dispatcher-3" #47 [65027] daemon prio=5 os_prio=31 cpu=0.01ms 
> elapsed=43.94s tid=0x000000010fd55600 nid=65027 waiting on condition  
> [0x000000017b326000]
>    java.lang.Thread.State: WAITING (parking)
> at jdk.internal.misc.Unsafe.park(java.base@21.0.6/Native Method)
>  - parking to wait for  <0x00000007c1f74ef0> (a 
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
> at 
> java.util.concurrent.locks.LockSupport.park(java.base@21.0.6/LockSupport.java:371)
> at 
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionNode.block(java.base@21.0.6/AbstractQueuedSynchronizer.java:519)
> at 
> java.util.concurrent.ForkJoinPool.unmanagedBlock(java.base@21.0.6/ForkJoinPool.java:3780)
> at 
> java.util.concurrent.ForkJoinPool.managedBlock(java.base@21.0.6/ForkJoinPool.java:3725)
> at 
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(java.base@21.0.6/AbstractQueuedSynchronizer.java:1712)
> at 
> java.util.concurrent.LinkedBlockingQueue.take(java.base@21.0.6/LinkedBlockingQueue.java:435)
> at 
> org.apache.spark.MapOutputTrackerMaster$MessageLoop.run(MapOutputTracker.scala:800)
> at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(java.base@21.0.6/ThreadPoolExecutor.java:1144)
> at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(java.base@21.0.6/ThreadPoolExecutor.java:642)
> at java.lang.Thread.runWith(java.base@21.0.6/Thread.java:1596)
> at java.lang.Thread.run(java.base@21.0.6/Thread.java:1583)
>  
>    Locked ownable synchronizers:
>  - <0x00000007c1f75568> (a java.util.concurrent.ThreadPoolExecutor$Worker)
>  
> "map-output-dispatcher-4" #48 [43779] daemon prio=5 os_prio=31 cpu=0.01ms 
> elapsed=43.94s tid=0x000000010fd55e00 nid=43779 waiting on condition  
> [0x000000017b532000]
>    java.lang.Thread.State: WAITING (parking)
> at jdk.internal.misc.Unsafe.park(java.base@21.0.6/Native Method)
>  - parking to wait for  <0x00000007c1f74ef0> (a 
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
> at 
> java.util.concurrent.locks.LockSupport.park(java.base@21.0.6/LockSupport.java:371)
> at 
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionNode.block(java.base@21.0.6/AbstractQueuedSynchronizer.java:519)
> at 
> java.util.concurrent.ForkJoinPool.unmanagedBlock(java.base@21.0.6/ForkJoinPool.java:3780)
> at 
> java.util.concurrent.ForkJoinPool.managedBlock(java.base@21.0.6/ForkJoinPool.java:3725)
> at 
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(java.base@21.0.6/AbstractQueuedSynchronizer.java:1712)
> at 
> java.util.concurrent.LinkedBlockingQueue.take(java.base@21.0.6/LinkedBlockingQueue.java:435)
> at 
> org.apache.spark.MapOutputTrackerMaster$MessageLoop.run(MapOutputTracker.scala:800)
> at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(java.base@21.0.6/ThreadPoolExecutor.java:1144)
> at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(java.base@21.0.6/ThreadPoolExecutor.java:642)
> at java.lang.Thread.runWith(java.base@21.0.6/Thread.java:1596)
> at java.lang.Thread.run(java.base@21.0.6/Thread.java:1583)
>  
>    Locked ownable synchronizers:
>  - <0x00000007c1f76000> (a java.util.concurrent.ThreadPoolExecutor$Worker)
>  
> "map-output-dispatcher-5" #49 [64259] daemon prio=5 os_prio=31 cpu=0.02ms 
> elapsed=43.94s tid=0x000000010fd56600 nid=64259 waiting on condition  
> [0x000000017b73e000]
>    java.lang.Thread.State: WAITING (parking)
> at jdk.internal.misc.Unsafe.park(java.base@21.0.6/Native Method)
>  - parking to wait for  <0x00000007c1f74ef0> (a 
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
> at 
> java.util.concurrent.locks.LockSupport.park(java.base@21.0.6/LockSupport.java:371)
> at 
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionNode.block(java.base@21.0.6/AbstractQueuedSynchronizer.java:519)
> at 
> java.util.concurrent.ForkJoinPool.unmanagedBlock(java.base@21.0.6/ForkJoinPool.java:3780)
> at 
> java.util.concurrent.ForkJoinPool.managedBlock(java.base@21.0.6/ForkJoinPool.java:3725)
> at 
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(java.base@21.0.6/AbstractQueuedSynchronizer.java:1712)
> at 
> java.util.concurrent.LinkedBlockingQueue.take(java.base@21.0.6/LinkedBlockingQueue.java:435)
> at 
> org.apache.spark.MapOutputTrackerMaster$MessageLoop.run(MapOutputTracker.scala:800)
> at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(java.base@21.0.6/ThreadPoolExecutor.java:1144)
> at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(java.base@21.0.6/ThreadPoolExecutor.java:642)
> at java.lang.Thread.runWith(java.base@21.0.6/Thread.java:1596)
> at java.lang.Thread.run(java.base@21.0.6/Thread.java:1583)
>  
>    Locked ownable synchronizers:
>  - <0x00000007c1f75bd0> (a java.util.concurrent.ThreadPoolExecutor$Worker)
>  
> "map-output-dispatcher-6" #50 [64003] daemon prio=5 os_prio=31 cpu=0.01ms 
> elapsed=43.94s tid=0x000000010fd5a200 nid=64003 waiting on condition  
> [0x000000017b94a000]
>    java.lang.Thread.State: WAITING (parking)
> at jdk.internal.misc.Unsafe.park(java.base@21.0.6/Native Method)
>  - parking to wait for  <0x00000007c1f74ef0> (a 
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
> at 
> java.util.concurrent.locks.LockSupport.park(java.base@21.0.6/LockSupport.java:371)
> at 
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionNode.block(java.base@21.0.6/AbstractQueuedSynchronizer.java:519)
> at 
> java.util.concurrent.ForkJoinPool.unmanagedBlock(java.base@21.0.6/ForkJoinPool.java:3780)
> at 
> java.util.concurrent.ForkJoinPool.managedBlock(java.base@21.0.6/ForkJoinPool.java:3725)
> at 
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(java.base@21.0.6/AbstractQueuedSynchronizer.java:1712)
> at 
> java.util.concurrent.LinkedBlockingQueue.take(java.base@21.0.6/LinkedBlockingQueue.java:435)
> at 
> org.apache.spark.MapOutputTrackerMaster$MessageLoop.run(MapOutputTracker.scala:800)
> at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(java.base@21.0.6/ThreadPoolExecutor.java:1144)
> at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(java.base@21.0.6/ThreadPoolExecutor.java:642)
> at java.lang.Thread.runWith(java.base@21.0.6/Thread.java:1596)
> at java.lang.Thread.run(java.base@21.0.6/Thread.java:1583)
>  
>    Locked ownable synchronizers:
>  - <0x00000007c1f752b0> (a java.util.concurrent.ThreadPoolExecutor$Worker)
>  
> "map-output-dispatcher-7" #51 [63747] daemon prio=5 os_prio=31 cpu=0.01ms 
> elapsed=43.94s tid=0x000000010fd5aa00 nid=63747 waiting on condition  
> [0x000000017bb56000]
>    java.lang.Thread.State: WAITING (parking)
> at jdk.internal.misc.Unsafe.park(java.base@21.0.6/Native Method)
>  - parking to wait for  <0x00000007c1f74ef0> (a 
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
> at 
> java.util.concurrent.locks.LockSupport.park(java.base@21.0.6/LockSupport.java:371)
> at 
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionNode.block(java.base@21.0.6/AbstractQueuedSynchronizer.java:519)
> at 
> java.util.concurrent.ForkJoinPool.unmanagedBlock(java.base@21.0.6/ForkJoinPool.java:3780)
> at 
> java.util.concurrent.ForkJoinPool.managedBlock(java.base@21.0.6/ForkJoinPool.java:3725)
> at 
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(java.base@21.0.6/AbstractQueuedSynchronizer.java:1712)
> at 
> java.util.concurrent.LinkedBlockingQueue.take(java.base@21.0.6/LinkedBlockingQueue.java:435)
> at 
> org.apache.spark.MapOutputTrackerMaster$MessageLoop.run(MapOutputTracker.scala:800)
> at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(java.base@21.0.6/ThreadPoolExecutor.java:1144)
> at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(java.base@21.0.6/ThreadPoolExecutor.java:642)
> at java.lang.Thread.runWith(java.base@21.0.6/Thread.java:1596)
> at java.lang.Thread.run(java.base@21.0.6/Thread.java:1583)
>  
>    Locked ownable synchronizers:
>  - <0x00000007c1f759b8> (a java.util.concurrent.ThreadPoolExecutor$Worker)
>  
> "dispatcher-BlockManagerMaster" #52 [44803] daemon prio=5 os_prio=31 
> cpu=2.75ms elapsed=43.92s tid=0x0000000130706400 nid=44803 waiting on 
> condition  [0x000000017bd62000]
>    java.lang.Thread.State: WAITING (parking)
> at jdk.internal.misc.Unsafe.park(java.base@21.0.6/Native Method)
>  - parking to wait for  <0x00000007c2c000f8> (a 
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
> at 
> java.util.concurrent.locks.LockSupport.park(java.base@21.0.6/LockSupport.java:371)
> at 
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionNode.block(java.base@21.0.6/AbstractQueuedSynchronizer.java:519)
> at 
> java.util.concurrent.ForkJoinPool.unmanagedBlock(java.base@21.0.6/ForkJoinPool.java:3780)
> at 
> java.util.concurrent.ForkJoinPool.managedBlock(java.base@21.0.6/ForkJoinPool.java:3725)
> at 
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(java.base@21.0.6/AbstractQueuedSynchronizer.java:1712)
> at 
> java.util.concurrent.LinkedBlockingQueue.take(java.base@21.0.6/LinkedBlockingQueue.java:435)
> at 
> org.apache.spark.rpc.netty.MessageLoop.org$apache$spark$rpc$netty$MessageLoop$$receiveLoop(MessageLoop.scala:70)
> at org.apache.spark.rpc.netty.MessageLoop$$anon$1.run(MessageLoop.scala:42)
> at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(java.base@21.0.6/ThreadPoolExecutor.java:1144)
> at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(java.base@21.0.6/ThreadPoolExecutor.java:642)
> at java.lang.Thread.runWith(java.base@21.0.6/Thread.java:1596)
> at java.lang.Thread.run(java.base@21.0.6/Thread.java:1583)
>  
>    Locked ownable synchronizers:
>  - <0x00000007c2c001f0> (a java.util.concurrent.ThreadPoolExecutor$Worker)
>  
> "dispatcher-BlockManagerEndpoint1" #53 [63235] daemon prio=5 os_prio=31 
> cpu=0.14ms elapsed=43.91s tid=0x000000010fd6f800 nid=63235 waiting on 
> condition  [0x000000017bf6e000]
>    java.lang.Thread.State: WAITING (parking)
> at jdk.internal.misc.Unsafe.park(java.base@21.0.6/Native Method)
>  - parking to wait for  <0x00000007c2c0ebe8> (a 
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
> at 
> java.util.concurrent.locks.LockSupport.park(java.base@21.0.6/LockSupport.java:371)
> at 
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionNode.block(java.base@21.0.6/AbstractQueuedSynchronizer.java:519)
> at 
> java.util.concurrent.ForkJoinPool.unmanagedBlock(java.base@21.0.6/ForkJoinPool.java:3780)
> at 
> java.util.concurrent.ForkJoinPool.managedBlock(java.base@21.0.6/ForkJoinPool.java:3725)
> at 
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(java.base@21.0.6/AbstractQueuedSynchronizer.java:1712)
> at 
> java.util.concurrent.LinkedBlockingQueue.take(java.base@21.0.6/LinkedBlockingQueue.java:435)
> at 
> org.apache.spark.rpc.netty.MessageLoop.org$apache$spark$rpc$netty$MessageLoop$$receiveLoop(MessageLoop.scala:70)
> at org.apache.spark.rpc.netty.MessageLoop$$anon$1.run(MessageLoop.scala:42)
> at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(java.base@21.0.6/ThreadPoolExecutor.java:1144)
> at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(java.base@21.0.6/ThreadPoolExecutor.java:642)
> at java.lang.Thread.runWith(java.base@21.0.6/Thread.java:1596)
> at java.lang.Thread.run(java.base@21.0.6/Thread.java:1583)
>  
>    Locked ownable synchronizers:
>  - <0x00000007c2c0ece0> (a java.util.concurrent.ThreadPoolExecutor$Worker)
>  
> "RemoteBlock-temp-file-clean-thread" #54 [62979] daemon prio=5 os_prio=31 
> cpu=3.68ms elapsed=43.90s tid=0x000000010fd7e000 nid=62979 waiting on 
> condition  [0x000000017c17a000]
>    java.lang.Thread.State: TIMED_WAITING (parking)
> at jdk.internal.misc.Unsafe.park(java.base@21.0.6/Native Method)
>  - parking to wait for  <0x00000007c335e458> (a 
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
> at 
> java.util.concurrent.locks.LockSupport.parkNanos(java.base@21.0.6/LockSupport.java:269)
> at 
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(java.base@21.0.6/AbstractQueuedSynchronizer.java:1852)
> at java.lang.ref.ReferenceQueue.await(java.base@21.0.6/ReferenceQueue.java:71)
> at 
> java.lang.ref.ReferenceQueue.remove0(java.base@21.0.6/ReferenceQueue.java:143)
> at 
> java.lang.ref.ReferenceQueue.remove(java.base@21.0.6/ReferenceQueue.java:218)
> at 
> org.apache.spark.storage.BlockManager$RemoteBlockDownloadFileManager.org$apache$spark$storage$BlockManager$RemoteBlockDownloadFileManager$$keepCleaning(BlockManager.scala:2297)
> at 
> org.apache.spark.storage.BlockManager$RemoteBlockDownloadFileManager$$anon$2.run(BlockManager.scala:2265)
>  
>    Locked ownable synchronizers:
>  - None
>  
> "dispatcher-HeartbeatReceiver" #56 [45571] daemon prio=5 os_prio=31 
> cpu=2.28ms elapsed=43.89s tid=0x000000010fd7a000 nid=45571 waiting on 
> condition  [0x000000017c386000]
>    java.lang.Thread.State: WAITING (parking)
> at jdk.internal.misc.Unsafe.park(java.base@21.0.6/Native Method)
>  - parking to wait for  <0x00000007c2c0eda8> (a 
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
> at 
> java.util.concurrent.locks.LockSupport.park(java.base@21.0.6/LockSupport.java:371)
> at 
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionNode.block(java.base@21.0.6/AbstractQueuedSynchronizer.java:519)
> at 
> java.util.concurrent.ForkJoinPool.unmanagedBlock(java.base@21.0.6/ForkJoinPool.java:3780)
> at 
> java.util.concurrent.ForkJoinPool.managedBlock(java.base@21.0.6/ForkJoinPool.java:3725)
> at 
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(java.base@21.0.6/AbstractQueuedSynchronizer.java:1712)
> at 
> java.util.concurrent.LinkedBlockingQueue.take(java.base@21.0.6/LinkedBlockingQueue.java:435)
> at 
> org.apache.spark.rpc.netty.MessageLoop.org$apache$spark$rpc$netty$MessageLoop$$receiveLoop(MessageLoop.scala:70)
> at org.apache.spark.rpc.netty.MessageLoop$$anon$1.run(MessageLoop.scala:42)
> at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(java.base@21.0.6/ThreadPoolExecutor.java:1144)
> at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(java.base@21.0.6/ThreadPoolExecutor.java:642)
> at java.lang.Thread.runWith(java.base@21.0.6/Thread.java:1596)
> at java.lang.Thread.run(java.base@21.0.6/Thread.java:1583)
>  
>    Locked ownable synchronizers:
>  - <0x00000007c2c0eea0> (a java.util.concurrent.ThreadPoolExecutor$Worker)
>  
> "heartbeat-receiver-event-loop-thread" #57 [46083] daemon prio=5 os_prio=31 
> cpu=4.55ms elapsed=43.89s tid=0x000000010fd7f600 nid=46083 waiting on 
> condition  [0x000000017c592000]
>    java.lang.Thread.State: TIMED_WAITING (parking)
> at jdk.internal.misc.Unsafe.park(java.base@21.0.6/Native Method)
>  - parking to wait for  <0x00000007c1800120> (a 
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
> at 
> java.util.concurrent.locks.LockSupport.parkNanos(java.base@21.0.6/LockSupport.java:269)
> at 
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(java.base@21.0.6/AbstractQueuedSynchronizer.java:1763)
> at 
> java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(java.base@21.0.6/ScheduledThreadPoolExecutor.java:1182)
> at 
> java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(java.base@21.0.6/ScheduledThreadPoolExecutor.java:899)
> at 
> java.util.concurrent.ThreadPoolExecutor.getTask(java.base@21.0.6/ThreadPoolExecutor.java:1070)
> at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(java.base@21.0.6/ThreadPoolExecutor.java:1130)
> at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(java.base@21.0.6/ThreadPoolExecutor.java:642)
> at java.lang.Thread.runWith(java.base@21.0.6/Thread.java:1596)
> at java.lang.Thread.run(java.base@21.0.6/Thread.java:1583)
>  
>    Locked ownable synchronizers:
>  - None
>  
> "netty-rpc-env-timeout" #58 [46339] daemon prio=5 os_prio=31 cpu=0.39ms 
> elapsed=43.89s tid=0x000000010fd84c00 nid=46339 waiting on condition  
> [0x000000017c79e000]
>    java.lang.Thread.State: TIMED_WAITING (parking)
> at jdk.internal.misc.Unsafe.park(java.base@21.0.6/Native Method)
>  - parking to wait for  <0x00000007c208ce38> (a 
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
> at 
> java.util.concurrent.locks.LockSupport.parkNanos(java.base@21.0.6/LockSupport.java:269)
> at 
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(java.base@21.0.6/AbstractQueuedSynchronizer.java:1763)
> at 
> java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(java.base@21.0.6/ScheduledThreadPoolExecutor.java:1182)
> at 
> java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(java.base@21.0.6/ScheduledThreadPoolExecutor.java:899)
> at 
> java.util.concurrent.ThreadPoolExecutor.getTask(java.base@21.0.6/ThreadPoolExecutor.java:1070)
> at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(java.base@21.0.6/ThreadPoolExecutor.java:1130)
> at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(java.base@21.0.6/ThreadPoolExecutor.java:642)
> at java.lang.Thread.runWith(java.base@21.0.6/Thread.java:1596)
> at java.lang.Thread.run(java.base@21.0.6/Thread.java:1583)
>  
>    Locked ownable synchronizers:
>  - None
>  
> "grpc-nio-boss-ELG-5-1" #59 [46595] daemon prio=5 os_prio=31 cpu=14.97ms 
> elapsed=43.84s tid=0x000000012aba5400 nid=46595 runnable  [0x000000017c9aa000]
>    java.lang.Thread.State: RUNNABLE
> at sun.nio.ch.KQueue.poll(java.base@21.0.6/Native Method)
> at 
> sun.nio.ch.KQueueSelectorImpl.doSelect(java.base@21.0.6/KQueueSelectorImpl.java:125)
> at 
> sun.nio.ch.SelectorImpl.lockAndDoSelect(java.base@21.0.6/SelectorImpl.java:130)
>  - locked <0x00000007c1802ca8> (a 
> io.netty.channel.nio.SelectedSelectionKeySet)
>  - locked <0x00000007c1803cd0> (a sun.nio.ch.KQueueSelectorImpl)
> at sun.nio.ch.SelectorImpl.select(java.base@21.0.6/SelectorImpl.java:142)
> at 
> io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62)
> at io.netty.channel.nio.NioIoHandler.select(NioIoHandler.java:657)
> at io.netty.channel.nio.NioIoHandler.run(NioIoHandler.java:434)
> at 
> io.netty.channel.SingleThreadIoEventLoop.runIo(SingleThreadIoEventLoop.java:225)
> at 
> io.netty.channel.SingleThreadIoEventLoop.run(SingleThreadIoEventLoop.java:196)
> at 
> io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:1193)
> at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
> at 
> io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
> at java.lang.Thread.runWith(java.base@21.0.6/Thread.java:1596)
> at java.lang.Thread.run(java.base@21.0.6/Thread.java:1583)
>  
>    Locked ownable synchronizers:
>  - <0x00000007c1801910> (a 
> java.util.concurrent.locks.ReentrantLock$NonfairSync)
>  
> "dispatcher-org.apache.spark.internal.plugin.PluginEndpoint" #61 [47107] 
> daemon prio=5 os_prio=31 cpu=0.12ms elapsed=43.83s tid=0x000000012abd6a00 
> nid=47107 waiting on condition  [0x000000017cbb6000]
>    java.lang.Thread.State: WAITING (parking)
> at jdk.internal.misc.Unsafe.park(java.base@21.0.6/Native Method)
>  - parking to wait for  <0x00000007c2766b50> (a 
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
> at 
> java.util.concurrent.locks.LockSupport.park(java.base@21.0.6/LockSupport.java:371)
> at 
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionNode.block(java.base@21.0.6/AbstractQueuedSynchronizer.java:519)
> at 
> java.util.concurrent.ForkJoinPool.unmanagedBlock(java.base@21.0.6/ForkJoinPool.java:3780)
> at 
> java.util.concurrent.ForkJoinPool.managedBlock(java.base@21.0.6/ForkJoinPool.java:3725)
> at 
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(java.base@21.0.6/AbstractQueuedSynchronizer.java:1712)
> at 
> java.util.concurrent.LinkedBlockingQueue.take(java.base@21.0.6/LinkedBlockingQueue.java:435)
> at 
> org.apache.spark.rpc.netty.MessageLoop.org$apache$spark$rpc$netty$MessageLoop$$receiveLoop(MessageLoop.scala:70)
> at org.apache.spark.rpc.netty.MessageLoop$$anon$1.run(MessageLoop.scala:42)
> at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(java.base@21.0.6/ThreadPoolExecutor.java:1144)
> at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(java.base@21.0.6/ThreadPoolExecutor.java:642)
> at java.lang.Thread.runWith(java.base@21.0.6/Thread.java:1596)
> at java.lang.Thread.run(java.base@21.0.6/Thread.java:1583)
>  
>    Locked ownable synchronizers:
>  - <0x00000007c2766c48> (a java.util.concurrent.ThreadPoolExecutor$Worker)
>  
> "dag-scheduler-event-loop" #62 [62211] daemon prio=5 os_prio=31 cpu=75.26ms 
> elapsed=43.79s tid=0x000000012abfd200 nid=62211 waiting for monitor entry  
> [0x000000017cdc0000]
>    java.lang.Thread.State: BLOCKED (on object monitor)
> at 
> org.apache.logging.log4j.core.appender.OutputStreamManager.writeBytes(OutputStreamManager.java:365)
>  - waiting to lock <0x00000007c1983c40> (a 
> org.apache.logging.log4j.core.appender.OutputStreamManager)
> at 
> org.apache.logging.log4j.core.layout.TextEncoderHelper.writeEncodedText(TextEncoderHelper.java:101)
> at 
> org.apache.logging.log4j.core.layout.TextEncoderHelper.encodeText(TextEncoderHelper.java:66)
> at 
> org.apache.logging.log4j.core.layout.StringBuilderEncoder.encode(StringBuilderEncoder.java:67)
> at 
> org.apache.logging.log4j.core.layout.StringBuilderEncoder.encode(StringBuilderEncoder.java:31)
> at 
> org.apache.logging.log4j.core.layout.PatternLayout.encode(PatternLayout.java:240)
> at 
> org.apache.logging.log4j.core.layout.PatternLayout.encode(PatternLayout.java:58)
> at 
> org.apache.logging.log4j.core.appender.AbstractOutputStreamAppender.directEncodeEvent(AbstractOutputStreamAppender.java:227)
> at 
> org.apache.logging.log4j.core.appender.AbstractOutputStreamAppender.tryAppend(AbstractOutputStreamAppender.java:220)
> at 
> org.apache.logging.log4j.core.appender.AbstractOutputStreamAppender.append(AbstractOutputStreamAppender.java:211)
> at 
> org.apache.logging.log4j.core.config.AppenderControl.tryCallAppender(AppenderControl.java:160)
> at 
> org.apache.logging.log4j.core.config.AppenderControl.callAppender0(AppenderControl.java:133)
> at 
> org.apache.logging.log4j.core.config.AppenderControl.callAppenderPreventRecursion(AppenderControl.java:124)
> at 
> org.apache.logging.log4j.core.config.AppenderControl.callAppender(AppenderControl.java:88)
> at 
> org.apache.logging.log4j.core.config.LoggerConfig.callAppenders(LoggerConfig.java:714)
> at 
> org.apache.logging.log4j.core.config.LoggerConfig.processLogEvent(LoggerConfig.java:672)
> at 
> org.apache.logging.log4j.core.config.LoggerConfig.log(LoggerConfig.java:648)
> at 
> org.apache.logging.log4j.core.config.LoggerConfig.log(LoggerConfig.java:584)
> at 
> org.apache.logging.log4j.core.config.AwaitCompletionReliabilityStrategy.log(AwaitCompletionReliabilityStrategy.java:92)
> at org.apache.logging.log4j.core.Logger.log(Logger.java:187)
> at 
> org.apache.logging.log4j.spi.AbstractLogger.tryLogMessage(AbstractLogger.java:2904)
> at 
> org.apache.logging.log4j.spi.AbstractLogger.logMessageTrackRecursion(AbstractLogger.java:2857)
> at 
> org.apache.logging.log4j.spi.AbstractLogger.logMessageSafely(AbstractLogger.java:2839)
> at 
> org.apache.logging.log4j.spi.AbstractLogger.logMessage(AbstractLogger.java:2623)
> at 
> org.apache.logging.log4j.spi.AbstractLogger.logIfEnabled(AbstractLogger.java:2371)
> at org.apache.logging.slf4j.Log4jLogger.info(Log4jLogger.java:173)
> at org.apache.spark.internal.Logging.$anonfun$logInfo$1(Logging.scala:187)
> at 
> org.apache.spark.internal.Logging$$Lambda/0x00000070014d7b68.apply$mcV$sp(Unknown
>  Source)
> at org.apache.spark.internal.Logging.withLogContext(Logging.scala:167)
> at org.apache.spark.internal.Logging.withLogContext$(Logging.scala:158)
> at 
> org.apache.spark.scheduler.DAGScheduler.withLogContext(DAGScheduler.scala:124)
> at org.apache.spark.internal.Logging.logInfo(Logging.scala:187)
> at org.apache.spark.internal.Logging.logInfo$(Logging.scala:184)
> at org.apache.spark.scheduler.DAGScheduler.logInfo(DAGScheduler.scala:124)
> at 
> org.apache.spark.scheduler.DAGScheduler.markStageAsFinished(DAGScheduler.scala:2935)
> at 
> org.apache.spark.scheduler.DAGScheduler.processShuffleMapStageCompletion(DAGScheduler.scala:2641)
> at 
> org.apache.spark.scheduler.DAGScheduler.handleTaskCompletion(DAGScheduler.scala:2094)
> at 
> org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.doOnReceive(DAGScheduler.scala:3254)
> at 
> org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.onReceive(DAGScheduler.scala:3194)
> at 
> org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.onReceive(DAGScheduler.scala:3183)
> at org.apache.spark.util.EventLoop$$anon$1.run(EventLoop.scala:50)
>  
>    Locked ownable synchronizers:
>  - None
>  
> "driver-heartbeater" #63 [47875] daemon prio=5 os_prio=31 cpu=3.45ms 
> elapsed=43.76s tid=0x0000000130063600 nid=47875 waiting on condition  
> [0x000000017cfce000]
>    java.lang.Thread.State: TIMED_WAITING (parking)
> at jdk.internal.misc.Unsafe.park(java.base@21.0.6/Native Method)
>  - parking to wait for  <0x00000007c2766dd8> (a 
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
> at 
> java.util.concurrent.locks.LockSupport.parkNanos(java.base@21.0.6/LockSupport.java:269)
> at 
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(java.base@21.0.6/AbstractQueuedSynchronizer.java:1763)
> at 
> java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(java.base@21.0.6/ScheduledThreadPoolExecutor.java:1182)
> at 
> java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(java.base@21.0.6/ScheduledThreadPoolExecutor.java:899)
> at 
> java.util.concurrent.ThreadPoolExecutor.getTask(java.base@21.0.6/ThreadPoolExecutor.java:1070)
> at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(java.base@21.0.6/ThreadPoolExecutor.java:1130)
> at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(java.base@21.0.6/ThreadPoolExecutor.java:642)
> at java.lang.Thread.runWith(java.base@21.0.6/Thread.java:1596)
> at java.lang.Thread.run(java.base@21.0.6/Thread.java:1583)
>  
>    Locked ownable synchronizers:
>  - None
>  
> "executor-kill-mark-cleanup" #64 [61955] daemon prio=5 os_prio=31 cpu=0.44ms 
> elapsed=43.75s tid=0x000000011fb12800 nid=61955 waiting on condition  
> [0x000000017d1da000]
>    java.lang.Thread.State: TIMED_WAITING (parking)
> at jdk.internal.misc.Unsafe.park(java.base@21.0.6/Native Method)
>  - parking to wait for  <0x00000007c276adf8> (a 
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
> at 
> java.util.concurrent.locks.LockSupport.parkNanos(java.base@21.0.6/LockSupport.java:269)
> at 
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(java.base@21.0.6/AbstractQueuedSynchronizer.java:1763)
> at 
> java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(java.base@21.0.6/ScheduledThreadPoolExecutor.java:1182)
> at 
> java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(java.base@21.0.6/ScheduledThreadPoolExecutor.java:899)
> at 
> java.util.concurrent.ThreadPoolExecutor.getTask(java.base@21.0.6/ThreadPoolExecutor.java:1070)
> at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(java.base@21.0.6/ThreadPoolExecutor.java:1130)
> at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(java.base@21.0.6/ThreadPoolExecutor.java:642)
> at java.lang.Thread.runWith(java.base@21.0.6/Thread.java:1596)
> at java.lang.Thread.run(java.base@21.0.6/Thread.java:1583)
>  
>    Locked ownable synchronizers:
>  - None
>  
> "executor-heartbeater" #65 [48643] daemon prio=5 os_prio=31 cpu=16.69ms 
> elapsed=43.75s tid=0x000000011fb11e00 nid=48643 waiting on condition  
> [0x000000017d3e6000]
>    java.lang.Thread.State: TIMED_WAITING (parking)
> at jdk.internal.misc.Unsafe.park(java.base@21.0.6/Native Method)
>  - parking to wait for  <0x00000007c1864390> (a 
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
> at 
> java.util.concurrent.locks.LockSupport.parkNanos(java.base@21.0.6/LockSupport.java:269)
> at 
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(java.base@21.0.6/AbstractQueuedSynchronizer.java:1763)
> at 
> java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(java.base@21.0.6/ScheduledThreadPoolExecutor.java:1182)
> at 
> java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(java.base@21.0.6/ScheduledThreadPoolExecutor.java:899)
> at 
> java.util.concurrent.ThreadPoolExecutor.getTask(java.base@21.0.6/ThreadPoolExecutor.java:1070)
> at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(java.base@21.0.6/ThreadPoolExecutor.java:1130)
> at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(java.base@21.0.6/ThreadPoolExecutor.java:642)
> at java.lang.Thread.runWith(java.base@21.0.6/Thread.java:1596)
> at java.lang.Thread.run(java.base@21.0.6/Thread.java:1583)
>  
>    Locked ownable synchronizers:
>  - None
>  
> "files-client-7-1" #66 [61699] daemon prio=5 os_prio=31 cpu=57.73ms 
> elapsed=43.74s tid=0x000000011fb17400 nid=61699 runnable  [0x000000017d5f2000]
>    java.lang.Thread.State: RUNNABLE
> at io.netty.channel.kqueue.Native.keventWait(Native Method)
> at io.netty.channel.kqueue.Native.keventWait(Native.java:133)
> at 
> io.netty.channel.kqueue.KQueueIoHandler.kqueueWait(KQueueIoHandler.java:187)
> at 
> io.netty.channel.kqueue.KQueueIoHandler.kqueueWait(KQueueIoHandler.java:179)
> at io.netty.channel.kqueue.KQueueIoHandler.run(KQueueIoHandler.java:234)
> at 
> io.netty.channel.SingleThreadIoEventLoop.runIo(SingleThreadIoEventLoop.java:225)
> at 
> io.netty.channel.SingleThreadIoEventLoop.run(SingleThreadIoEventLoop.java:196)
> at 
> io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:1193)
> at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
> at 
> io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
> at java.lang.Thread.runWith(java.base@21.0.6/Thread.java:1596)
> at java.lang.Thread.run(java.base@21.0.6/Thread.java:1583)
>  
>    Locked ownable synchronizers:
>  - <0x00000007c1867560> (a 
> java.util.concurrent.locks.ReentrantLock$NonfairSync)
>  
> "rpc-server-4-1" #67 [49411] daemon prio=5 os_prio=31 cpu=44.33ms 
> elapsed=43.74s tid=0x000000013070b400 nid=49411 runnable  [0x000000017d7fe000]
>    java.lang.Thread.State: RUNNABLE
> at io.netty.channel.kqueue.Native.keventWait(Native Method)
> at io.netty.channel.kqueue.Native.keventWait(Native.java:133)
> at 
> io.netty.channel.kqueue.KQueueIoHandler.kqueueWait(KQueueIoHandler.java:187)
> at 
> io.netty.channel.kqueue.KQueueIoHandler.kqueueWait(KQueueIoHandler.java:179)
> at io.netty.channel.kqueue.KQueueIoHandler.run(KQueueIoHandler.java:234)
> at 
> io.netty.channel.SingleThreadIoEventLoop.runIo(SingleThreadIoEventLoop.java:225)
> at 
> io.netty.channel.SingleThreadIoEventLoop.run(SingleThreadIoEventLoop.java:196)
> at 
> io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:1193)
> at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
> at 
> io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
> at java.lang.Thread.runWith(java.base@21.0.6/Thread.java:1596)
> at java.lang.Thread.run(java.base@21.0.6/Thread.java:1583)
>  
>    Locked ownable synchronizers:
>  - <0x00000007c20f9508> (a 
> java.util.concurrent.locks.ReentrantLock$NonfairSync)
>  
> "process reaper" #68 [49667] daemon prio=10 os_prio=31 cpu=1.59ms 
> elapsed=43.68s tid=0x000000011fb19c00 nid=49667 waiting on condition  
> [0x000000016fec2000]
>    java.lang.Thread.State: TIMED_WAITING (parking)
> at jdk.internal.misc.Unsafe.park(java.base@21.0.6/Native Method)
>  - parking to wait for  <0x00000007c1ee7f88> (a 
> java.util.concurrent.SynchronousQueue$Transferer)
> at 
> java.util.concurrent.locks.LockSupport.parkNanos(java.base@21.0.6/LockSupport.java:410)
> at 
> java.util.concurrent.LinkedTransferQueue$DualNode.await(java.base@21.0.6/LinkedTransferQueue.java:452)
> at 
> java.util.concurrent.SynchronousQueue$Transferer.xferLifo(java.base@21.0.6/SynchronousQueue.java:194)
> at 
> java.util.concurrent.SynchronousQueue.xfer(java.base@21.0.6/SynchronousQueue.java:235)
> at 
> java.util.concurrent.SynchronousQueue.poll(java.base@21.0.6/SynchronousQueue.java:338)
> at 
> java.util.concurrent.ThreadPoolExecutor.getTask(java.base@21.0.6/ThreadPoolExecutor.java:1069)
> at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(java.base@21.0.6/ThreadPoolExecutor.java:1130)
> at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(java.base@21.0.6/ThreadPoolExecutor.java:642)
> at java.lang.Thread.runWith(java.base@21.0.6/Thread.java:1596)
> at java.lang.Thread.run(java.base@21.0.6/Thread.java:1583)
> at 
> jdk.internal.misc.InnocuousThread.run(java.base@21.0.6/InnocuousThread.java:186)
>  
>    Locked ownable synchronizers:
>  - None
>  
> "shuffle-boss-9-1" #71 [61195] daemon prio=5 os_prio=31 cpu=1.73ms 
> elapsed=43.66s tid=0x000000012ac0c400 nid=61195 runnable  [0x000000017da0a000]
>    java.lang.Thread.State: RUNNABLE
> at io.netty.channel.kqueue.Native.keventWait(Native Method)
> at io.netty.channel.kqueue.Native.keventWait(Native.java:133)
> at 
> io.netty.channel.kqueue.KQueueIoHandler.kqueueWait(KQueueIoHandler.java:187)
> at 
> io.netty.channel.kqueue.KQueueIoHandler.kqueueWait(KQueueIoHandler.java:179)
> at io.netty.channel.kqueue.KQueueIoHandler.run(KQueueIoHandler.java:234)
> at 
> io.netty.channel.SingleThreadIoEventLoop.runIo(SingleThreadIoEventLoop.java:225)
> at 
> io.netty.channel.SingleThreadIoEventLoop.run(SingleThreadIoEventLoop.java:196)
> at 
> io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:1193)
> at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
> at 
> io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
> at java.lang.Thread.runWith(java.base@21.0.6/Thread.java:1596)
> at java.lang.Thread.run(java.base@21.0.6/Thread.java:1583)
>  
>    Locked ownable synchronizers:
>  - <0x00000007c18c6490> (a 
> java.util.concurrent.locks.ReentrantLock$NonfairSync)
>  
> "Spark Context Cleaner" #72 [60675] daemon prio=5 os_prio=31 cpu=23.76ms 
> elapsed=43.61s tid=0x000000012ac15a00 nid=60675 waiting on condition  
> [0x000000017dc16000]
>    java.lang.Thread.State: TIMED_WAITING (parking)
> at jdk.internal.misc.Unsafe.park(java.base@21.0.6/Native Method)
>  - parking to wait for  <0x00000007c335c778> (a 
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
> at 
> java.util.concurrent.locks.LockSupport.parkNanos(java.base@21.0.6/LockSupport.java:269)
> at 
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(java.base@21.0.6/AbstractQueuedSynchronizer.java:1852)
> at java.lang.ref.ReferenceQueue.await(java.base@21.0.6/ReferenceQueue.java:71)
> at 
> java.lang.ref.ReferenceQueue.remove0(java.base@21.0.6/ReferenceQueue.java:143)
> at 
> java.lang.ref.ReferenceQueue.remove(java.base@21.0.6/ReferenceQueue.java:218)
> at 
> org.apache.spark.ContextCleaner.$anonfun$keepCleaning$1(ContextCleaner.scala:192)
> at 
> org.apache.spark.ContextCleaner$$Lambda/0x000000700175ea50.apply$mcV$sp(Unknown
>  Source)
> at org.apache.spark.util.Utils$.tryOrStopSparkContext(Utils.scala:1277)
> at 
> org.apache.spark.ContextCleaner.org$apache$spark$ContextCleaner$$keepCleaning(ContextCleaner.scala:190)
> at org.apache.spark.ContextCleaner$$anon$1.run(ContextCleaner.scala:80)
>  
>    Locked ownable synchronizers:
>  - None
>  
> "context-cleaner-periodic-gc" #73 [60163] daemon prio=5 os_prio=31 cpu=0.03ms 
> elapsed=43.61s tid=0x000000013071b400 nid=60163 waiting on condition  
> [0x000000017de22000]
>    java.lang.Thread.State: TIMED_WAITING (parking)
> at jdk.internal.misc.Unsafe.park(java.base@21.0.6/Native Method)
>  - parking to wait for  <0x00000007c2c0ef48> (a 
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
> at 
> java.util.concurrent.locks.LockSupport.parkNanos(java.base@21.0.6/LockSupport.java:269)
> at 
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(java.base@21.0.6/AbstractQueuedSynchronizer.java:1763)
> at 
> java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(java.base@21.0.6/ScheduledThreadPoolExecutor.java:1182)
> at 
> java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(java.base@21.0.6/ScheduledThreadPoolExecutor.java:899)
> at 
> java.util.concurrent.ThreadPoolExecutor.getTask(java.base@21.0.6/ThreadPoolExecutor.java:1070)
> at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(java.base@21.0.6/ThreadPoolExecutor.java:1130)
> at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(java.base@21.0.6/ThreadPoolExecutor.java:642)
> at java.lang.Thread.runWith(java.base@21.0.6/Thread.java:1596)
> at java.lang.Thread.run(java.base@21.0.6/Thread.java:1583)
>  
>    Locked ownable synchronizers:
>  - None
>  
> "spark-listener-group-appStatus" #32 [59651] daemon prio=5 os_prio=31 
> cpu=47.82ms elapsed=43.61s tid=0x000000011fb24400 nid=59651 waiting on 
> condition  [0x000000017e02e000]
>    java.lang.Thread.State: WAITING (parking)
> at jdk.internal.misc.Unsafe.park(java.base@21.0.6/Native Method)
>  - parking to wait for  <0x00000007c1f72e88> (a 
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
> at 
> java.util.concurrent.locks.LockSupport.park(java.base@21.0.6/LockSupport.java:371)
> at 
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionNode.block(java.base@21.0.6/AbstractQueuedSynchronizer.java:519)
> at 
> java.util.concurrent.ForkJoinPool.unmanagedBlock(java.base@21.0.6/ForkJoinPool.java:3780)
> at 
> java.util.concurrent.ForkJoinPool.managedBlock(java.base@21.0.6/ForkJoinPool.java:3725)
> at 
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(java.base@21.0.6/AbstractQueuedSynchronizer.java:1712)
> at 
> java.util.concurrent.LinkedBlockingQueue.take(java.base@21.0.6/LinkedBlockingQueue.java:435)
> at 
> org.apache.spark.scheduler.AsyncEventQueue.$anonfun$dispatch$1(AsyncEventQueue.scala:111)
> at 
> org.apache.spark.scheduler.AsyncEventQueue$$Lambda/0x0000007001760690.apply$mcJ$sp(Unknown
>  Source)
> at scala.runtime.java8.JFunction0$mcJ$sp.apply(JFunction0$mcJ$sp.scala:17)
> at scala.util.DynamicVariable.withValue(DynamicVariable.scala:59)
> at 
> org.apache.spark.scheduler.AsyncEventQueue.org$apache$spark$scheduler$AsyncEventQueue$$dispatch(AsyncEventQueue.scala:101)
> at 
> org.apache.spark.scheduler.AsyncEventQueue$$anon$2.$anonfun$run$1(AsyncEventQueue.scala:97)
> at 
> org.apache.spark.scheduler.AsyncEventQueue$$anon$2$$Lambda/0x00000070017603c0.apply$mcV$sp(Unknown
>  Source)
> at org.apache.spark.util.Utils$.tryOrStopSparkContext(Utils.scala:1277)
> at 
> org.apache.spark.scheduler.AsyncEventQueue$$anon$2.run(AsyncEventQueue.scala:97)
>  
>    Locked ownable synchronizers:
>  - None
>  
> "spark-listener-group-executorManagement" #55 [59139] daemon prio=5 
> os_prio=31 cpu=4.62ms elapsed=43.61s tid=0x000000011fb24c00 nid=59139 waiting 
> on condition  [0x000000017e23a000]
>    java.lang.Thread.State: WAITING (parking)
> at jdk.internal.misc.Unsafe.park(java.base@21.0.6/Native Method)
>  - parking to wait for  <0x00000007c275f708> (a 
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
> at 
> java.util.concurrent.locks.LockSupport.park(java.base@21.0.6/LockSupport.java:371)
> at 
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionNode.block(java.base@21.0.6/AbstractQueuedSynchronizer.java:519)
> at 
> java.util.concurrent.ForkJoinPool.unmanagedBlock(java.base@21.0.6/ForkJoinPool.java:3780)
> at 
> java.util.concurrent.ForkJoinPool.managedBlock(java.base@21.0.6/ForkJoinPool.java:3725)
> at 
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(java.base@21.0.6/AbstractQueuedSynchronizer.java:1712)
> at 
> java.util.concurrent.LinkedBlockingQueue.take(java.base@21.0.6/LinkedBlockingQueue.java:435)
> at 
> org.apache.spark.scheduler.AsyncEventQueue.$anonfun$dispatch$1(AsyncEventQueue.scala:111)
> at 
> org.apache.spark.scheduler.AsyncEventQueue$$Lambda/0x0000007001760690.apply$mcJ$sp(Unknown
>  Source)
> at scala.runtime.java8.JFunction0$mcJ$sp.apply(JFunction0$mcJ$sp.scala:17)
> at scala.util.DynamicVariable.withValue(DynamicVariable.scala:59)
> at 
> org.apache.spark.scheduler.AsyncEventQueue.org$apache$spark$scheduler$AsyncEventQueue$$dispatch(AsyncEventQueue.scala:101)
> at 
> org.apache.spark.scheduler.AsyncEventQueue$$anon$2.$anonfun$run$1(AsyncEventQueue.scala:97)
> at 
> org.apache.spark.scheduler.AsyncEventQueue$$anon$2$$Lambda/0x00000070017603c0.apply$mcV$sp(Unknown
>  Source)
> at org.apache.spark.util.Utils$.tryOrStopSparkContext(Utils.scala:1277)
> at 
> org.apache.spark.scheduler.AsyncEventQueue$$anon$2.run(AsyncEventQueue.scala:97)
>  
>    Locked ownable synchronizers:
>  - None
>  
> "spark-listener-group-shared" #60 [50179] daemon prio=5 os_prio=31 cpu=7.03ms 
> elapsed=43.61s tid=0x000000012ac12800 nid=50179 waiting on condition  
> [0x000000017e446000]
>    java.lang.Thread.State: WAITING (parking)
> at jdk.internal.misc.Unsafe.park(java.base@21.0.6/Native Method)
>  - parking to wait for  <0x00000007c2760698> (a 
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
> at 
> java.util.concurrent.locks.LockSupport.park(java.base@21.0.6/LockSupport.java:371)
> at 
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionNode.block(java.base@21.0.6/AbstractQueuedSynchronizer.java:519)
> at 
> java.util.concurrent.ForkJoinPool.unmanagedBlock(java.base@21.0.6/ForkJoinPool.java:3780)
> at 
> java.util.concurrent.ForkJoinPool.managedBlock(java.base@21.0.6/ForkJoinPool.java:3725)
> at 
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(java.base@21.0.6/AbstractQueuedSynchronizer.java:1712)
> at 
> java.util.concurrent.LinkedBlockingQueue.take(java.base@21.0.6/LinkedBlockingQueue.java:435)
> at 
> org.apache.spark.scheduler.AsyncEventQueue.$anonfun$dispatch$1(AsyncEventQueue.scala:111)
> at 
> org.apache.spark.scheduler.AsyncEventQueue$$Lambda/0x0000007001760690.apply$mcJ$sp(Unknown
>  Source)
> at scala.runtime.java8.JFunction0$mcJ$sp.apply(JFunction0$mcJ$sp.scala:17)
> at scala.util.DynamicVariable.withValue(DynamicVariable.scala:59)
> at 
> org.apache.spark.scheduler.AsyncEventQueue.org$apache$spark$scheduler$AsyncEventQueue$$dispatch(AsyncEventQueue.scala:101)
> at 
> org.apache.spark.scheduler.AsyncEventQueue$$anon$2.$anonfun$run$1(AsyncEventQueue.scala:97)
> at 
> org.apache.spark.scheduler.AsyncEventQueue$$anon$2$$Lambda/0x00000070017603c0.apply$mcV$sp(Unknown
>  Source)
> at org.apache.spark.util.Utils$.tryOrStopSparkContext(Utils.scala:1277)
> at 
> org.apache.spark.scheduler.AsyncEventQueue$$anon$2.run(AsyncEventQueue.scala:97)
>  
>    Locked ownable synchronizers:
>  - None
>  
> "element-tracking-store-worker" #74 [50435] daemon prio=5 os_prio=31 
> cpu=6.03ms elapsed=43.60s tid=0x000000011fb48400 nid=50435 waiting on 
> condition  [0x000000017e652000]
>    java.lang.Thread.State: WAITING (parking)
> at jdk.internal.misc.Unsafe.park(java.base@21.0.6/Native Method)
>  - parking to wait for  <0x00000007c1f73cf8> (a 
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
> at 
> java.util.concurrent.locks.LockSupport.park(java.base@21.0.6/LockSupport.java:371)
> at 
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionNode.block(java.base@21.0.6/AbstractQueuedSynchronizer.java:519)
> at 
> java.util.concurrent.ForkJoinPool.unmanagedBlock(java.base@21.0.6/ForkJoinPool.java:3780)
> at 
> java.util.concurrent.ForkJoinPool.managedBlock(java.base@21.0.6/ForkJoinPool.java:3725)
> at 
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(java.base@21.0.6/AbstractQueuedSynchronizer.java:1712)
> at 
> java.util.concurrent.LinkedBlockingQueue.take(java.base@21.0.6/LinkedBlockingQueue.java:435)
> at 
> java.util.concurrent.ThreadPoolExecutor.getTask(java.base@21.0.6/ThreadPoolExecutor.java:1070)
> at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(java.base@21.0.6/ThreadPoolExecutor.java:1130)
> at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(java.base@21.0.6/ThreadPoolExecutor.java:642)
> at java.lang.Thread.runWith(java.base@21.0.6/Thread.java:1596)
> at java.lang.Thread.run(java.base@21.0.6/Thread.java:1583)
>  
>    Locked ownable synchronizers:
>  - None
>  
> "grpc-nio-worker-ELG-6-1" #75 [50691] daemon prio=5 os_prio=31 cpu=107.47ms 
> elapsed=42.48s tid=0x000000012a01e000 nid=50691 runnable  [0x000000017e85e000]
>    java.lang.Thread.State: RUNNABLE
> at sun.nio.ch.KQueue.poll(java.base@21.0.6/Native Method)
> at 
> sun.nio.ch.KQueueSelectorImpl.doSelect(java.base@21.0.6/KQueueSelectorImpl.java:125)
> at 
> sun.nio.ch.SelectorImpl.lockAndDoSelect(java.base@21.0.6/SelectorImpl.java:130)
>  - locked <0x00000007c1826910> (a 
> io.netty.channel.nio.SelectedSelectionKeySet)
>  - locked <0x00000007c1827938> (a sun.nio.ch.KQueueSelectorImpl)
> at sun.nio.ch.SelectorImpl.select(java.base@21.0.6/SelectorImpl.java:142)
> at 
> io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62)
> at io.netty.channel.nio.NioIoHandler.select(NioIoHandler.java:657)
> at io.netty.channel.nio.NioIoHandler.run(NioIoHandler.java:434)
> at 
> io.netty.channel.SingleThreadIoEventLoop.runIo(SingleThreadIoEventLoop.java:225)
> at 
> io.netty.channel.SingleThreadIoEventLoop.run(SingleThreadIoEventLoop.java:196)
> at 
> io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:1193)
> at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
> at 
> io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
> at java.lang.Thread.runWith(java.base@21.0.6/Thread.java:1596)
> at java.lang.Thread.run(java.base@21.0.6/Thread.java:1583)
>  
>    Locked ownable synchronizers:
>  - <0x00000007c1825588> (a 
> java.util.concurrent.locks.ReentrantLock$NonfairSync)
>  
> "grpc-default-executor-0" #76 [50947] daemon prio=5 os_prio=31 cpu=223.83ms 
> elapsed=40.26s tid=0x000000012b89c200 nid=50947 waiting on condition  
> [0x000000017ea6a000]
>    java.lang.Thread.State: TIMED_WAITING (parking)
> at jdk.internal.misc.Unsafe.park(java.base@21.0.6/Native Method)
>  - parking to wait for  <0x00000007c1828380> (a 
> java.util.concurrent.SynchronousQueue$Transferer)
> at 
> java.util.concurrent.locks.LockSupport.parkNanos(java.base@21.0.6/LockSupport.java:410)
> at 
> java.util.concurrent.LinkedTransferQueue$DualNode.await(java.base@21.0.6/LinkedTransferQueue.java:452)
> at 
> java.util.concurrent.SynchronousQueue$Transferer.xferLifo(java.base@21.0.6/SynchronousQueue.java:194)
> at 
> java.util.concurrent.SynchronousQueue.xfer(java.base@21.0.6/SynchronousQueue.java:235)
> at 
> java.util.concurrent.SynchronousQueue.poll(java.base@21.0.6/SynchronousQueue.java:338)
> at 
> java.util.concurrent.ThreadPoolExecutor.getTask(java.base@21.0.6/ThreadPoolExecutor.java:1069)
> at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(java.base@21.0.6/ThreadPoolExecutor.java:1130)
> at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(java.base@21.0.6/ThreadPoolExecutor.java:642)
> at java.lang.Thread.runWith(java.base@21.0.6/Thread.java:1596)
> at java.lang.Thread.run(java.base@21.0.6/Thread.java:1583)
>  
>    Locked ownable synchronizers:
>  - None
>  
> "pool-30-thread-1" #77 [57603] prio=5 os_prio=31 cpu=2.49ms elapsed=40.24s 
> tid=0x000000011fc61c00 nid=57603 waiting on condition  [0x000000017ec76000]
>    java.lang.Thread.State: TIMED_WAITING (parking)
> at jdk.internal.misc.Unsafe.park(java.base@21.0.6/Native Method)
>  - parking to wait for  <0x00000007c3140f88> (a 
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
> at 
> java.util.concurrent.locks.LockSupport.parkNanos(java.base@21.0.6/LockSupport.java:269)
> at 
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(java.base@21.0.6/AbstractQueuedSynchronizer.java:1763)
> at 
> java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(java.base@21.0.6/ScheduledThreadPoolExecutor.java:1182)
> at 
> java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(java.base@21.0.6/ScheduledThreadPoolExecutor.java:899)
> at 
> java.util.concurrent.ThreadPoolExecutor.getTask(java.base@21.0.6/ThreadPoolExecutor.java:1070)
> at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(java.base@21.0.6/ThreadPoolExecutor.java:1130)
> at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(java.base@21.0.6/ThreadPoolExecutor.java:642)
> at java.lang.Thread.runWith(java.base@21.0.6/Thread.java:1596)
> at java.lang.Thread.run(java.base@21.0.6/Thread.java:1583)
>  
>    Locked ownable synchronizers:
>  - None
>  
> "Cleaner-0" #78 [57347] daemon prio=8 os_prio=31 cpu=0.04ms elapsed=40.14s 
> tid=0x000000010f91be00 nid=57347 waiting on condition  [0x000000017ee82000]
>    java.lang.Thread.State: TIMED_WAITING (parking)
> at jdk.internal.misc.Unsafe.park(java.base@21.0.6/Native Method)
>  - parking to wait for  <0x00000007c3361f28> (a 
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
> at 
> java.util.concurrent.locks.LockSupport.parkNanos(java.base@21.0.6/LockSupport.java:269)
> at 
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(java.base@21.0.6/AbstractQueuedSynchronizer.java:1852)
> at java.lang.ref.ReferenceQueue.await(java.base@21.0.6/ReferenceQueue.java:71)
> at 
> java.lang.ref.ReferenceQueue.remove0(java.base@21.0.6/ReferenceQueue.java:143)
> at 
> java.lang.ref.ReferenceQueue.remove(java.base@21.0.6/ReferenceQueue.java:218)
> at jdk.internal.ref.CleanerImpl.run(java.base@21.0.6/CleanerImpl.java:140)
> at java.lang.Thread.runWith(java.base@21.0.6/Thread.java:1596)
> at java.lang.Thread.run(java.base@21.0.6/Thread.java:1583)
> at 
> jdk.internal.misc.InnocuousThread.run(java.base@21.0.6/InnocuousThread.java:186)
>  
>    Locked ownable synchronizers:
>  - None
>  
> "grpc-default-executor-1" #79 [23815] daemon prio=5 os_prio=31 cpu=161.58ms 
> elapsed=39.89s tid=0x000000010f91cc00 nid=23815 waiting on condition  
> [0x000000017f08e000]
>    java.lang.Thread.State: TIMED_WAITING (parking)
> at jdk.internal.misc.Unsafe.park(java.base@21.0.6/Native Method)
>  - parking to wait for  <0x00000007c1828380> (a 
> java.util.concurrent.SynchronousQueue$Transferer)
> at 
> java.util.concurrent.locks.LockSupport.parkNanos(java.base@21.0.6/LockSupport.java:410)
> at 
> java.util.concurrent.LinkedTransferQueue$DualNode.await(java.base@21.0.6/LinkedTransferQueue.java:452)
> at 
> java.util.concurrent.SynchronousQueue$Transferer.xferLifo(java.base@21.0.6/SynchronousQueue.java:194)
> at 
> java.util.concurrent.SynchronousQueue.xfer(java.base@21.0.6/SynchronousQueue.java:235)
> at 
> java.util.concurrent.SynchronousQueue.poll(java.base@21.0.6/SynchronousQueue.java:338)
> at 
> java.util.concurrent.ThreadPoolExecutor.getTask(java.base@21.0.6/ThreadPoolExecutor.java:1069)
> at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(java.base@21.0.6/ThreadPoolExecutor.java:1130)
> at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(java.base@21.0.6/ThreadPoolExecutor.java:642)
> at java.lang.Thread.runWith(java.base@21.0.6/Thread.java:1596)
> at java.lang.Thread.run(java.base@21.0.6/Thread.java:1583)
>  
>    Locked ownable synchronizers:
>  - None
>  
> "pool-31-thread-1" #81 [56835] prio=5 os_prio=31 cpu=2.46ms elapsed=39.43s 
> tid=0x000000012b968000 nid=56835 waiting on condition  [0x000000017f29a000]
>    java.lang.Thread.State: TIMED_WAITING (parking)
> at jdk.internal.misc.Unsafe.park(java.base@21.0.6/Native Method)
>  - parking to wait for  <0x00000007c3365838> (a 
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
> at 
> java.util.concurrent.locks.LockSupport.parkNanos(java.base@21.0.6/LockSupport.java:269)
> at 
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(java.base@21.0.6/AbstractQueuedSynchronizer.java:1763)
> at 
> java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(java.base@21.0.6/ScheduledThreadPoolExecutor.java:1182)
> at 
> java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(java.base@21.0.6/ScheduledThreadPoolExecutor.java:899)
> at 
> java.util.concurrent.ThreadPoolExecutor.getTask(java.base@21.0.6/ThreadPoolExecutor.java:1070)
> at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(java.base@21.0.6/ThreadPoolExecutor.java:1130)
> at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(java.base@21.0.6/ThreadPoolExecutor.java:642)
> at java.lang.Thread.runWith(java.base@21.0.6/Thread.java:1596)
> at java.lang.Thread.run(java.base@21.0.6/Thread.java:1583)
>  
>    Locked ownable synchronizers:
>  - None
>  
> "Executor task launch worker for task 0.0 in stage 0.0 (TID 0)" #85 [51971] 
> daemon prio=5 os_prio=31 cpu=33.13ms elapsed=38.52s tid=0x000000010f946800 
> nid=51971 waiting on condition  [0x000000017f8be000]
>    java.lang.Thread.State: TIMED_WAITING (parking)
> at jdk.internal.misc.Unsafe.park(java.base@21.0.6/Native Method)
>  - parking to wait for  <0x00000007c27673c0> (a 
> java.util.concurrent.SynchronousQueue$Transferer)
> at 
> java.util.concurrent.locks.LockSupport.parkNanos(java.base@21.0.6/LockSupport.java:410)
> at 
> java.util.concurrent.LinkedTransferQueue$DualNode.await(java.base@21.0.6/LinkedTransferQueue.java:452)
> at 
> java.util.concurrent.SynchronousQueue$Transferer.xferLifo(java.base@21.0.6/SynchronousQueue.java:194)
> at 
> java.util.concurrent.SynchronousQueue.xfer(java.base@21.0.6/SynchronousQueue.java:235)
> at 
> java.util.concurrent.SynchronousQueue.poll(java.base@21.0.6/SynchronousQueue.java:338)
> at 
> java.util.concurrent.ThreadPoolExecutor.getTask(java.base@21.0.6/ThreadPoolExecutor.java:1069)
> at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(java.base@21.0.6/ThreadPoolExecutor.java:1130)
> at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(java.base@21.0.6/ThreadPoolExecutor.java:642)
> at java.lang.Thread.runWith(java.base@21.0.6/Thread.java:1596)
> at java.lang.Thread.run(java.base@21.0.6/Thread.java:1583)
>  
>    Locked ownable synchronizers:
>  - None
>  
> "Executor task launch worker for task 1.0 in stage 0.0 (TID 1)" #86 [55299] 
> daemon prio=5 os_prio=31 cpu=34.34ms elapsed=38.52s tid=0x000000010f947000 
> nid=55299 waiting on condition  [0x000000017faca000]
>    java.lang.Thread.State: TIMED_WAITING (parking)
> at jdk.internal.misc.Unsafe.park(java.base@21.0.6/Native Method)
>  - parking to wait for  <0x00000007c27673c0> (a 
> java.util.concurrent.SynchronousQueue$Transferer)
> at 
> java.util.concurrent.locks.LockSupport.parkNanos(java.base@21.0.6/LockSupport.java:410)
> at 
> java.util.concurrent.LinkedTransferQueue$DualNode.await(java.base@21.0.6/LinkedTransferQueue.java:452)
> at 
> java.util.concurrent.SynchronousQueue$Transferer.xferLifo(java.base@21.0.6/SynchronousQueue.java:194)
> at 
> java.util.concurrent.SynchronousQueue.xfer(java.base@21.0.6/SynchronousQueue.java:235)
> at 
> java.util.concurrent.SynchronousQueue.poll(java.base@21.0.6/SynchronousQueue.java:338)
> at 
> java.util.concurrent.ThreadPoolExecutor.getTask(java.base@21.0.6/ThreadPoolExecutor.java:1069)
> at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(java.base@21.0.6/ThreadPoolExecutor.java:1130)
> at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(java.base@21.0.6/ThreadPoolExecutor.java:642)
> at java.lang.Thread.runWith(java.base@21.0.6/Thread.java:1596)
> at java.lang.Thread.run(java.base@21.0.6/Thread.java:1583)
>  
>    Locked ownable synchronizers:
>  - None
>  
> "Executor task launch worker for task 2.0 in stage 0.0 (TID 2)" #87 [54787] 
> daemon prio=5 os_prio=31 cpu=44.52ms elapsed=38.52s tid=0x000000010f94c400 
> nid=54787 waiting on condition  [0x000000017fcd6000]
>    java.lang.Thread.State: TIMED_WAITING (parking)
> at jdk.internal.misc.Unsafe.park(java.base@21.0.6/Native Method)
>  - parking to wait for  <0x00000007c27673c0> (a 
> java.util.concurrent.SynchronousQueue$Transferer)
> at 
> java.util.concurrent.locks.LockSupport.parkNanos(java.base@21.0.6/LockSupport.java:410)
> at 
> java.util.concurrent.LinkedTransferQueue$DualNode.await(java.base@21.0.6/LinkedTransferQueue.java:452)
> at 
> java.util.concurrent.SynchronousQueue$Transferer.xferLifo(java.base@21.0.6/SynchronousQueue.java:194)
> at 
> java.util.concurrent.SynchronousQueue.xfer(java.base@21.0.6/SynchronousQueue.java:235)
> at 
> java.util.concurrent.SynchronousQueue.poll(java.base@21.0.6/SynchronousQueue.java:338)
> at 
> java.util.concurrent.ThreadPoolExecutor.getTask(java.base@21.0.6/ThreadPoolExecutor.java:1069)
> at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(java.base@21.0.6/ThreadPoolExecutor.java:1130)
> at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(java.base@21.0.6/ThreadPoolExecutor.java:642)
> at java.lang.Thread.runWith(java.base@21.0.6/Thread.java:1596)
> at java.lang.Thread.run(java.base@21.0.6/Thread.java:1583)
>  
>    Locked ownable synchronizers:
>  - None
>  
> "Executor task launch worker for task 3.0 in stage 0.0 (TID 3)" #88 [54531] 
> daemon prio=5 os_prio=31 cpu=43.28ms elapsed=38.52s tid=0x000000010f94cc00 
> nid=54531 waiting on condition  [0x000000017fee2000]
>    java.lang.Thread.State: TIMED_WAITING (parking)
> at jdk.internal.misc.Unsafe.park(java.base@21.0.6/Native Method)
>  - parking to wait for  <0x00000007c27673c0> (a 
> java.util.concurrent.SynchronousQueue$Transferer)
> at 
> java.util.concurrent.locks.LockSupport.parkNanos(java.base@21.0.6/LockSupport.java:410)
> at 
> java.util.concurrent.LinkedTransferQueue$DualNode.await(java.base@21.0.6/LinkedTransferQueue.java:452)
> at 
> java.util.concurrent.SynchronousQueue$Transferer.xferLifo(java.base@21.0.6/SynchronousQueue.java:194)
> at 
> java.util.concurrent.SynchronousQueue.xfer(java.base@21.0.6/SynchronousQueue.java:235)
> at 
> java.util.concurrent.SynchronousQueue.poll(java.base@21.0.6/SynchronousQueue.java:338)
> at 
> java.util.concurrent.ThreadPoolExecutor.getTask(java.base@21.0.6/ThreadPoolExecutor.java:1069)
> at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(java.base@21.0.6/ThreadPoolExecutor.java:1130)
> at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(java.base@21.0.6/ThreadPoolExecutor.java:642)
> at java.lang.Thread.runWith(java.base@21.0.6/Thread.java:1596)
> at java.lang.Thread.run(java.base@21.0.6/Thread.java:1583)
>  
>    Locked ownable synchronizers:
>  - None
>  
> "Executor task launch worker for task 4.0 in stage 0.0 (TID 4)" #89 [52739] 
> daemon prio=5 os_prio=31 cpu=63.84ms elapsed=38.52s tid=0x000000010fc79a00 
> nid=52739 waiting on condition  [0x000000030c206000]
>    java.lang.Thread.State: TIMED_WAITING (parking)
> at jdk.internal.misc.Unsafe.park(java.base@21.0.6/Native Method)
>  - parking to wait for  <0x00000007c27673c0> (a 
> java.util.concurrent.SynchronousQueue$Transferer)
> at 
> java.util.concurrent.locks.LockSupport.parkNanos(java.base@21.0.6/LockSupport.java:410)
> at 
> java.util.concurrent.LinkedTransferQueue$DualNode.await(java.base@21.0.6/LinkedTransferQueue.java:452)
> at 
> java.util.concurrent.SynchronousQueue$Transferer.xferLifo(java.base@21.0.6/SynchronousQueue.java:194)
> at 
> java.util.concurrent.SynchronousQueue.xfer(java.base@21.0.6/SynchronousQueue.java:235)
> at 
> java.util.concurrent.SynchronousQueue.poll(java.base@21.0.6/SynchronousQueue.java:338)
> at 
> java.util.concurrent.ThreadPoolExecutor.getTask(java.base@21.0.6/ThreadPoolExecutor.java:1069)
> at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(java.base@21.0.6/ThreadPoolExecutor.java:1130)
> at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(java.base@21.0.6/ThreadPoolExecutor.java:642)
> at java.lang.Thread.runWith(java.base@21.0.6/Thread.java:1596)
> at java.lang.Thread.run(java.base@21.0.6/Thread.java:1583)
>  
>    Locked ownable synchronizers:
>  - None
>  
> "Executor task launch worker for task 5.0 in stage 0.0 (TID 5)" #90 [54275] 
> daemon prio=5 os_prio=31 cpu=66.52ms elapsed=38.52s tid=0x000000010fc7a200 
> nid=54275 waiting on condition  [0x000000030c412000]
>    java.lang.Thread.State: TIMED_WAITING (parking)
> at jdk.internal.misc.Unsafe.park(java.base@21.0.6/Native Method)
>  - parking to wait for  <0x00000007c27673c0> (a 
> java.util.concurrent.SynchronousQueue$Transferer)
> at 
> java.util.concurrent.locks.LockSupport.parkNanos(java.base@21.0.6/LockSupport.java:410)
> at 
> java.util.concurrent.LinkedTransferQueue$DualNode.await(java.base@21.0.6/LinkedTransferQueue.java:452)
> at 
> java.util.concurrent.SynchronousQueue$Transferer.xferLifo(java.base@21.0.6/SynchronousQueue.java:194)
> at 
> java.util.concurrent.SynchronousQueue.xfer(java.base@21.0.6/SynchronousQueue.java:235)
> at 
> java.util.concurrent.SynchronousQueue.poll(java.base@21.0.6/SynchronousQueue.java:338)
> at 
> java.util.concurrent.ThreadPoolExecutor.getTask(java.base@21.0.6/ThreadPoolExecutor.java:1069)
> at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(java.base@21.0.6/ThreadPoolExecutor.java:1130)
> at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(java.base@21.0.6/ThreadPoolExecutor.java:642)
> at java.lang.Thread.runWith(java.base@21.0.6/Thread.java:1596)
> at java.lang.Thread.run(java.base@21.0.6/Thread.java:1583)
>  
>    Locked ownable synchronizers:
>  - None
>  
> "Executor task launch worker for task 0.0 in stage 1.0 (TID 10)" #91 [53763] 
> daemon prio=5 os_prio=31 cpu=49.09ms elapsed=38.52s tid=0x000000010fc7aa00 
> nid=53763 waiting on condition  [0x000000030c61e000]
>    java.lang.Thread.State: TIMED_WAITING (parking)
> at jdk.internal.misc.Unsafe.park(java.base@21.0.6/Native Method)
>  - parking to wait for  <0x00000007c27673c0> (a 
> java.util.concurrent.SynchronousQueue$Transferer)
> at 
> java.util.concurrent.locks.LockSupport.parkNanos(java.base@21.0.6/LockSupport.java:410)
> at 
> java.util.concurrent.LinkedTransferQueue$DualNode.await(java.base@21.0.6/LinkedTransferQueue.java:452)
> at 
> java.util.concurrent.SynchronousQueue$Transferer.xferLifo(java.base@21.0.6/SynchronousQueue.java:194)
> at 
> java.util.concurrent.SynchronousQueue.xfer(java.base@21.0.6/SynchronousQueue.java:235)
> at 
> java.util.concurrent.SynchronousQueue.poll(java.base@21.0.6/SynchronousQueue.java:338)
> at 
> java.util.concurrent.ThreadPoolExecutor.getTask(java.base@21.0.6/ThreadPoolExecutor.java:1069)
> at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(java.base@21.0.6/ThreadPoolExecutor.java:1130)
> at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(java.base@21.0.6/ThreadPoolExecutor.java:642)
> at java.lang.Thread.runWith(java.base@21.0.6/Thread.java:1596)
> at java.lang.Thread.run(java.base@21.0.6/Thread.java:1583)
>  
>    Locked ownable synchronizers:
>  - None
>  
> "Executor task launch worker for task 2.0 in stage 1.0 (TID 12)" #92 [53507] 
> daemon prio=5 os_prio=31 cpu=42.68ms elapsed=38.52s tid=0x000000010fc7b200 
> nid=53507 waiting on condition  [0x000000030c82a000]
>    java.lang.Thread.State: TIMED_WAITING (parking)
> at jdk.internal.misc.Unsafe.park(java.base@21.0.6/Native Method)
>  - parking to wait for  <0x00000007c27673c0> (a 
> java.util.concurrent.SynchronousQueue$Transferer)
> at 
> java.util.concurrent.locks.LockSupport.parkNanos(java.base@21.0.6/LockSupport.java:410)
> at 
> java.util.concurrent.LinkedTransferQueue$DualNode.await(java.base@21.0.6/LinkedTransferQueue.java:452)
> at 
> java.util.concurrent.SynchronousQueue$Transferer.xferLifo(java.base@21.0.6/SynchronousQueue.java:194)
> at 
> java.util.concurrent.SynchronousQueue.xfer(java.base@21.0.6/SynchronousQueue.java:235)
> at 
> java.util.concurrent.SynchronousQueue.poll(java.base@21.0.6/SynchronousQueue.java:338)
> at 
> java.util.concurrent.ThreadPoolExecutor.getTask(java.base@21.0.6/ThreadPoolExecutor.java:1069)
> at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(java.base@21.0.6/ThreadPoolExecutor.java:1130)
> at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(java.base@21.0.6/ThreadPoolExecutor.java:642)
> at java.lang.Thread.runWith(java.base@21.0.6/Thread.java:1596)
> at java.lang.Thread.run(java.base@21.0.6/Thread.java:1583)
>  
>    Locked ownable synchronizers:
>  - None
>  
> "Executor task launch worker for task 8.0 in stage 0.0 (TID 8)" #93 [65795] 
> daemon prio=5 os_prio=31 cpu=30.79ms elapsed=38.52s tid=0x000000010fc7ba00 
> nid=65795 waiting on condition  [0x000000030ca36000]
>    java.lang.Thread.State: TIMED_WAITING (parking)
> at jdk.internal.misc.Unsafe.park(java.base@21.0.6/Native Method)
>  - parking to wait for  <0x00000007c27673c0> (a 
> java.util.concurrent.SynchronousQueue$Transferer)
> at 
> java.util.concurrent.locks.LockSupport.parkNanos(java.base@21.0.6/LockSupport.java:410)
> at 
> java.util.concurrent.LinkedTransferQueue$DualNode.await(java.base@21.0.6/LinkedTransferQueue.java:452)
> at 
> java.util.concurrent.SynchronousQueue$Transferer.xferLifo(java.base@21.0.6/SynchronousQueue.java:194)
> at 
> java.util.concurrent.SynchronousQueue.xfer(java.base@21.0.6/SynchronousQueue.java:235)
> at 
> java.util.concurrent.SynchronousQueue.poll(java.base@21.0.6/SynchronousQueue.java:338)
> at 
> java.util.concurrent.ThreadPoolExecutor.getTask(java.base@21.0.6/ThreadPoolExecutor.java:1069)
> at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(java.base@21.0.6/ThreadPoolExecutor.java:1130)
> at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(java.base@21.0.6/ThreadPoolExecutor.java:642)
> at java.lang.Thread.runWith(java.base@21.0.6/Thread.java:1596)
> at java.lang.Thread.run(java.base@21.0.6/Thread.java:1583)
>  
>    Locked ownable synchronizers:
>  - None
>  
> "Executor task launch worker for task 1.0 in stage 1.0 (TID 11)" #94 [66307] 
> daemon prio=5 os_prio=31 cpu=56.59ms elapsed=38.52s tid=0x000000010fc7c200 
> nid=66307 waiting on condition  [0x000000030cc42000]
>    java.lang.Thread.State: TIMED_WAITING (parking)
> at jdk.internal.misc.Unsafe.park(java.base@21.0.6/Native Method)
>  - parking to wait for  <0x00000007c27673c0> (a 
> java.util.concurrent.SynchronousQueue$Transferer)
> at 
> java.util.concurrent.locks.LockSupport.parkNanos(java.base@21.0.6/LockSupport.java:410)
> at 
> java.util.concurrent.LinkedTransferQueue$DualNode.await(java.base@21.0.6/LinkedTransferQueue.java:452)
> at 
> java.util.concurrent.SynchronousQueue$Transferer.xferLifo(java.base@21.0.6/SynchronousQueue.java:194)
> at 
> java.util.concurrent.SynchronousQueue.xfer(java.base@21.0.6/SynchronousQueue.java:235)
> at 
> java.util.concurrent.SynchronousQueue.poll(java.base@21.0.6/SynchronousQueue.java:338)
> at 
> java.util.concurrent.ThreadPoolExecutor.getTask(java.base@21.0.6/ThreadPoolExecutor.java:1069)
> at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(java.base@21.0.6/ThreadPoolExecutor.java:1130)
> at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(java.base@21.0.6/ThreadPoolExecutor.java:642)
> at java.lang.Thread.runWith(java.base@21.0.6/Thread.java:1596)
> at java.lang.Thread.run(java.base@21.0.6/Thread.java:1583)
>  
>    Locked ownable synchronizers:
>  - None
>  
> "task-result-getter-0" #121 [66863] daemon prio=5 os_prio=31 cpu=3.28ms 
> elapsed=38.30s tid=0x000000010e858000 nid=66863 waiting on condition  
> [0x000000030ce4e000]
>    java.lang.Thread.State: WAITING (parking)
> at jdk.internal.misc.Unsafe.park(java.base@21.0.6/Native Method)
>  - parking to wait for  <0x00000007c2765b20> (a 
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
> at 
> java.util.concurrent.locks.LockSupport.park(java.base@21.0.6/LockSupport.java:371)
> at 
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionNode.block(java.base@21.0.6/AbstractQueuedSynchronizer.java:519)
> at 
> java.util.concurrent.ForkJoinPool.unmanagedBlock(java.base@21.0.6/ForkJoinPool.java:3780)
> at 
> java.util.concurrent.ForkJoinPool.managedBlock(java.base@21.0.6/ForkJoinPool.java:3725)
> at 
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(java.base@21.0.6/AbstractQueuedSynchronizer.java:1712)
> at 
> java.util.concurrent.LinkedBlockingQueue.take(java.base@21.0.6/LinkedBlockingQueue.java:435)
> at 
> java.util.concurrent.ThreadPoolExecutor.getTask(java.base@21.0.6/ThreadPoolExecutor.java:1070)
> at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(java.base@21.0.6/ThreadPoolExecutor.java:1130)
> at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(java.base@21.0.6/ThreadPoolExecutor.java:642)
> at java.lang.Thread.runWith(java.base@21.0.6/Thread.java:1596)
> at java.lang.Thread.run(java.base@21.0.6/Thread.java:1583)
>  
>    Locked ownable synchronizers:
>  - None
>  
> "task-result-getter-1" #122 [87043] daemon prio=5 os_prio=31 cpu=2.88ms 
> elapsed=38.30s tid=0x00000001305aa400 nid=87043 runnable  [0x000000030d058000]
>    java.lang.Thread.State: RUNNABLE
> at java.io.FileOutputStream.writeBytes(java.base@21.0.6/Native Method)
> at java.io.FileOutputStream.write(java.base@21.0.6/FileOutputStream.java:367)
> at 
> java.io.BufferedOutputStream.flushBuffer(java.base@21.0.6/BufferedOutputStream.java:125)
> at 
> java.io.BufferedOutputStream.implFlush(java.base@21.0.6/BufferedOutputStream.java:252)
> at 
> java.io.BufferedOutputStream.flush(java.base@21.0.6/BufferedOutputStream.java:240)
> at java.io.PrintStream.implWrite(java.base@21.0.6/PrintStream.java:645)
> at java.io.PrintStream.write(java.base@21.0.6/PrintStream.java:623)
> at 
> org.apache.logging.log4j.core.util.CloseShieldOutputStream.write(CloseShieldOutputStream.java:53)
> at 
> org.apache.logging.log4j.core.appender.OutputStreamManager.writeToDestination(OutputStreamManager.java:263)
>  - locked <0x00000007c1983c40> (a 
> org.apache.logging.log4j.core.appender.OutputStreamManager)
> at 
> org.apache.logging.log4j.core.appender.OutputStreamManager.flushBuffer(OutputStreamManager.java:296)
>  - locked <0x00000007c1983c40> (a 
> org.apache.logging.log4j.core.appender.OutputStreamManager)
> at 
> org.apache.logging.log4j.core.appender.OutputStreamManager.flush(OutputStreamManager.java:307)
>  - locked <{color:#de350b}*0x00000007c1983c40*{color}> (a 
> org.apache.logging.log4j.core.appender.OutputStreamManager)
> at 
> org.apache.logging.log4j.core.appender.AbstractOutputStreamAppender.directEncodeEvent(AbstractOutputStreamAppender.java:229)
> at 
> org.apache.logging.log4j.core.appender.AbstractOutputStreamAppender.tryAppend(AbstractOutputStreamAppender.java:220)
> at 
> org.apache.logging.log4j.core.appender.AbstractOutputStreamAppender.append(AbstractOutputStreamAppender.java:211)
> at 
> org.apache.logging.log4j.core.config.AppenderControl.tryCallAppender(AppenderControl.java:160)
> at 
> org.apache.logging.log4j.core.config.AppenderControl.callAppender0(AppenderControl.java:133)
> at 
> org.apache.logging.log4j.core.config.AppenderControl.callAppenderPreventRecursion(AppenderControl.java:124)
> at 
> org.apache.logging.log4j.core.config.AppenderControl.callAppender(AppenderControl.java:88)
> at 
> org.apache.logging.log4j.core.config.LoggerConfig.callAppenders(LoggerConfig.java:714)
> at 
> org.apache.logging.log4j.core.config.LoggerConfig.processLogEvent(LoggerConfig.java:672)
> at 
> org.apache.logging.log4j.core.config.LoggerConfig.log(LoggerConfig.java:648)
> at 
> org.apache.logging.log4j.core.config.LoggerConfig.log(LoggerConfig.java:584)
> at 
> org.apache.logging.log4j.core.config.AwaitCompletionReliabilityStrategy.log(AwaitCompletionReliabilityStrategy.java:92)
> at org.apache.logging.log4j.core.Logger.log(Logger.java:187)
> at 
> org.apache.logging.log4j.spi.AbstractLogger.tryLogMessage(AbstractLogger.java:2904)
> at 
> org.apache.logging.log4j.spi.AbstractLogger.logMessageTrackRecursion(AbstractLogger.java:2857)
> at 
> org.apache.logging.log4j.spi.AbstractLogger.logMessageSafely(AbstractLogger.java:2839)
> at 
> org.apache.logging.log4j.spi.AbstractLogger.logMessage(AbstractLogger.java:2623)
> at 
> org.apache.logging.log4j.spi.AbstractLogger.logIfEnabled(AbstractLogger.java:2371)
> at org.apache.logging.slf4j.Log4jLogger.info(Log4jLogger.java:173)
> at org.apache.spark.internal.Logging.$anonfun$logInfo$1(Logging.scala:187)
> at 
> org.apache.spark.internal.Logging$$Lambda/0x00000070014d7b68.apply$mcV$sp(Unknown
>  Source)
> at org.apache.spark.internal.Logging.withLogContext(Logging.scala:167)
> at org.apache.spark.internal.Logging.withLogContext$(Logging.scala:158)
> at 
> org.apache.spark.scheduler.TaskSchedulerImpl.withLogContext(TaskSchedulerImpl.scala:83)
> at org.apache.spark.internal.Logging.logInfo(Logging.scala:187)
> at org.apache.spark.internal.Logging.logInfo$(Logging.scala:184)
> at 
> org.apache.spark.scheduler.TaskSchedulerImpl.logInfo(TaskSchedulerImpl.scala:83)
> at 
> org.apache.spark.scheduler.TaskSchedulerImpl.taskSetFinished(TaskSchedulerImpl.scala:359)
>  - locked <0x00000007c2cce260> (a 
> org.apache.spark.scheduler.TaskSchedulerImpl)
> at 
> org.apache.spark.scheduler.TaskSetManager.maybeFinishTaskSet(TaskSetManager.scala:601)
> at 
> org.apache.spark.scheduler.TaskSetManager.handleSuccessfulTask(TaskSetManager.scala:885)
> at 
> org.apache.spark.scheduler.TaskSchedulerImpl.handleSuccessfulTask(TaskSchedulerImpl.scala:873)
>  - locked <0x00000007c2cce260> (a 
> org.apache.spark.scheduler.TaskSchedulerImpl)
> at 
> org.apache.spark.scheduler.TaskResultGetter$$anon$3.$anonfun$run$1(TaskResultGetter.scala:122)
> at 
> org.apache.spark.scheduler.TaskResultGetter$$anon$3$$Lambda/0x00000070021b9998.apply$mcV$sp(Unknown
>  Source)
> at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.scala:18)
> at org.apache.spark.util.Utils$.logUncaughtExceptions(Utils.scala:1925)
> at 
> org.apache.spark.scheduler.TaskResultGetter$$anon$3.run(TaskResultGetter.scala:64)
> at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(java.base@21.0.6/ThreadPoolExecutor.java:1144)
> at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(java.base@21.0.6/ThreadPoolExecutor.java:642)
> at java.lang.Thread.runWith(java.base@21.0.6/Thread.java:1596)
> at java.lang.Thread.run(java.base@21.0.6/Thread.java:1583)
>  
>    Locked ownable synchronizers:
>  - <0x00000007c1682d88> (a 
> java.util.concurrent.locks.ReentrantLock$NonfairSync)
>  - <0x00000007c1682e68> (a 
> java.util.concurrent.locks.ReentrantLock$NonfairSync)
>  - <0x00000007c444cf38> (a java.util.concurrent.ThreadPoolExecutor$Worker)
>  
> "task-result-getter-2" #123 [67331] daemon prio=5 os_prio=31 cpu=2.27ms 
> elapsed=38.30s tid=0x000000010e857600 nid=67331 waiting on condition  
> [0x000000030d266000]
>    java.lang.Thread.State: WAITING (parking)
> at jdk.internal.misc.Unsafe.park(java.base@21.0.6/Native Method)
>  - parking to wait for  <0x00000007c2765b20> (a 
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
> at 
> java.util.concurrent.locks.LockSupport.park(java.base@21.0.6/LockSupport.java:371)
> at 
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionNode.block(java.base@21.0.6/AbstractQueuedSynchronizer.java:519)
> at 
> java.util.concurrent.ForkJoinPool.unmanagedBlock(java.base@21.0.6/ForkJoinPool.java:3780)
> at 
> java.util.concurrent.ForkJoinPool.managedBlock(java.base@21.0.6/ForkJoinPool.java:3725)
> at 
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(java.base@21.0.6/AbstractQueuedSynchronizer.java:1712)
> at 
> java.util.concurrent.LinkedBlockingQueue.take(java.base@21.0.6/LinkedBlockingQueue.java:435)
> at 
> java.util.concurrent.ThreadPoolExecutor.getTask(java.base@21.0.6/ThreadPoolExecutor.java:1070)
> at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(java.base@21.0.6/ThreadPoolExecutor.java:1130)
> at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(java.base@21.0.6/ThreadPoolExecutor.java:642)
> at java.lang.Thread.runWith(java.base@21.0.6/Thread.java:1596)
> at java.lang.Thread.run(java.base@21.0.6/Thread.java:1583)
>  
>    Locked ownable synchronizers:
>  - None
>  
> "task-result-getter-3" #124 [86531] daemon prio=5 os_prio=31 cpu=1.48ms 
> elapsed=38.30s tid=0x000000012a01d600 nid=86531 waiting on condition  
> [0x000000030d472000]
>    java.lang.Thread.State: WAITING (parking)
> at jdk.internal.misc.Unsafe.park(java.base@21.0.6/Native Method)
>  - parking to wait for  <0x00000007c2765b20> (a 
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
> at 
> java.util.concurrent.locks.LockSupport.park(java.base@21.0.6/LockSupport.java:371)
> at 
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionNode.block(java.base@21.0.6/AbstractQueuedSynchronizer.java:519)
> at 
> java.util.concurrent.ForkJoinPool.unmanagedBlock(java.base@21.0.6/ForkJoinPool.java:3780)
> at 
> java.util.concurrent.ForkJoinPool.managedBlock(java.base@21.0.6/ForkJoinPool.java:3725)
> at 
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(java.base@21.0.6/AbstractQueuedSynchronizer.java:1712)
> at 
> java.util.concurrent.LinkedBlockingQueue.take(java.base@21.0.6/LinkedBlockingQueue.java:435)
> at 
> java.util.concurrent.ThreadPoolExecutor.getTask(java.base@21.0.6/ThreadPoolExecutor.java:1070)
> at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(java.base@21.0.6/ThreadPoolExecutor.java:1130)
> at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(java.base@21.0.6/ThreadPoolExecutor.java:642)
> at java.lang.Thread.runWith(java.base@21.0.6/Thread.java:1596)
> at java.lang.Thread.run(java.base@21.0.6/Thread.java:1583)
>  
>    Locked ownable synchronizers:
>  - None
>  
> "SparkConnectExecuteThread_opId=58cbe037-7053-44b7-b62e-f41027f1962c" #130 
> [56347] daemon prio=5 os_prio=31 cpu=185.69ms elapsed=38.14s 
> tid=0x000000012bd71c00 nid=56347 waiting on condition  [0x000000017f4a4000]
>    java.lang.Thread.State: WAITING (parking)
> at jdk.internal.misc.Unsafe.park(java.base@21.0.6/Native Method)
>  - parking to wait for  <0x00000007c4460318> (a 
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
> at 
> java.util.concurrent.locks.LockSupport.park(java.base@21.0.6/LockSupport.java:371)
> at 
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionNode.block(java.base@21.0.6/AbstractQueuedSynchronizer.java:519)
> at 
> java.util.concurrent.ForkJoinPool.unmanagedBlock(java.base@21.0.6/ForkJoinPool.java:3780)
> at 
> java.util.concurrent.ForkJoinPool.managedBlock(java.base@21.0.6/ForkJoinPool.java:3725)
> at 
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(java.base@21.0.6/AbstractQueuedSynchronizer.java:1712)
> at 
> java.util.concurrent.LinkedBlockingQueue.take(java.base@21.0.6/LinkedBlockingQueue.java:435)
> at 
> org.apache.spark.sql.execution.adaptive.AdaptiveSparkPlanExec.waitForNextStageToMaterialize(AdaptiveSparkPlanExec.scala:597)
> at 
> org.apache.spark.sql.execution.adaptive.AdaptiveSparkPlanExec.$anonfun$withFinalPlanUpdate$1(AdaptiveSparkPlanExec.scala:410)
> at 
> org.apache.spark.sql.execution.adaptive.AdaptiveSparkPlanExec$$Lambda/0x00000070022addd8.apply$mcV$sp(Unknown
>  Source)
> at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.scala:18)
> at org.apache.spark.sql.SparkSession.withActive(SparkSession.scala:804)
> at 
> org.apache.spark.sql.execution.adaptive.AdaptiveSparkPlanExec.withFinalPlanUpdate(AdaptiveSparkPlanExec.scala:291)
>  - locked <0x00000007c44607a0> (a java.lang.Object)
> at 
> org.apache.spark.sql.execution.adaptive.AdaptiveSparkPlanExec.doExecute(AdaptiveSparkPlanExec.scala:715)
> at 
> org.apache.spark.sql.execution.SparkPlan.$anonfun$executeRDD$1(SparkPlan.scala:189)
> at 
> org.apache.spark.sql.execution.SparkPlan$$Lambda/0x0000007001e8b970.apply(Unknown
>  Source)
> at scala.util.Try$.apply(Try.scala:217)
> at org.apache.spark.util.Utils$.doTryWithCallerStacktrace(Utils.scala:1376)
> at org.apache.spark.util.LazyTry.tryT$lzycompute(LazyTry.scala:46)
>  - locked <0x00000007c44607c0> (a org.apache.spark.util.LazyTry)
> at org.apache.spark.util.LazyTry.tryT(LazyTry.scala:46)
> at org.apache.spark.util.LazyTry.get(LazyTry.scala:58)
> at 
> org.apache.spark.sql.execution.SparkPlan.$anonfun$execute$1(SparkPlan.scala:202)
> at 
> org.apache.spark.sql.execution.SparkPlan$$Lambda/0x0000007001edf6b0.apply(Unknown
>  Source)
> at 
> org.apache.spark.sql.execution.SparkPlan.$anonfun$executeQuery$1(SparkPlan.scala:261)
> at 
> org.apache.spark.sql.execution.SparkPlan$$Lambda/0x0000007001ee6bb8.apply(Unknown
>  Source)
> at 
> org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:151)
> at org.apache.spark.sql.execution.SparkPlan.executeQuery(SparkPlan.scala:258)
> at org.apache.spark.sql.execution.SparkPlan.execute(SparkPlan.scala:198)
> at 
> org.apache.spark.sql.connect.execution.SparkConnectPlanExecution.$anonfun$processAsArrowBatches$3(SparkConnectPlanExecution.scala:212)
> at 
> org.apache.spark.sql.connect.execution.SparkConnectPlanExecution$$Lambda/0x0000007001eb4f90.apply(Unknown
>  Source)
> at 
> org.apache.spark.sql.execution.SQLExecution$.$anonfun$withNewExecutionId0$8(SQLExecution.scala:176)
> at 
> org.apache.spark.sql.execution.SQLExecution$$$Lambda/0x0000007001ebc2b8.apply(Unknown
>  Source)
> at 
> org.apache.spark.sql.execution.SQLExecution$.withSessionTagsApplied(SQLExecution.scala:284)
> at 
> org.apache.spark.sql.execution.SQLExecution$.$anonfun$withNewExecutionId0$7(SQLExecution.scala:138)
> at 
> org.apache.spark.sql.execution.SQLExecution$$$Lambda/0x0000007001ebc000.apply(Unknown
>  Source)
> at 
> org.apache.spark.JobArtifactSet$.withActiveJobArtifactState(JobArtifactSet.scala:94)
> at 
> org.apache.spark.sql.artifact.ArtifactManager.$anonfun$withResources$1(ArtifactManager.scala:112)
> at 
> org.apache.spark.sql.artifact.ArtifactManager$$Lambda/0x0000007001a98778.apply(Unknown
>  Source)
> at org.apache.spark.util.Utils$.withContextClassLoader(Utils.scala:185)
> at 
> org.apache.spark.sql.artifact.ArtifactManager.withClassLoaderIfNeeded(ArtifactManager.scala:102)
> at 
> org.apache.spark.sql.artifact.ArtifactManager.withResources(ArtifactManager.scala:111)
> at 
> org.apache.spark.sql.execution.SQLExecution$.$anonfun$withNewExecutionId0$6(SQLExecution.scala:138)
> at 
> org.apache.spark.sql.execution.SQLExecution$$$Lambda/0x0000007001eb7388.apply(Unknown
>  Source)
> at 
> org.apache.spark.sql.execution.SQLExecution$.withSQLConfPropagated(SQLExecution.scala:307)
> at 
> org.apache.spark.sql.execution.SQLExecution$.$anonfun$withNewExecutionId0$1(SQLExecution.scala:137)
> at 
> org.apache.spark.sql.execution.SQLExecution$$$Lambda/0x0000007001eb5240.apply(Unknown
>  Source)
> at org.apache.spark.sql.SparkSession.withActive(SparkSession.scala:804)
> at 
> org.apache.spark.sql.execution.SQLExecution$.withNewExecutionId0(SQLExecution.scala:91)
> at 
> org.apache.spark.sql.execution.SQLExecution$.withNewExecutionId(SQLExecution.scala:249)
> at 
> org.apache.spark.sql.connect.execution.SparkConnectPlanExecution.processAsArrowBatches(SparkConnectPlanExecution.scala:211)
> at 
> org.apache.spark.sql.connect.execution.SparkConnectPlanExecution.handlePlan(SparkConnectPlanExecution.scala:79)
> at 
> org.apache.spark.sql.connect.execution.ExecuteThreadRunner.$anonfun$executeInternal$1(ExecuteThreadRunner.scala:225)
> at 
> org.apache.spark.sql.connect.execution.ExecuteThreadRunner.$anonfun$executeInternal$1$adapted(ExecuteThreadRunner.scala:197)
> at 
> org.apache.spark.sql.connect.execution.ExecuteThreadRunner$$Lambda/0x0000007001bbf5b8.apply(Unknown
>  Source)
> at 
> org.apache.spark.sql.connect.service.SessionHolder.$anonfun$withSession$2(SessionHolder.scala:396)
> at 
> org.apache.spark.sql.connect.service.SessionHolder$$Lambda/0x0000007001a9a250.apply(Unknown
>  Source)
> at org.apache.spark.sql.SparkSession.withActive(SparkSession.scala:804)
> at 
> org.apache.spark.sql.connect.service.SessionHolder.$anonfun$withSession$1(SessionHolder.scala:396)
> at 
> org.apache.spark.sql.connect.service.SessionHolder$$Lambda/0x0000007001a984c8.apply(Unknown
>  Source)
> at 
> org.apache.spark.JobArtifactSet$.withActiveJobArtifactState(JobArtifactSet.scala:94)
> at 
> org.apache.spark.sql.artifact.ArtifactManager.$anonfun$withResources$1(ArtifactManager.scala:112)
> at 
> org.apache.spark.sql.artifact.ArtifactManager$$Lambda/0x0000007001a98778.apply(Unknown
>  Source)
> at org.apache.spark.util.Utils$.withContextClassLoader(Utils.scala:185)
> at 
> org.apache.spark.sql.artifact.ArtifactManager.withClassLoaderIfNeeded(ArtifactManager.scala:102)
> at 
> org.apache.spark.sql.artifact.ArtifactManager.withResources(ArtifactManager.scala:111)
> at 
> org.apache.spark.sql.connect.service.SessionHolder.withSession(SessionHolder.scala:395)
> at 
> org.apache.spark.sql.connect.execution.ExecuteThreadRunner.executeInternal(ExecuteThreadRunner.scala:197)
> at 
> org.apache.spark.sql.connect.execution.ExecuteThreadRunner.org$apache$spark$sql$connect$execution$ExecuteThreadRunner$$execute(ExecuteThreadRunner.scala:126)
> at 
> org.apache.spark.sql.connect.execution.ExecuteThreadRunner$ExecutionThread.run(ExecuteThreadRunner.scala:334)
>  
>    Locked ownable synchronizers:
>  - None
>  
> "SparkConnectGRPCSender_opId=58cbe037-7053-44b7-b62e-f41027f1962c_startIndex=2"
>  #133 [56087] daemon prio=5 os_prio=31 cpu=1.11ms elapsed=37.98s 
> tid=0x000000012bd1c400 nid=56087 waiting for monitor entry  
> [0x000000017f6b1000]
>    java.lang.Thread.State: BLOCKED (on object monitor)
> at 
> org.apache.logging.log4j.core.appender.OutputStreamManager.writeBytes(OutputStreamManager.java:365)
>  - waiting to lock <{color:#de350b}*0x00000007c1983c40*{color}> (a 
> org.apache.logging.log4j.core.appender.OutputStreamManager)
> at 
> org.apache.logging.log4j.core.layout.TextEncoderHelper.writeEncodedText(TextEncoderHelper.java:101)
> at 
> org.apache.logging.log4j.core.layout.TextEncoderHelper.encodeText(TextEncoderHelper.java:66)
> at 
> org.apache.logging.log4j.core.layout.StringBuilderEncoder.encode(StringBuilderEncoder.java:67)
> at 
> org.apache.logging.log4j.core.layout.StringBuilderEncoder.encode(StringBuilderEncoder.java:31)
> at 
> org.apache.logging.log4j.core.layout.PatternLayout.encode(PatternLayout.java:240)
> at 
> org.apache.logging.log4j.core.layout.PatternLayout.encode(PatternLayout.java:58)
> at 
> org.apache.logging.log4j.core.appender.AbstractOutputStreamAppender.directEncodeEvent(AbstractOutputStreamAppender.java:227)
> at 
> org.apache.logging.log4j.core.appender.AbstractOutputStreamAppender.tryAppend(AbstractOutputStreamAppender.java:220)
> at 
> org.apache.logging.log4j.core.appender.AbstractOutputStreamAppender.append(AbstractOutputStreamAppender.java:211)
> at 
> org.apache.logging.log4j.core.config.AppenderControl.tryCallAppender(AppenderControl.java:160)
> at 
> org.apache.logging.log4j.core.config.AppenderControl.callAppender0(AppenderControl.java:133)
> at 
> org.apache.logging.log4j.core.config.AppenderControl.callAppenderPreventRecursion(AppenderControl.java:124)
> at 
> org.apache.logging.log4j.core.config.AppenderControl.callAppender(AppenderControl.java:88)
> at 
> org.apache.logging.log4j.core.config.LoggerConfig.callAppenders(LoggerConfig.java:714)
> at 
> org.apache.logging.log4j.core.config.LoggerConfig.processLogEvent(LoggerConfig.java:672)
> at 
> org.apache.logging.log4j.core.config.LoggerConfig.log(LoggerConfig.java:648)
> at 
> org.apache.logging.log4j.core.config.LoggerConfig.log(LoggerConfig.java:584)
> at 
> org.apache.logging.log4j.core.config.AwaitCompletionReliabilityStrategy.log(AwaitCompletionReliabilityStrategy.java:92)
> at org.apache.logging.log4j.core.Logger.log(Logger.java:187)
> at 
> org.apache.logging.log4j.spi.AbstractLogger.tryLogMessage(AbstractLogger.java:2904)
> at 
> org.apache.logging.log4j.spi.AbstractLogger.logMessageTrackRecursion(AbstractLogger.java:2857)
> at 
> org.apache.logging.log4j.spi.AbstractLogger.logMessageSafely(AbstractLogger.java:2839)
> at 
> org.apache.logging.log4j.spi.AbstractLogger.logMessage(AbstractLogger.java:2623)
> at 
> org.apache.logging.log4j.spi.AbstractLogger.logIfEnabled(AbstractLogger.java:2371)
> at org.apache.logging.slf4j.Log4jLogger.info(Log4jLogger.java:173)
> at org.apache.spark.internal.Logging.$anonfun$logInfo$1(Logging.scala:187)
> at 
> org.apache.spark.internal.Logging$$Lambda/0x00000070014d7b68.apply$mcV$sp(Unknown
>  Source)
> at org.apache.spark.internal.Logging.withLogContext(Logging.scala:167)
> at org.apache.spark.internal.Logging.withLogContext$(Logging.scala:158)
> at 
> org.apache.spark.sql.connect.execution.ExecuteGrpcResponseSender.withLogContext(ExecuteGrpcResponseSender.scala:43)
> at org.apache.spark.internal.Logging.logInfo(Logging.scala:187)
> at org.apache.spark.internal.Logging.logInfo$(Logging.scala:184)
> at 
> org.apache.spark.sql.connect.execution.ExecuteGrpcResponseSender.logInfo(ExecuteGrpcResponseSender.scala:43)
> at 
> org.apache.spark.sql.connect.execution.ExecuteGrpcResponseSender.execute(ExecuteGrpcResponseSender.scala:331)
> at 
> org.apache.spark.sql.connect.execution.ExecuteGrpcResponseSender$$anon$1.run(ExecuteGrpcResponseSender.scala:102)
>  
>    Locked ownable synchronizers:
>  - None
>  
> "shuffle-exchange-0" #134 [86019] daemon prio=5 os_prio=31 cpu=15.89ms 
> elapsed=37.95s tid=0x000000012c14f600 nid=86019 waiting on condition  
> [0x000000030d67e000]
>    java.lang.Thread.State: TIMED_WAITING (parking)
> at jdk.internal.misc.Unsafe.park(java.base@21.0.6/Native Method)
>  - parking to wait for  <0x00000007c444d0a8> (a 
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
> at 
> java.util.concurrent.locks.LockSupport.parkNanos(java.base@21.0.6/LockSupport.java:269)
> at 
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(java.base@21.0.6/AbstractQueuedSynchronizer.java:1763)
> at 
> java.util.concurrent.LinkedBlockingQueue.poll(java.base@21.0.6/LinkedBlockingQueue.java:460)
> at 
> java.util.concurrent.ThreadPoolExecutor.getTask(java.base@21.0.6/ThreadPoolExecutor.java:1069)
> at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(java.base@21.0.6/ThreadPoolExecutor.java:1130)
> at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(java.base@21.0.6/ThreadPoolExecutor.java:642)
> at java.lang.Thread.runWith(java.base@21.0.6/Thread.java:1596)
> at java.lang.Thread.run(java.base@21.0.6/Thread.java:1583)
>  
>    Locked ownable synchronizers:
>  - None
>  
> "Attach Listener" #137 [68103] daemon prio=9 os_prio=31 cpu=1.00ms 
> elapsed=0.13s tid=0x000000010e873a00 nid=68103 waiting on condition  
> [0x0000000000000000]
>    java.lang.Thread.State: RUNNABLE
>  
>    Locked ownable synchronizers:
>  - None
>  
> "G1 Conc#1" os_prio=31 cpu=63.91ms elapsed=44.67s tid=0x000000012f6b8100 
> nid=41731 runnable  
>  
> "GC Thread#8" os_prio=31 cpu=16.39ms elapsed=44.67s tid=0x000000012f7aa200 
> nid=41987 runnable  
>  
> "GC Thread#7" os_prio=31 cpu=28.43ms elapsed=44.67s tid=0x000000012f7a9c90 
> nid=33795 runnable  
>  
> "GC Thread#6" os_prio=31 cpu=13.51ms elapsed=44.67s tid=0x000000012f7a9720 
> nid=33539 runnable  
>  
> "GC Thread#5" os_prio=31 cpu=28.57ms elapsed=45.12s tid=0x000000012f622400 
> nid=43011 runnable  
>  
> "GC Thread#4" os_prio=31 cpu=23.59ms elapsed=45.12s tid=0x000000012f621e90 
> nid=43267 runnable  
>  
> "GC Thread#3" os_prio=31 cpu=33.29ms elapsed=45.12s tid=0x000000012f621920 
> nid=32771 runnable  
>  
> "GC Thread#2" os_prio=31 cpu=23.20ms elapsed=45.12s tid=0x000000012f6213b0 
> nid=26627 runnable  
>  
> "GC Thread#1" os_prio=31 cpu=27.39ms elapsed=45.12s tid=0x000000012f620e40 
> nid=26115 runnable  
>  
> "VM Thread" os_prio=31 cpu=36.00ms elapsed=45.28s tid=0x000000011800a740 
> nid=18179 runnable  
>  
> "VM Periodic Task Thread" os_prio=31 cpu=12.12ms elapsed=45.28s 
> tid=0x0000000118007a70 nid=20995 waiting on condition  
>  
> "G1 Service" os_prio=31 cpu=1.37ms elapsed=45.28s tid=0x000000010f704ba0 
> nid=16643 runnable  
>  
> "G1 Refine#0" os_prio=31 cpu=15.57ms elapsed=45.28s tid=0x000000010f894800 
> nid=13059 runnable  
>  
> "G1 Conc#0" os_prio=31 cpu=64.76ms elapsed=45.28s tid=0x000000010f704100 
> nid=12547 runnable  
>  
> "G1 Main Marker" os_prio=31 cpu=0.75ms elapsed=45.28s tid=0x00000001180042b0 
> nid=12035 runnable  
>  
> "GC Thread#0" os_prio=31 cpu=25.43ms elapsed=45.28s tid=0x000000010f6071f0 
> nid=13827 runnable  
>  
> JNI global refs: 52, weak refs: 4
>  
> will be opening a PR for the same..



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org