Attila Doroszlai created HDDS-12313:
---------------------------------------
Summary: Intermittent fork timeout in
TestHSync#testConcurrentExceptionHandling
Key: HDDS-12313
URL: https://issues.apache.org/jira/browse/HDDS-12313
Project: Apache Ozone
Issue Type: Sub-task
Affects Versions: 2.0.0
Reporter: Attila Doroszlai
Assignee: Wei-Chiu Chuang
{{KeyOutputStream#close()}} is blocked, waiting for write lock:
{code}
"main"
java.lang.Thread.State: WAITING
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:836)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireQueued(AbstractQueuedSynchronizer.java:870)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer.acquire(AbstractQueuedSynchronizer.java:1199)
at
java.util.concurrent.locks.ReentrantLock$NonfairSync.lock(ReentrantLock.java:209)
at java.util.concurrent.locks.ReentrantLock.lock(ReentrantLock.java:285)
at
org.apache.hadoop.ozone.client.io.KeyOutputStream.doInWriteLock(KeyOutputStream.java:258)
at
org.apache.hadoop.ozone.client.io.KeyOutputStream.close(KeyOutputStream.java:647)
at
org.apache.hadoop.ozone.client.io.OzoneOutputStream.close(OzoneOutputStream.java:105)
at
org.apache.hadoop.fs.ozone.OzoneFSOutputStream.close(OzoneFSOutputStream.java:70)
at
org.apache.hadoop.fs.FSDataOutputStream$PositionCache.close(FSDataOutputStream.java:77)
at
org.apache.hadoop.fs.FSDataOutputStream.close(FSDataOutputStream.java:106)
at
org.apache.hadoop.fs.ozone.TestHSync.testConcurrentExceptionHandling(TestHSync.java:853)
{code}
Write lock is held by thread waiting for free buffer in BufferPool:
{code}
"Syncer-6"
java.lang.Thread.State: WAITING
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at
org.apache.hadoop.hdds.scm.storage.BufferPool.allocateBuffer(BufferPool.java:98)
at
org.apache.hadoop.hdds.scm.storage.BlockOutputStream.allocateNewBufferIfNeeded(BlockOutputStream.java:429)
at
org.apache.hadoop.hdds.scm.storage.BlockOutputStream.write(BlockOutputStream.java:386)
at
org.apache.hadoop.ozone.client.io.BlockOutputStreamEntry.write(BlockOutputStreamEntry.java:186)
at
org.apache.hadoop.ozone.client.io.KeyOutputStream.writeToOutputStream(KeyOutputStream.java:307)
at
org.apache.hadoop.ozone.client.io.KeyOutputStream.handleWrite(KeyOutputStream.java:283)
at
org.apache.hadoop.ozone.client.io.KeyOutputStream.lambda$write$1(KeyOutputStream.java:249)
at
org.apache.hadoop.ozone.client.io.KeyOutputStream$$Lambda$1871/1147820709.run(Unknown
Source)
at
org.apache.hadoop.ozone.client.io.KeyOutputStream.doInWriteLock(KeyOutputStream.java:260)
at
org.apache.hadoop.ozone.client.io.KeyOutputStream.write(KeyOutputStream.java:248)
at
org.apache.hadoop.ozone.client.io.OzoneOutputStream.write(OzoneOutputStream.java:94)
at
org.apache.hadoop.fs.ozone.OzoneFSOutputStream.lambda$write$1(OzoneFSOutputStream.java:58)
at
org.apache.hadoop.fs.ozone.OzoneFSOutputStream$$Lambda$1870/2092940497.run(Unknown
Source)
at
org.apache.hadoop.hdds.tracing.TracingUtil.executeInSpan(TracingUtil.java:184)
at
org.apache.hadoop.hdds.tracing.TracingUtil.executeInNewSpan(TracingUtil.java:149)
at
org.apache.hadoop.fs.ozone.OzoneFSOutputStream.write(OzoneFSOutputStream.java:54)
at
org.apache.hadoop.fs.FSDataOutputStream$PositionCache.write(FSDataOutputStream.java:62)
at java.io.DataOutputStream.write(DataOutputStream.java:107)
at java.io.FilterOutputStream.write(FilterOutputStream.java:97)
at
org.apache.hadoop.fs.ozone.TestHSync.lambda$runConcurrentWriteHSyncWithException$9(TestHSync.java:873)
at
org.apache.hadoop.fs.ozone.TestHSync$$Lambda$2189/1164139233.run(Unknown Source)
{code}
5/100 runs:
https://github.com/adoroszlai/ozone/actions/runs/13271802577
*
https://github.com/adoroszlai/ozone-build-results/blob/master/2024/12/08/35245/it-filesystem/2024-12-08T14-37-06_480-jvmRun1.dump
*
https://github.com/adoroszlai/ozone-build-results/blob/master/2025/01/27/36193/it-filesystem/2025-01-27T07-34-21_545-jvmRun1.dump
*
https://github.com/adoroszlai/ozone-build-results/blob/master/2025/01/29/36234/it-filesystem/2025-01-29T14-38-56_041-jvmRun1.dump
*
https://github.com/adoroszlai/ozone-build-results/blob/master/2025/02/10/36510/it-filesystem/2025-02-10T19-18-58_666-jvmRun1.dump
*
https://github.com/adoroszlai/ozone-build-results/blob/master/2025/02/11/36535/it-filesystem/2025-02-11T09-36-12_792-jvmRun1.dump
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
