Pratyush Bhatt created HDDS-11269:
-------------------------------------
Summary: KeyOutputStream throws "java.io.IOException: : Stream is
closed"
Key: HDDS-11269
URL: https://issues.apache.org/jira/browse/HDDS-11269
Project: Apache Ozone
Issue Type: Bug
Reporter: Pratyush Bhatt
RegionServers are getting aborted on a build that has HDDS-11193 present. This
is happening almost every time, even on a fresh deployment.
{code:java}
2024-08-01 12:32:01,175 ERROR
org.apache.hadoop.hbase.regionserver.HRegionServer: ***** ABORTING region
server vc0134.xyz,22101,1722484321791: Failed log close in log roller *****
org.apache.hadoop.hbase.regionserver.wal.FailedLogCloseException:
ofs://ozone1721625030/hbasevol-01082024/hbasebuck-new/hbase/WALs/vc0134.xyz,22101,1722484321791/vc0134.xyz.com%2C22101%2C1722484321791.vc0134.xyz%2C22101%2C1722484321791.regiongroup-7.1722539713402,
unflushedEntries=3
at
org.apache.hadoop.hbase.regionserver.wal.FSHLog.doReplaceWriter(FSHLog.java:428)
at
org.apache.hadoop.hbase.regionserver.wal.FSHLog.doReplaceWriter(FSHLog.java:71)
at
org.apache.hadoop.hbase.regionserver.wal.AbstractFSWAL.replaceWriter(AbstractFSWAL.java:830)
at
org.apache.hadoop.hbase.regionserver.wal.AbstractFSWAL.rollWriter(AbstractFSWAL.java:889)
at
org.apache.hadoop.hbase.wal.AbstractWALRoller$RollController.rollWal(AbstractWALRoller.java:304)
at
org.apache.hadoop.hbase.wal.AbstractWALRoller.run(AbstractWALRoller.java:211)
Caused by:
org.apache.hadoop.hbase.regionserver.wal.FailedSyncBeforeLogCloseException:
java.io.IOException: : Stream is closed! Key:
hbase/WALs/vc0134.xyz,22101,1722484321791/vc0134.xyz%2C22101%2C1722484321791.vc0134.xyz%2C22101%2C1722484321791.regiongroup-7.1722539713402
at
org.apache.hadoop.hbase.regionserver.wal.FSHLog$SafePointZigZagLatch.checkIfSyncFailed(FSHLog.java:908)
at
org.apache.hadoop.hbase.regionserver.wal.FSHLog$SafePointZigZagLatch.waitSafePoint(FSHLog.java:922)
at
org.apache.hadoop.hbase.regionserver.wal.FSHLog.doReplaceWriter(FSHLog.java:368)
... 5 more
Caused by: java.io.IOException: : Stream is closed! Key:
hbase/WALs/vc0134.xyz,22101,1722484321791/vc0134.xyz%2C22101%2C1722484321791.vc0134.xyz%2C22101%2C1722484321791.regiongroup-7.1722539713402
at
org.apache.hadoop.ozone.client.io.KeyOutputStream.checkNotClosed(KeyOutputStream.java:739)
at
org.apache.hadoop.ozone.client.io.KeyOutputStream.flush(KeyOutputStream.java:441)
at
org.apache.hadoop.ozone.client.io.OzoneOutputStream.flush(OzoneOutputStream.java:99)
at
org.apache.hadoop.hdds.tracing.TracingUtil.executeInSpan(TracingUtil.java:184)
at
org.apache.hadoop.hdds.tracing.TracingUtil.executeInNewSpan(TracingUtil.java:149)
at
org.apache.hadoop.fs.ozone.OzoneFSOutputStream.flush(OzoneFSOutputStream.java:64)
at java.io.FilterOutputStream.flush(FilterOutputStream.java:140)
at java.io.DataOutputStream.flush(DataOutputStream.java:123)
at
org.apache.hadoop.hbase.regionserver.wal.ProtobufLogWriter.sync(ProtobufLogWriter.java:80)
at
org.apache.hadoop.hbase.regionserver.wal.FSHLog$SyncRunner.run(FSHLog.java:669)
{code}
cc: [~duongnguyen] [~ashishkr] [~weichiu]
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
