[
https://issues.apache.org/jira/browse/HDDS-11269?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17870598#comment-17870598
]
Duong commented on HDDS-11269:
------------------------------
This is probably the same as HDDS-11239. An error in exception handling closes
the KeyOutputStream. "java.io.IOException: Stream is closed" is a consequence.
> KeyOutputStream throws "java.io.IOException: : Stream is closed"
> ----------------------------------------------------------------
>
> Key: HDDS-11269
> URL: https://issues.apache.org/jira/browse/HDDS-11269
> Project: Apache Ozone
> Issue Type: Bug
> Reporter: Pratyush Bhatt
> Priority: Major
>
> RegionServers are getting aborted on a build that has HDDS-11193 present.
> This is happening almost every time, even on a fresh deployment.
> {code:java}
> 2024-08-01 12:32:01,175 ERROR
> org.apache.hadoop.hbase.regionserver.HRegionServer: ***** ABORTING region
> server vc0134.xyz,22101,1722484321791: Failed log close in log roller *****
> org.apache.hadoop.hbase.regionserver.wal.FailedLogCloseException:
> ofs://ozone1721625030/hbasevol-01082024/hbasebuck-new/hbase/WALs/vc0134.xyz,22101,1722484321791/vc0134.xyz.com%2C22101%2C1722484321791.vc0134.xyz%2C22101%2C1722484321791.regiongroup-7.1722539713402,
> unflushedEntries=3
> at
> org.apache.hadoop.hbase.regionserver.wal.FSHLog.doReplaceWriter(FSHLog.java:428)
> at
> org.apache.hadoop.hbase.regionserver.wal.FSHLog.doReplaceWriter(FSHLog.java:71)
> at
> org.apache.hadoop.hbase.regionserver.wal.AbstractFSWAL.replaceWriter(AbstractFSWAL.java:830)
> at
> org.apache.hadoop.hbase.regionserver.wal.AbstractFSWAL.rollWriter(AbstractFSWAL.java:889)
> at
> org.apache.hadoop.hbase.wal.AbstractWALRoller$RollController.rollWal(AbstractWALRoller.java:304)
> at
> org.apache.hadoop.hbase.wal.AbstractWALRoller.run(AbstractWALRoller.java:211)
> Caused by:
> org.apache.hadoop.hbase.regionserver.wal.FailedSyncBeforeLogCloseException:
> java.io.IOException: : Stream is closed! Key:
> hbase/WALs/vc0134.xyz,22101,1722484321791/vc0134.xyz%2C22101%2C1722484321791.vc0134.xyz%2C22101%2C1722484321791.regiongroup-7.1722539713402
> at
> org.apache.hadoop.hbase.regionserver.wal.FSHLog$SafePointZigZagLatch.checkIfSyncFailed(FSHLog.java:908)
> at
> org.apache.hadoop.hbase.regionserver.wal.FSHLog$SafePointZigZagLatch.waitSafePoint(FSHLog.java:922)
> at
> org.apache.hadoop.hbase.regionserver.wal.FSHLog.doReplaceWriter(FSHLog.java:368)
> ... 5 more
> Caused by: java.io.IOException: : Stream is closed! Key:
> hbase/WALs/vc0134.xyz,22101,1722484321791/vc0134.xyz%2C22101%2C1722484321791.vc0134.xyz%2C22101%2C1722484321791.regiongroup-7.1722539713402
> at
> org.apache.hadoop.ozone.client.io.KeyOutputStream.checkNotClosed(KeyOutputStream.java:739)
> at
> org.apache.hadoop.ozone.client.io.KeyOutputStream.flush(KeyOutputStream.java:441)
> at
> org.apache.hadoop.ozone.client.io.OzoneOutputStream.flush(OzoneOutputStream.java:99)
> at
> org.apache.hadoop.hdds.tracing.TracingUtil.executeInSpan(TracingUtil.java:184)
> at
> org.apache.hadoop.hdds.tracing.TracingUtil.executeInNewSpan(TracingUtil.java:149)
> at
> org.apache.hadoop.fs.ozone.OzoneFSOutputStream.flush(OzoneFSOutputStream.java:64)
> at java.io.FilterOutputStream.flush(FilterOutputStream.java:140)
> at java.io.DataOutputStream.flush(DataOutputStream.java:123)
> at
> org.apache.hadoop.hbase.regionserver.wal.ProtobufLogWriter.sync(ProtobufLogWriter.java:80)
> at
> org.apache.hadoop.hbase.regionserver.wal.FSHLog$SyncRunner.run(FSHLog.java:669)
> {code}
> cc: [~duongnguyen] [~ashishkr] [~weichiu]
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
