[
https://issues.apache.org/jira/browse/HDDS-11239?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17868987#comment-17868987
]
Duong commented on HDDS-11239:
------------------------------
Thanks for reporting this [~ashishkr]. There are usually some warning logs
before this error, starting with "Encountered exception" when hitting this
error. Can you see if those exist in your logs when you encounter this error?
> Flakiness in KeyOutputStream exception handling
> -----------------------------------------------
>
> Key: HDDS-11239
> URL: https://issues.apache.org/jira/browse/HDDS-11239
> Project: Apache Ozone
> Issue Type: Sub-task
> Reporter: Duong
> Priority: Major
>
> {code:java}
> 2024-07-23 05:32:26,367 ERROR
> org.apache.hadoop.hbase.regionserver.CompactSplit: Compaction failed
> region=11loadtesttabtyui22aaqw23,a22221f2,1721725742844.9bcd68da6a7f7c0150854b20b2340a41.,
> storeName=9bcd68da6a7f7c0150854b20b2340a41/ch8, priority=13,
> startTime=1721737880870
> java.io.IOException: java.lang.IllegalArgumentException
> at
> org.apache.hadoop.hbase.regionserver.wal.AbstractFSWAL.ensureIOException(AbstractFSWAL.java:858)
> at
> org.apache.hadoop.hbase.regionserver.wal.AbstractFSWAL.blockOnSync(AbstractFSWAL.java:853)
> at
> org.apache.hadoop.hbase.regionserver.wal.FSHLog.publishSyncThenBlockOnCompletion(FSHLog.java:809)
> at
> org.apache.hadoop.hbase.regionserver.wal.FSHLog.sync(FSHLog.java:858)
> at
> org.apache.hadoop.hbase.regionserver.wal.FSHLog.sync(FSHLog.java:847)
> at
> org.apache.hadoop.hbase.regionserver.wal.WALUtil.doFullMarkerAppendTransaction(WALUtil.java:163)
> at
> org.apache.hadoop.hbase.regionserver.wal.WALUtil.writeMarker(WALUtil.java:140)
> at
> org.apache.hadoop.hbase.regionserver.wal.WALUtil.writeCompactionMarker(WALUtil.java:74)
> at
> org.apache.hadoop.hbase.regionserver.HStore.writeCompactionWalRecord(HStore.java:1262)
> at
> org.apache.hadoop.hbase.regionserver.HStore.lambda$replaceStoreFiles$3(HStore.java:1272)
> at
> org.apache.hadoop.hbase.regionserver.StoreEngine.replaceStoreFiles(StoreEngine.java:483)
> at
> org.apache.hadoop.hbase.regionserver.HStore.replaceStoreFiles(HStore.java:1270)
> at
> org.apache.hadoop.hbase.regionserver.HStore.doCompaction(HStore.java:1206)
> at
> org.apache.hadoop.hbase.regionserver.HStore.compact(HStore.java:1188)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.compact(HRegion.java:2261)
> at
> org.apache.hadoop.hbase.regionserver.CompactSplit$CompactionRunner.doCompaction(CompactSplit.java:625)
> at
> org.apache.hadoop.hbase.regionserver.CompactSplit$CompactionRunner.run(CompactSplit.java:673)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> Caused by: java.lang.IllegalArgumentException
> at
> org.apache.hadoop.ozone.shaded.com.google.common.base.Preconditions.checkArgument(Preconditions.java:129)
> at
> org.apache.hadoop.ozone.client.io.KeyOutputStream.handleException(KeyOutputStream.java:318)
> at
> org.apache.hadoop.ozone.client.io.KeyOutputStream.handleFlushOrClose(KeyOutputStream.java:499)
> at
> org.apache.hadoop.ozone.client.io.KeyOutputStream.hsync(KeyOutputStream.java:463)
> at
> org.apache.hadoop.ozone.client.io.OzoneOutputStream.hsync(OzoneOutputStream.java:118)
> at
> org.apache.hadoop.hdds.tracing.TracingUtil.executeInSpan(TracingUtil.java:184)
> at
> org.apache.hadoop.hdds.tracing.TracingUtil.executeInNewSpan(TracingUtil.java:149)
> at
> org.apache.hadoop.fs.ozone.OzoneFSOutputStream.hsync(OzoneFSOutputStream.java:80)
> at
> org.apache.hadoop.fs.ozone.OzoneFSOutputStream.hflush(OzoneFSOutputStream.java:75)
> at
> org.apache.hadoop.fs.FSDataOutputStream.hflush(FSDataOutputStream.java:136)
> at org.apache.h
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
