[
https://issues.apache.org/jira/browse/HDFS-4530?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13586723#comment-13586723
]
Liang Xie commented on HDFS-4530:
---------------------------------
this's the front-end application oom stack trace:
2013-02-26 03:19:38,685 FATAL
org.apache.hadoop.hbase.regionserver.HRegionServer: ABORTING region server
sd-ml-hadoop24.bj,11600,1361419025919: Replay of HLog required. Forcing server
shutdown
org.apache.hadoop.hbase.DroppedSnapshotException: region:
real_distribution_sms_message,86,1361802699279.37bde9f1697340148f622ee262260f9a.
at
org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:1485)
at
org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:1364)
at
org.apache.hadoop.hbase.regionserver.HRegion.flushcache(HRegion.java:1305)
at
org.apache.hadoop.hbase.regionserver.MemStoreFlusher.flushRegion(MemStoreFlusher.java:413)
at
org.apache.hadoop.hbase.regionserver.MemStoreFlusher.flushOneForGlobalPressure(MemStoreFlusher.java:209)
at
org.apache.hadoop.hbase.regionserver.MemStoreFlusher.run(MemStoreFlusher.java:230)
at java.lang.Thread.run(Thread.java:662)
Caused by: java.lang.OutOfMemoryError: Direct buffer memory
at java.nio.Bits.reserveMemory(Bits.java:632)
at java.nio.DirectByteBuffer.<init>(DirectByteBuffer.java:97)
at java.nio.ByteBuffer.allocateDirect(ByteBuffer.java:288)
at
org.apache.hadoop.hdfs.util.DirectBufferPool.getBuffer(DirectBufferPool.java:70)
at
org.apache.hadoop.hdfs.BlockReaderLocal.<init>(BlockReaderLocal.java:315)
at
org.apache.hadoop.hdfs.BlockReaderLocal.newBlockReader(BlockReaderLocal.java:208)
at
org.apache.hadoop.hdfs.DFSClient.getLocalBlockReader(DFSClient.java:766)
at
org.apache.hadoop.hdfs.DFSInputStream.getBlockReader(DFSInputStream.java:888)
at
org.apache.hadoop.hdfs.DFSInputStream.blockSeekTo(DFSInputStream.java:455)
at
org.apache.hadoop.hdfs.DFSInputStream.readWithStrategy(DFSInputStream.java:645)
at org.apache.hadoop.hdfs.DFSInputStream.read(DFSInputStream.java:689)
at java.io.DataInputStream.readFully(DataInputStream.java:178)
at
org.apache.hadoop.hbase.io.hfile.FixedFileTrailer.readFromStream(FixedFileTrailer.java:312)
at
org.apache.hadoop.hbase.io.hfile.HFile.pickReaderVersion(HFile.java:543)
at
org.apache.hadoop.hbase.io.hfile.HFile.createReaderWithEncoding(HFile.java:589)
at
org.apache.hadoop.hbase.regionserver.StoreFile$Reader.<init>(StoreFile.java:1252)
at
org.apache.hadoop.hbase.regionserver.StoreFile.open(StoreFile.java:512)
at
org.apache.hadoop.hbase.regionserver.StoreFile.createReader(StoreFile.java:602)
at
org.apache.hadoop.hbase.regionserver.Store.validateStoreFile(Store.java:1654)
at org.apache.hadoop.hbase.regionserver.Store.commitFile(Store.java:793)
at org.apache.hadoop.hbase.regionserver.Store.access$500(Store.java:109)
at
org.apache.hadoop.hbase.regionserver.Store$StoreFlusherImpl.commit(Store.java:2295)
at
org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:1466)
... 6 more
> return buffer into direct bufferPool in BlockReaderLocal as possible
> --------------------------------------------------------------------
>
> Key: HDFS-4530
> URL: https://issues.apache.org/jira/browse/HDFS-4530
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: datanode
> Affects Versions: 3.0.0
> Reporter: Liang Xie
> Assignee: Liang Xie
> Attachments: HDFS-4530.txt
>
>
> {code}
> public synchronized void close() throws IOException {
> dataIn.close();
> if (checksumIn != null) {
> checksumIn.close();
> }
> if (slowReadBuff != null) {
> bufferPool.returnBuffer(slowReadBuff);
> slowReadBuff = null;
> }
> if (checksumBuff != null) {
> bufferPool.returnBuffer(checksumBuff);
> checksumBuff = null;
> }
> startOffset = -1;
> checksum = null;
> }
> {code}
> If there's an IOException occurred in dataIn.close(), then the
> slowReadBuff&checksumBuff could not be returned anymore. let's make a
> trivial change to reduce this risk.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
