[
https://issues.apache.org/jira/browse/HDFS-7276?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14197115#comment-14197115
]
stack commented on HDFS-7276:
-----------------------------
This seems to be holding up progress:
{code}
"sync.0" #35 prio=5 os_prio=0 tid=0x00007f5c281d2000 nid=0x6f7e in
Object.wait() [0x00007f5bf71e5000]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
at java.lang.Object.wait(Object.java:502)
at
org.apache.hadoop.hdfs.util.ByteArrayManager$FixedLengthManager.allocate(ByteArrayManager.java:174)
- locked <0x00000002280915c0> (a
org.apache.hadoop.hdfs.util.ByteArrayManager$FixedLengthManager)
at
org.apache.hadoop.hdfs.util.ByteArrayManager$Impl.newByteArray(ByteArrayManager.java:374)
at
org.apache.hadoop.hdfs.DFSOutputStream.createPacket(DFSOutputStream.java:192)
at
org.apache.hadoop.hdfs.DFSOutputStream.writeChunk(DFSOutputStream.java:1868)
- locked <0x00000002279bbc40> (a org.apache.hadoop.hdfs.DFSOutputStream)
at
org.apache.hadoop.fs.FSOutputSummer.writeChecksumChunks(FSOutputSummer.java:206)
at
org.apache.hadoop.fs.FSOutputSummer.flushBuffer(FSOutputSummer.java:163)
- locked <0x00000002279bbc40> (a org.apache.hadoop.hdfs.DFSOutputStream)
at org.apache.hadoop.fs.FSOutputSummer.flush(FSOutputSummer.java:182)
at java.io.FilterOutputStream.flush(FilterOutputStream.java:140)
at java.io.DataOutputStream.flush(DataOutputStream.java:123)
at
org.apache.hadoop.hbase.regionserver.wal.ProtobufLogWriter.sync(ProtobufLogWriter.java:167)
at
org.apache.hadoop.hbase.regionserver.wal.FSHLog$SyncRunner.run(FSHLog.java:1302)
at java.lang.Thread.run(Thread.java:745)
{code}
> Limit the number of byte arrays used by DFSOutputStream
> -------------------------------------------------------
>
> Key: HDFS-7276
> URL: https://issues.apache.org/jira/browse/HDFS-7276
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: hdfs-client
> Reporter: Tsz Wo Nicholas Sze
> Assignee: Tsz Wo Nicholas Sze
> Fix For: 2.6.0
>
> Attachments: h7276_20141021.patch, h7276_20141022.patch,
> h7276_20141023.patch, h7276_20141024.patch, h7276_20141027.patch,
> h7276_20141027b.patch, h7276_20141028.patch, h7276_20141029.patch,
> h7276_20141029b.patch, h7276_20141030.patch, h7276_20141031.patch
>
>
> When there are a lot of DFSOutputStream's writing concurrently, the number of
> outstanding packets could be large. The byte arrays created by those packets
> could occupy a lot of memory.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
