[
https://issues.apache.org/jira/browse/HDFS-14820?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16926304#comment-16926304
]
Lisheng Sun edited comment on HDFS-14820 at 9/10/19 3:38 AM:
-------------------------------------------------------------
hi [~elgoiri]
{quote}What is the current default value? 8KB?
{quote}
as follow code, current default value is 8KB.
{code:java}
final DataOutputStream out = new DataOutputStream(new BufferedOutputStream(
peer.getOutputStream()));
public BufferedOutputStream(OutputStream out) {
this(out, 8192);
}
{code}
i have updated buffer is 512B, taken a lot tests and the resut is ok. I can do
the pressure test and use the new buffer in our prodution environment later.
i agree your suggestion，we can first make it configurable and make the default
the old value.
Adjust the buffer according to user need.
was (Author: leosun08):
hi [~elgoiri]
{quote}What is the current default value? 8KB?
{quote}
as follow code, current default value is 8KB.
{code:java}
final DataOutputStream out = new DataOutputStream(new BufferedOutputStream(
peer.getOutputStream()));
public BufferedOutputStream(OutputStream out) {
this(out, 8192);
}
{code}
i have updated buffer is 512B, taken a test and the resut is ok.
i agree your suggestion，make it configurable and make the default the old
value.
> The default 8KB buffer of
> BlockReaderRemote#newBlockReader#BufferedOutputStream is too big
> -------------------------------------------------------------------------------------------
>
> Key: HDFS-14820
> URL: https://issues.apache.org/jira/browse/HDFS-14820
> Project: Hadoop HDFS
> Issue Type: Bug
> Reporter: Lisheng Sun
> Assignee: Lisheng Sun
> Priority: Major
> Attachments: HDFS-14820.001.patch
>
>
> this issue is similar to HDFS-14535.
> {code:java}
> public static BlockReader newBlockReader(String file,
> ExtendedBlock block,
> Token<BlockTokenIdentifier> blockToken,
> long startOffset, long len,
> boolean verifyChecksum,
> String clientName,
> Peer peer, DatanodeID datanodeID,
> PeerCache peerCache,
> CachingStrategy cachingStrategy,
> int networkDistance) throws IOException {
> // in and out will be closed when sock is closed (by the caller)
> final DataOutputStream out = new DataOutputStream(new BufferedOutputStream(
> peer.getOutputStream()));
> new Sender(out).readBlock(block, blockToken, clientName, startOffset, len,
> verifyChecksum, cachingStrategy);
> }
> public BufferedOutputStream(OutputStream out) {
> this(out, 8192);
> }
> {code}
> Sender#readBlock parameter( block,blockToken, clientName, startOffset, len,
> verifyChecksum, cachingStrategy) could not use such a big buffer.
> So i think it should reduce BufferedOutputStream buffer.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
