[
https://issues.apache.org/jira/browse/HDFS-8901?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15435642#comment-15435642
]
Zhe Zhang commented on HDFS-8901:
---------------------------------
Sorry about the delay. I just finished reviewing the patch. It needs a rebase,
most likely due to HDFS-8905.
# The below code can be moved to {{BlockReaderUtil}}:
{code}
+ //Behave exactly as the readAll() call
+ ByteBuffer tmp = buf.duplicate();
+ tmp.limit(tmp.position() + len);
+ tmp = tmp.slice();
...
{code}
# This is a little confusing and actually IntelliJ can't find where {{ret}} is
declared. Can we change to regular assignments?
{code}
int nread = 0, ret;
{code}
# Just to clarify, is the plan to add a {{ByteBuffer}} version of the below
positional read public API?
{code}
public int read(long position, byte[] buffer, int offset, int length)
{code}
# Why was this not needed before but is needed now?
{code}
+ strategy.getReadBuffer().flip();
{code}
# Below formatting looks odd. Maybe an IDE setting issue?
{code}
+ private static void calculateChunkPositionsInBuf(int cellSize,
+ AlignedStripe[] stripes,
+ StripingCell[] cells,
+ ByteBuffer bu
{code}
# So with the changes in {{DFSInputStream}} and {{DFSStripedInputStream}},
every {{StripingChunk}} should use {{ByteBuffer}} instead of byte array right?
Can we unify the usage of {{byteBuffer}} and {{chunkBuffer}}? Looks like
{{ChunkByteBuffer}} can just handle a special case of a single {{ByteBuffer}}?
# Just to clarify, we will improve performance of the below operation only
after implementing the new public API for {{ByteBuffer}}-based positional read
right? Because otherwise {{chunk.getChunkBuffer}} will be an indirect buffer
backed by a regular on-heap array.
{code}
- chunk.copyTo(decodeInputs[i]);
+ if (chunk.useChunkBuffer()) {
+ chunk.getChunkBuffer().copyTo(decodeInputs[i]);
+ }
{code}
> Use ByteBuffer in striping positional read
> ------------------------------------------
>
> Key: HDFS-8901
> URL: https://issues.apache.org/jira/browse/HDFS-8901
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Reporter: Kai Zheng
> Assignee: Youwei Wang
> Attachments: HDFS-8901-v10.patch, HDFS-8901-v2.patch,
> HDFS-8901-v3.patch, HDFS-8901-v4.patch, HDFS-8901-v5.patch,
> HDFS-8901-v6.patch, HDFS-8901-v7.patch, HDFS-8901-v8.patch,
> HDFS-8901-v9.patch, HDFS-8901.v11.patch, HDFS-8901.v12.patch,
> HDFS-8901.v13.patch, HDFS-8901.v14.patch, initial-poc.patch
>
>
> Native erasure coder prefers to direct ByteBuffer for performance
> consideration. To prepare for it, this change uses ByteBuffer through the
> codes in implementing striping position read. It will also fix avoiding
> unnecessary data copying between striping read chunk buffers and decode input
> buffers.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
