[
https://issues.apache.org/jira/browse/HDFS-6865?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
James Thomas updated HDFS-6865:
-------------------------------
Attachment: HDFS-6865.6.patch
[~tlipcon], thanks for the review. Made all of your suggested changes except
the one about calling {{writeChunk}} with larger buffers -- if you look at
{{DFSOutputStream#writeChunk}}, you'll see that it has some assertions that
check that only a single chunk has been passed in. It also calls
{{Packet#writeData}} and {{Packet#writeChecksum}}, which expect that the
written data will not overflow the packet. So we would need some logic that
checks whether the write will overflow the packet and, if so, writes only
enough to fill the packet, queues it, and then allocates a new packet and
writes the remainder to that. Sounds like this is best left for later.
> Byte array native checksumming on client side (HDFS changes)
> ------------------------------------------------------------
>
> Key: HDFS-6865
> URL: https://issues.apache.org/jira/browse/HDFS-6865
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: hdfs-client, performance
> Reporter: James Thomas
> Assignee: James Thomas
> Attachments: HDFS-6865.2.patch, HDFS-6865.3.patch, HDFS-6865.4.patch,
> HDFS-6865.5.patch, HDFS-6865.6.patch, HDFS-6865.patch
>
>
> Refactor FSOutputSummer to buffer data and use the native checksum
> calculation functionality introduced in HADOOP-10975.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
