[
https://issues.apache.org/jira/browse/HDFS-6865?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14112573#comment-14112573
]
stack commented on HDFS-6865:
-----------------------------
I tried this patch on a small cluster doing TestHLogPE. All seems to work same
as w/o the patch; no obvious regression. This test is not good for seeing
benefit of this patch being a single file write contended over by many threads
appending and trying to sync as fast as they can. That said, comparing perf
stats of tests where there was no native available to tip of branch-2 as of
last night and then to branch-2+this patch, there is no discernible gain/loss
with this patch in place.
Indirectly related, as a user, how would I know this improvement is in effect?
Its 'on' all the time but what say, if native is not available, how as a user
do I get a clue I'm missing out on nice checksum speedup? For myself, I hacked
in logging into DFSClient so could confirm this patch was in effect. I could
make a new issue to add this formally. Might help especially in the client case.
> Byte array native checksumming on client side (HDFS changes)
> ------------------------------------------------------------
>
> Key: HDFS-6865
> URL: https://issues.apache.org/jira/browse/HDFS-6865
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: hdfs-client, performance
> Reporter: James Thomas
> Assignee: James Thomas
> Attachments: HDFS-6865.2.patch, HDFS-6865.3.patch, HDFS-6865.4.patch,
> HDFS-6865.5.patch, HDFS-6865.6.patch, HDFS-6865.7.patch, HDFS-6865.8.patch,
> HDFS-6865.patch
>
>
> Refactor FSOutputSummer to buffer data and use the native checksum
> calculation functionality introduced in HADOOP-10975.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
