[
https://issues.apache.org/jira/browse/HADOOP-1649?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12515114
]
Raghu Angadi commented on HADOOP-1649:
--------------------------------------
With randomwriter (with 10 maps per client) on 13 nodes, I am able to
consistently reproduce 3% peformance regression compared pre-1134 trunk.
io.file.buffer.size is 64K and io.bytes.per.checksum is 512.
* pre-1134 : 3190 sec
* cur trunk : 3290 sec.
Only once, trunk took 3600 secs, not sure why.
Either of the following fixed this gap :
# matching buffer size for in/out sockets and for block file while writing a
block with buffering on pre-1134
# set io.bytes.per.checksum 64K
Regd matching the buffer size :
I will attach a patch that does this. But one negative thing about this is that
it has 3 extra copies compared to pre-1134. But this does not affect the
benchmarks since usually our benchmarks are not cpu bound on datanodes. The
following occur because datanode processes one checksum chunk at a time :
* from in-socket buffer to small buffer (of size io.bytes.per.checksum)
* small buffer to mirror-socket buffer,
* small buffer to block file buffer.
It personally feels pretty bad to be responsible for so many copies even if
benchmarks are not affected. (May be in Hadoop-0.15), we can avoid first two
copies by changing the write loop a little to do larger buffer reads instead of
on checksum chunk. The third can be avoided if checksum is sent on different
socket.
Another regression we have seen is with TestDFSIO. This one is not affected
noticeably on small clusters (<100) but aggregate rate came down by about 30%
on 500 node cluster. This is a shorter test in terms of time and I will look
into this as well.
> Performance regression with Block CRCs
> --------------------------------------
>
> Key: HADOOP-1649
> URL: https://issues.apache.org/jira/browse/HADOOP-1649
> Project: Hadoop
> Issue Type: Bug
> Affects Versions: 0.14.0
> Reporter: Raghu Angadi
> Assignee: Raghu Angadi
> Priority: Blocker
> Fix For: 0.14.0
>
>
> Performance is noticeably affected by Block Level CRCs patch (HADOOP-1134).
> This is more noticeable on writes (randomriter test etc).
> With random writer, it takes 20-25% on small cluster (20 nodes) and many be
> 10% on larger cluster.
> There are a few differences in how data is written with 1134. As soon as I
> can reproduce this, I think it will be easier to fix.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
