[
https://issues.apache.org/jira/browse/HDFS-6561?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
James Thomas updated HDFS-6561:
-------------------------------
Attachment: HDFS-6561.patch
Patch attached.
Was finally able to get some perf stat output comparing a 1 GB put operation
with this change + HDFS-6560 and without. I ran the put against a
MiniDFSCluster so that everything was in a single process and we could easily
measure cycle savings across the cluster. We see about a 10% reduction in CPU
cycles and a 4% reduction in wall clock time. These measurements were taken on
my personal computer (SSD, quad core i7).
****WITHOUT CHANGES****
james@james-ThinkPad-T530:~/Desktop/hadoop-common$ perf stat -r 20 java
org.junit.runner.JUnitCore org.apache.hadoop.hdfs.TestRWCycles
Performance counter stats for 'java org.junit.runner.JUnitCore
org.apache.hadoop.hdfs.TestWriteCycles' (20 runs):
14848.579075 task-clock (msec) # 1.637 CPUs utilized
( +- 0.37% )
75,748 context-switches # 0.005 M/sec
( +- 0.29% )
3,855 cpu-migrations # 0.260 K/sec
( +- 1.66% )
75,120 page-faults # 0.005 M/sec
( +- 2.15% )
50,376,887,490 cycles # 3.393 GHz
( +- 0.40% )
31,427,207,136 stalled-cycles-frontend # 62.38% frontend cycles idle
( +- 0.62% )
<not supported> stalled-cycles-backend
46,918,968,840 instructions # 0.93 insns per cycle
# 0.67 stalled cycles per insn
( +- 0.26% )
6,523,666,030 branches # 439.346 M/sec
( +- 0.39% )
304,740,080 branch-misses # 4.67% of all branches
( +- 0.19% )
9.068159220 seconds time elapsed
( +- 0.33% )
****WITH HDFS-6560 and HDFS-6561 (client-side checksum calculation and DN
checksum verification both nativized)****
james@james-ThinkPad-T530:~/Desktop/hadoop-common$ perf stat -r 20 java
org.junit.runner.JUnitCore org.apache.hadoop.hdfs.TestWriteCycles
Performance counter stats for 'java org.junit.runner.JUnitCore
org.apache.hadoop.hdfs.TestRWCycles' (20 runs):
13703.187333 task-clock (msec) # 1.575 CPUs utilized
( +- 0.74% )
79,256 context-switches # 0.006 M/sec
( +- 0.27% )
3,161 cpu-migrations # 0.231 K/sec
( +- 2.45% )
72,138 page-faults # 0.005 M/sec
( +- 2.35% )
46,553,899,994 cycles # 3.397 GHz
( +- 0.83% )
30,670,949,579 stalled-cycles-frontend # 65.88% frontend cycles idle
( +- 1.08% )
<not supported> stalled-cycles-backend
35,616,207,543 instructions # 0.77 insns per cycle
# 0.86 stalled cycles per insn
( +- 0.73% )
6,614,919,487 branches # 482.729 M/sec
( +- 0.73% )
313,683,166 branch-misses # 4.74% of all branches
( +- 0.56% )
8.699152006 seconds time elapsed
( +- 0.44% )
> Byte array native checksumming on client side
> ---------------------------------------------
>
> Key: HDFS-6561
> URL: https://issues.apache.org/jira/browse/HDFS-6561
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: datanode, hdfs-client, performance
> Reporter: James Thomas
> Assignee: James Thomas
> Attachments: HDFS-6561.patch
>
>
--
This message was sent by Atlassian JIRA
(v6.2#6252)
