hi Noah, while we're still on the hadoop topic ... I was also trying out the TestDFSIO tests ceph v/s hadoop. The Read tests on ceph takes about 1.5x the hdfs time. The write tests are worse about ... 2.5x the time on hdfs, but I guess we have additional journaling overheads for the writes on ceph. But there should be no such overheads for the read ?
Is this something that you are already aware of ? [ by the way these results are after applying the data locality patches ] 13/07/09 12:16:52 INFO fs.TestDFSIO: ----- TestDFSIO ----- : read (For CephFS) 13/07/09 12:16:52 INFO fs.TestDFSIO: Date & time: Tue Jul 09 12:16:52 PDT 2013 13/07/09 12:16:52 INFO fs.TestDFSIO: Number of files: 300 13/07/09 12:16:52 INFO fs.TestDFSIO: Total MBytes processed: 458070 13/07/09 12:16:52 INFO fs.TestDFSIO: Throughput mb/sec: 27.356665605451205 13/07/09 12:16:52 INFO fs.TestDFSIO: Average IO rate mb/sec: 30.80777931213379 13/07/09 12:16:52 INFO fs.TestDFSIO: IO rate std deviation: 10.731377923459378 13/07/09 12:16:52 INFO fs.TestDFSIO: Test exec time sec: 388.969 13/07/09 12:16:52 INFO fs.TestDFSIO: 13/07/09 04:50:34 INFO fs.TestDFSIO: ----- TestDFSIO ----- : read (For HDFS) 13/07/09 04:50:34 INFO fs.TestDFSIO: Date & time: Tue Jul 09 04:50:34 PDT 2013 13/07/09 04:50:34 INFO fs.TestDFSIO: Number of files: 300 13/07/09 04:50:34 INFO fs.TestDFSIO: Total MBytes processed: 456388 13/07/09 04:50:34 INFO fs.TestDFSIO: Throughput mb/sec: 40.62858541727874 13/07/09 04:50:34 INFO fs.TestDFSIO: Average IO rate mb/sec: 48.635948181152344 13/07/09 04:50:34 INFO fs.TestDFSIO: IO rate std deviation: 27.46651216689178 13/07/09 04:50:34 INFO fs.TestDFSIO: Test exec time sec: 270.682 13/07/09 04:50:34 INFO fs.TestDFSIO: Thanks KC
_______________________________________________ ceph-users mailing list [email protected] http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
