Ah, thanks, that makes sense. I see bug 14225 opened for the backport.
I'm looking at http://tracker.ceph.com/projects/ceph-releases/wiki/HOWTO_backport_commits, I'll see if I can get a PR up for that. -emile On 1/4/16, 3:11 PM, "Jason Dillaman" <dilla...@redhat.com> wrote: >There was a bug in the rbd CLI bench-write tool that would result in the same >offset being re-written [1]. Since writeback cache is enabled (by default), >in your example only 4MB would be written to the OSD at the conclusion of the >test. The fix should have been scheduled for backport to Hammer but it looks >like it was missed. I will open a new tracker ticket to start that process. > >[1] >https://github.com/ceph/ceph/commit/333f3a01a9916c781f266078391c580efb81a0fc > >-- > >Jason Dillaman > > >----- Original Message ----- >> From: "Emile Snyder" <emsny...@ebay.com> >> To: ceph-users@lists.ceph.com >> Sent: Monday, January 4, 2016 3:51:25 PM >> Subject: [ceph-users] rbd bench-write vs dd performance confusion >> >> Hi all, >> >> I'm trying to get comfortable with managing and benchmarking ceph clusters, >> and I'm struggling to understan rbd bench-write results vs using dd against >> mounted rbd images. >> >> I have a 6 node test cluster running version 0.94.5, 2 nodes per rack, 20 >> OSDs per node. Write journals are on the same disk as their OSD. My rbd pool >> is set for 3 replicas, with 2 on different hosts in a given rack, and 3rd on >> some host in a different rack. >> >> >> I created a test 100GB image with 4MB object size, created a VM client, and >> mounted the image at /dev/rbd1. >> >> In a shell on one of my 6 storage nodes I have 'iostat 2' running. >> >> Now my confusion; If I run on the client: >> >> 'sudo dd if=/dev/zero of=/dev/rbd1 bs=4M count=1000 iflag=fullblock >> oflag=direct' >> >> I see '4194304000 bytes (4.2 GB) copied, 18.5798 s, 226 MB/s' and the iostat >> on the storage node shows almost all 20 disks sustaining 4-16MB/s writes. >> >> However, if I run >> >> 'rbd --cluster <clustername> bench-write test-4m-image --io-size 4000000 >> --io-threads 1 --io-total 40000000000 --io-pattern rand' >> >> I see 'elapsed: 12 ops: 10000 ops/sec: 805.86 bytes/sec: >> 3223441447.72' but the iostat shows the disks basically all at 0.00kb_wrtn/s >> for the duration of the run. >> >> So that's bench-write reporting 3.2 GB/s with iostat showing *nothing* >> happening, while dd writes 226 MB/s and iostat lights up. Am I >> misunderstanding what rbd-bench is supposed to do? >> >> Thanks, >> -Emile >> _______________________________________________ >> ceph-users mailing list >> ceph-users@lists.ceph.com >> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com >> _______________________________________________ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
