On 2018-06-12 01:01, Jialin Liu wrote:
> Hello Ceph Community,
>
> I used libradosstriper api to test the striping feature, it doesn't seem to
> improve the performance at all, can anyone advise what's wrong with my
> settings:
>
> The rados object store testbed at my center has
> osd: 48
> oss: 4
> monitor:2
> pg number: 1024
> replicated size: 3
>
> I have implemented a benchmark code [1] with libradosstriper api.
>
> I then used 1 processes 1 thread to do the test, by varying a few settings:
>
> * stripe count from 1 to 48,
> * and object size from 1MB to 128 MB (with stripe size 1MB, stripe size needs
> to be smaller than the rados object size),
> * and file size from 100MB to 1.6GB,
>
> The peak bandwidth among all tests is only 130MB/s, no difference in
> different tests.
>
> I suspect that the IO got serialized in the rados layer, with some uncertain
> evidence in the libradosstriper source code (note the for loop):
>> ...
>> ·Striper::file_to_extents(cct(), format.c_str(), &l, off, len, 0, extents);
>> ·FOR (vector<ObjectExtent>::iterator p = extents.begin(); p !=
>> extents.end(); ++p) {
>> r = m_ioCtx.aio_write(p->oid.name [2], rados_completion, oid_bl,
>> p->length, p->offset);
>> }
>> ...
Could you please correct me if I misused or misunderstood any things?
Thanks much.
Best,
Jialin
NERSC/LBNL
_______________________________________________
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
The striper source is correct, the for loop does aio writes so there is
no serialization blocking.
/Maged
Links:
------
[1]
https://github.com/NERSC/object-store/blob/master/tests/ceph/vpic_io/librados_test.c
[2] http://oid.name/_______________________________________________
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
