Hi sorry, i missed this mail. > During writes, does the CPU usage on your RadosGW node go way up? No, CPU stay the same & very low (< 10%)
When upload small files(300KB/file) over RadosGW: - using 1 process: upload bandwidth ~ 3MB/s - using 100 processes: upload bandwidth ~ 15MB/s When upload big files(3GB/file) over RadosGW: - using 1 process: upload bandwidth ~ 70MB/s (Therefore i don't upload big files using multi-processes any more :D) Maybe, RadosGW have a problem when write many smail files. Or it's a problem of CEPH when simultaneously write many smail files into a bucket, that already have millions files? On Wed, Sep 25, 2013 at 7:24 PM, Mark Nelson <mark.nel...@inktank.com>wrote: > On 09/25/2013 02:49 AM, Chu Duc Minh wrote: > >> I have a CEPH cluster with 9 nodes (6 data nodes & 3 mon/mds nodes) >> And i setup 4 separate nodes to test performance of Rados-GW: >> - 2 node run Rados-GW >> - 2 node run multi-process put file to [multi] Rados-GW >> >> Result: >> a) When i use 1 RadosGW node & 1 upload-node, speed upload = 50MB/s >> /upload-node, Rados-GW input/output speed = 50MB/s >> >> b) When i use 2 RadosGW node & 1 upload-node, speed upload = 50MB/s >> /upload-node; each RadosGW have input/output = 25MB/s ==> sum >> input/ouput of 2 Rados-GW = 50MB/s >> >> c) When i use 1 RadosGW node & 2 upload-node, speed upload = 25MB/s >> /upload-node ==> sum output of 2 upload-node = 50MB/s, RadosGW have >> input/output = 50MB/s >> >> d) When i use 2 RadosGW node & 2 upload-node, speed upload = 25MB/s >> /upload-node ==> sum output of 2 upload-node = 50MB/s; each RadosGW have >> input/output = 25MB/s ==> sum input/ouput of 2 Rados-GW = 50MB/s >> >> _*Problem*_: i can pass limit 50MB/s when put file over Rados-GW, >> >> regardless of the number Rados-GW nodes and upload-nodes. >> When i use this CEPH cluster over librados (openstack/kvm), i can easily >> achieve > 300MB/s >> >> I don't know why performance of RadosGW is so low. What's bottleneck? >> > > During writes, does the CPU usage on your RadosGW node go way up? > > If this is a test cluster, you might want to try the wip-6286 build from > our gitbuilder site. There is a fix that depending on the size of your > objects, could have a big impact on performance. We're currently > investigating some other radosgw performance issues as well, so stay tuned. > :) > > Mark > > > >> Thank you very much! >> >> >> >> >> ______________________________**_________________ >> ceph-users mailing list >> ceph-users@lists.ceph.com >> http://lists.ceph.com/**listinfo.cgi/ceph-users-ceph.**com<http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com> >> >> > ______________________________**_________________ > ceph-users mailing list > ceph-users@lists.ceph.com > http://lists.ceph.com/**listinfo.cgi/ceph-users-ceph.**com<http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com> >
_______________________________________________ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
