Re: [ceph-users] What do you use to benchmark your rgw?

2018-04-05 Thread Thomas Bennett 
Hi Mathew,

We approached the problem by first running swift-bench for performance
tuning and configuration. Since it was the easiest to get up and running
and test the gateway.

Then we wrote a python script using python boto and python futures to model
our usecase and test s3.

We found the most effective performance improvement was to place the
*.rgw.buckets.index’s on nvme backed osds.

Cheers,
Tom

On Wed, 28 Mar 2018 at 11:18, Matthew Vernon  wrote:

> Hi,
>
> What are people here using to benchmark their S3 service (i.e. the rgw)?
> rados bench is great for some things, but doesn't tell me about what
> performance I can get from my rgws.
>
> It seems that there used to be rest-bench, but that isn't in Jewel
> AFAICT; I had a bit of a look at cosbench but it looks fiddly to set up
> and a bit under-maintained (the most recent version doesn't work out of
> the box, and the PR to fix that has been languishing for a while).
>
> This doesn't seem like an unusual thing to want to do, so I'd like to
> know what other ceph folk are using (and, if you like, the numbers you
> get from the benchmarkers)...?
>
> Thanks,
>
> Matthew
>
>
> --
>  The Wellcome Sanger Institute is operated by Genome Research
>  Limited, a charity registered in England with number 1021457 and a
>  company registered in England with number 2742969, whose registered
>  office is 215 Euston Road, London, NW1 2BE
> 
> .
> ___
> ceph-users mailing list
> ceph-users@lists.ceph.com
> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
>
-- 
Thomas Bennett

SKA South Africa
Science Processing Team

Office: +27 21 5067341
Mobile: +27 79 5237105
___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

Re: [ceph-users] What do you use to benchmark your rgw?

2018-04-03 Thread Mohamad Gebai 

On 03/28/2018 11:11 AM, Mark Nelson wrote:
> Personally I usually use a modified version of Mark Seger's getput
> tool here:
>
> https://github.com/markhpc/getput/tree/wip-fix-timing
>
> The difference between this version and upstream is primarily to make
> getput more accurate/useful when using something like CBT for
> orchestration instead of the included orchestration wrapper (gpsuite).
>
> CBT can use this version of getput and run relatively accurate
> mutli-client tests without requiring quite as much setup as cosbench. 
> Having said that, many folks have used cosbench effectively and I
> suspect that might be a good option for many people.  I'm not sure how
> much development is happening these days, I think the primary author
> may no longer be working on the project.
>

AFAIK the project is still alive. Adding Mark.

Mohamad


> Mark
>
> On 03/28/2018 09:21 AM, David Byte wrote:
>> I use cosbench (the last rc works well enough). I can get multiple
>> GB/s from my 6 node cluster with 2 RGWs.
>>
>> David Byte
>> Sr. Technical Strategist
>> IHV Alliances and Embedded
>> SUSE
>>
>> Sent from my iPhone. Typos are Apple's fault.
>>
>> On Mar 28, 2018, at 5:26 AM, Janne Johansson > > wrote:
>>
>>> s3cmd and cli version of cyberduck to test it end-to-end using
>>> parallelism if possible.
>>>
>>> Getting some 100MB/s at most, from 500km distance over https against
>>> 5*radosgw behind HAProxy.
>>>
>>>
>>> 2018-03-28 11:17 GMT+02:00 Matthew Vernon >> >:
>>>
>>>     Hi,
>>>
>>>     What are people here using to benchmark their S3 service (i.e.
>>>     the rgw)?
>>>     rados bench is great for some things, but doesn't tell me about
>>> what
>>>     performance I can get from my rgws.
>>>
>>>     It seems that there used to be rest-bench, but that isn't in Jewel
>>>     AFAICT; I had a bit of a look at cosbench but it looks fiddly to
>>>     set up
>>>     and a bit under-maintained (the most recent version doesn't work
>>>     out of
>>>     the box, and the PR to fix that has been languishing for a while).
>>>
>>>     This doesn't seem like an unusual thing to want to do, so I'd
>>> like to
>>>     know what other ceph folk are using (and, if you like, the
>>>     numbers you
>>>     get from the benchmarkers)...?
>>>
>>>     Thanks,
>>>
>>>     Matthew
>>>
>>>
>>>     --
>>>      The Wellcome Sanger Institute is operated by Genome Research
>>>      Limited, a charity registered in England with number 1021457 and a
>>>      company registered in England with number 2742969, whose
>>> registered
>>>      office is 215 Euston Road, London, NW1 2BE.
>>>     ___
>>>     ceph-users mailing list
>>>     ceph-users@lists.ceph.com 
>>>     http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
>>>     
>>>
>>>
>>>
>>>
>>> -- 
>>> May the most significant bit of your life be positive.
>>> ___
>>> ceph-users mailing list
>>> ceph-users@lists.ceph.com 
>>> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
>>
>>
>> ___
>> ceph-users mailing list
>> ceph-users@lists.ceph.com
>> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
>
> ___
> ceph-users mailing list
> ceph-users@lists.ceph.com
> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

Re: [ceph-users] What do you use to benchmark your rgw?

2018-03-28 Thread Janne Johansson 
2018-03-28 16:21 GMT+02:00 David Byte :

> I use cosbench (the last rc works well enough). I can get multiple GB/s
> from my 6 node cluster with 2 RGWs.
>
>
> To add info to this, it's not unexpectedly low for us, we know the
S3+https layer added latencies,
and it is EC pools on cheap+large spin disks optimized for cheap
almost-passive storage.

We get something close to 50% of what rados-bench gives us at the back end
of the RGWs,
that might be a more interesting value, since that bench doesn't have to do
any http chunking
nor MD5 sum the incoming files and so on.


> s3cmd and cli version of cyberduck to test it end-to-end using parallelism
> if possible.
>
> Getting some 100MB/s at most, from 500km distance over https against
> 5*radosgw behind HAProxy.
>
>
> 2018-03-28 11:17 GMT+02:00 Matthew Vernon :
>
>> Hi,
>>
>> What are people here using to benchmark their S3 service (i.e. the rgw)?
>> rados bench is great for some things, but doesn't tell me about what
>> performance I can get from my rgws.
>
>
-- 
May the most significant bit of your life be positive.
___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

Re: [ceph-users] What do you use to benchmark your rgw?

2018-03-28 Thread Mark Nelson 
Personally I usually use a modified version of Mark Seger's getput tool 
here:


https://github.com/markhpc/getput/tree/wip-fix-timing

The difference between this version and upstream is primarily to make 
getput more accurate/useful when using something like CBT for 
orchestration instead of the included orchestration wrapper (gpsuite).


CBT can use this version of getput and run relatively accurate 
mutli-client tests without requiring quite as much setup as cosbench.  
Having said that, many folks have used cosbench effectively and I 
suspect that might be a good option for many people.  I'm not sure how 
much development is happening these days, I think the primary author may 
no longer be working on the project.


Mark

On 03/28/2018 09:21 AM, David Byte wrote:
I use cosbench (the last rc works well enough). I can get multiple 
GB/s from my 6 node cluster with 2 RGWs.


David Byte
Sr. Technical Strategist
IHV Alliances and Embedded
SUSE

Sent from my iPhone. Typos are Apple's fault.

On Mar 28, 2018, at 5:26 AM, Janne Johansson > wrote:


s3cmd and cli version of cyberduck to test it end-to-end using 
parallelism if possible.


Getting some 100MB/s at most, from 500km distance over https against 
5*radosgw behind HAProxy.



2018-03-28 11:17 GMT+02:00 Matthew Vernon >:


Hi,

What are people here using to benchmark their S3 service (i.e.
the rgw)?
rados bench is great for some things, but doesn't tell me about what
performance I can get from my rgws.

It seems that there used to be rest-bench, but that isn't in Jewel
AFAICT; I had a bit of a look at cosbench but it looks fiddly to
set up
and a bit under-maintained (the most recent version doesn't work
out of
the box, and the PR to fix that has been languishing for a while).

This doesn't seem like an unusual thing to want to do, so I'd like to
know what other ceph folk are using (and, if you like, the
numbers you
get from the benchmarkers)...?

Thanks,

Matthew


--
 The Wellcome Sanger Institute is operated by Genome Research
 Limited, a charity registered in England with number 1021457 and a
 company registered in England with number 2742969, whose registered
 office is 215 Euston Road, London, NW1 2BE.
___
ceph-users mailing list
ceph-users@lists.ceph.com 
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com





--
May the most significant bit of your life be positive.
___
ceph-users mailing list
ceph-users@lists.ceph.com 
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com



___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com


___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

Re: [ceph-users] What do you use to benchmark your rgw?

2018-03-28 Thread David Byte 
I use cosbench (the last rc works well enough). I can get multiple GB/s from my 
6 node cluster with 2 RGWs.

David Byte
Sr. Technical Strategist
IHV Alliances and Embedded
SUSE

Sent from my iPhone. Typos are Apple's fault.

On Mar 28, 2018, at 5:26 AM, Janne Johansson 
> wrote:

s3cmd and cli version of cyberduck to test it end-to-end using parallelism if 
possible.

Getting some 100MB/s at most, from 500km distance over https against 5*radosgw 
behind HAProxy.


2018-03-28 11:17 GMT+02:00 Matthew Vernon 
>:
Hi,

What are people here using to benchmark their S3 service (i.e. the rgw)?
rados bench is great for some things, but doesn't tell me about what
performance I can get from my rgws.

It seems that there used to be rest-bench, but that isn't in Jewel
AFAICT; I had a bit of a look at cosbench but it looks fiddly to set up
and a bit under-maintained (the most recent version doesn't work out of
the box, and the PR to fix that has been languishing for a while).

This doesn't seem like an unusual thing to want to do, so I'd like to
know what other ceph folk are using (and, if you like, the numbers you
get from the benchmarkers)...?

Thanks,

Matthew


--
 The Wellcome Sanger Institute is operated by Genome Research
 Limited, a charity registered in England with number 1021457 and a
 company registered in England with number 2742969, whose registered
 office is 215 Euston Road, London, NW1 2BE.
___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com



--
May the most significant bit of your life be positive.
___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

Re: [ceph-users] What do you use to benchmark your rgw?

2018-03-28 Thread Janne Johansson 
s3cmd and cli version of cyberduck to test it end-to-end using parallelism
if possible.

Getting some 100MB/s at most, from 500km distance over https against
5*radosgw behind HAProxy.


2018-03-28 11:17 GMT+02:00 Matthew Vernon :

> Hi,
>
> What are people here using to benchmark their S3 service (i.e. the rgw)?
> rados bench is great for some things, but doesn't tell me about what
> performance I can get from my rgws.
>
> It seems that there used to be rest-bench, but that isn't in Jewel
> AFAICT; I had a bit of a look at cosbench but it looks fiddly to set up
> and a bit under-maintained (the most recent version doesn't work out of
> the box, and the PR to fix that has been languishing for a while).
>
> This doesn't seem like an unusual thing to want to do, so I'd like to
> know what other ceph folk are using (and, if you like, the numbers you
> get from the benchmarkers)...?
>
> Thanks,
>
> Matthew
>
>
> --
>  The Wellcome Sanger Institute is operated by Genome Research
>  Limited, a charity registered in England with number 1021457 and a
>  company registered in England with number 2742969, whose registered
>  office is 215 Euston Road, London, NW1 2BE.
> ___
> ceph-users mailing list
> ceph-users@lists.ceph.com
> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
>



-- 
May the most significant bit of your life be positive.
___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

[ceph-users] What do you use to benchmark your rgw?

2018-03-28 Thread Matthew Vernon 
Hi,

What are people here using to benchmark their S3 service (i.e. the rgw)?
rados bench is great for some things, but doesn't tell me about what
performance I can get from my rgws.

It seems that there used to be rest-bench, but that isn't in Jewel
AFAICT; I had a bit of a look at cosbench but it looks fiddly to set up
and a bit under-maintained (the most recent version doesn't work out of
the box, and the PR to fix that has been languishing for a while).

This doesn't seem like an unusual thing to want to do, so I'd like to
know what other ceph folk are using (and, if you like, the numbers you
get from the benchmarkers)...?

Thanks,

Matthew


-- 
 The Wellcome Sanger Institute is operated by Genome Research 
 Limited, a charity registered in England with number 1021457 and a 
 company registered in England with number 2742969, whose registered 
 office is 215 Euston Road, London, NW1 2BE. 
___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com