Re: [ceph-users] Data distribution question

D1Q=cqFccwnwHGRorPuRWs36Dw=4MlyqfSU36xG5-kC0nVslDEynoWLQUUf5xCScKrouvU=u3DTgcPLZDP6XwXpcmvDKsNEPswrgE9ICvhnQzrZ1WY=> .. Dan (Apologies for terseness, I'm mobile) On Tue, 30 Apr 2019, 20:02 Shain Miley, <mailto:smi...@npr.org>> wrote: Here is the per pool pg_num info: 'data' pg_num 6

Re: [ceph-users] Data distribution question

04/30/2019 06:34 PM, Shain Miley wrote: Hi, We have a cluster with 235 osd's running version 12.2.11 with a combination of 4 and 6 TB drives. The data distribution across osd's varies from 52% to 94%. I have been trying to figure out how to get this a bit more balanced as we are running into '

[ceph-users] Data distribution question

the data more uniformly? Thanks in advance, Shain -- NPR | Shain Miley | Manager of Infrastructure, Digital Media | smi...@npr.org | 202.513.3649 ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

Re: [ceph-users] crush map has straw_calc_version=0 and legacy tunables on luminous

let me know so I don’t end up running the cluster in a sub-optimal state for no reason. Thanks, Shain -- Shain Miley | Manager of Systems and Infrastructure, Digital Media | smi...@npr.org | 202.513.3649 From: ceph-users on behalf of Shain Miley Date: Monday, February 4, 2019 at 3:03 PM

[ceph-users] crush map has straw_calc_version=0 and legacy tunables on luminous

to the cluster. Thanks in advance, Shain -- Shain Miley | Manager of Systems and Infrastructure, Digital Media | smi...@npr.org | 202.513.3649 ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

Re: [ceph-users] Problem replacing osd with ceph-deploy

f this 'backfillfull' errors I am seeing. I am assuming this isn't normal...however this would be the first bluestore osd added to this cluster...so I am not really sure. Thanks in advance, Shain -- NPR | Shain Miley | Manager of Infrastructure, Digital Me

[ceph-users] Problem replacing osd with ceph-deploy

'backfillfull' errors I am seeing. I am assuming this isn't normal...however this would be the first bluestore osd added to this cluster...so I am not really sure. Thanks in advance, Shain -- NPR | Shain Miley | Manager of Infrastructure, Digi

Re: [ceph-users] Ceph Jewel and Ubuntu 16.04

: On 16/04/18 18:32, Shain Miley wrote: Hello, We are currently running Ceph Jewel (10.2.10) on Ubuntu 14.04 in production.  We have been running into a kernel panic bug off an on for a while and I am starting to look into upgrading as a possible solution.  We are currently running version 4.4.0-31

[ceph-users] Ceph Jewel and Ubuntu 16.04

both Ubuntu and Ceph..in which order should the upgrade occur? Thanks in advance for any insight that you are able to provide me on this issue! Shain -- NPR | Shain Miley | Manager of Infrastructure, Digital Media | smi...@npr.org | 202.513.3649

Re: [ceph-users] Jewel upgrade and feature set mismatch

osd161 3.edeaca14 npr_archive_orpheus_img.rbd 871904'299183126872064 watch Do you think those 4 write operations are enough to make me think twice about a reboot? Thanks again, Shain On 05/24/2017 08:31 AM, Ilya Dryomov wrote: On Wed, May 24, 2017 at 1:47 PM, Shain Miley <

[ceph-users] Jewel upgrade and feature set mismatch

Hello, We just upgraded from Hammer to Jewel, and after the cluster once again reported a healthy state I set the crush tunables to ‘optimal’ (from legacy). 12 hours later and the cluster is almost done with the pg remapping under the new rules. The issue I am having is the server where we

Re: [ceph-users] Hammer to Jewel upgrade questions

- Le 16 Mai 17, à 20:43, Shain Miley <smi...@npr.org <mailto:smi...@npr.org>> a écrit : Hello, I am going to be upgrading our production Ceph cluster from Hammer/Ubuntu 14.04 to Jewel/Ubuntu 16.04 and I wanted to ask a question and sani

[ceph-users] Hammer to Jewel upgrade questions

The two questions I have are: 1)Am I missing anything from the steps above...based on prior experiences performing upgrades of this kind? 2)Should I upgrade to Ubuntu 16.04 first and then upgrade Ceph...or vice versa? Thanks in advance, Shain -- NPR | Shain Miley | Manager of Infrastructure

Re: [ceph-users] active+clean+inconsistent and pg repair

e next long test should hopefully work fine: > # smartctl -t long /dev/sdX > > If not repeat the above with new found defekt lba. > > Ive done this three time successfully - but not with an error on a primary pg. > > After that you can start the osd with > > # systemctl

Re: [ceph-users] active+clean+inconsistent and pg repair

Brian, Never mind...looking back though some older emails I do see an indication of a problem with that drive. I will fail out the osd and replace the drive. Thanks again for the help, Shian On 03/17/2017 03:08 PM, Shain Miley wrote: This sender failed our fraud detection checks

Re: [ceph-users] active+clean+inconsistent and pg repair

to detect with a simple list of file size) and differing attributes ("attr -l", but the error logs are usually precise in identifying the problematic PG copy). On Fri, Mar 17, 2017 at 8:16 AM, Shain Miley <smi...@npr.org <mailto:smi...@npr.org>> wrote: Hello, Ce

[ceph-users] active+clean+inconsistent and pg repair

indicates there are times where running this command is less safe than others...and I would like to be sure before I do so. Thanks, Shain -- NPR | Shain Miley | Manager of Infrastructure, Digital Media |smi...@npr.org | 202.513.3649 ___ ceph-users

Re: [ceph-users] Ceph-deploy and git.ceph.com

I ended up using a newer version of ceph-deploy and things went more smoothly after that. Thanks again to everyone for all the help! Shain > On Mar 16, 2017, at 10:29 AM, Shain Miley <smi...@npr.org> wrote: > > This sender failed our fraud detection checks and may not be

Re: [ceph-users] noout, nodown and blocked requests

ubleshooting/troubleshooting-osd/ > "STOPPING W/OUT REBALANCING". > > > Your clients are hanging because of the no down flag > > > See this blog for no-out, no-down flags experiements > > https://www.sebastien-han.fr/blog/2013/04/17/some-ceph-experimen

Re: [ceph-users] Unable to boot OS on cluster node

coln > >> On Mar 10, 2017, at 11:33 AM, Shain Miley <smi...@npr.org> wrote: >> >> Hello, >> >> We had an issue with one of our Dell 720xd servers and now the raid card >> cannot seem to boot from the Ubuntu OS drive volume. >> >> I woul

Re: [ceph-users] Ceph-deploy and git.ceph.com

to execute command: apt-get -q update Does anyone know if there is still an issue ongoing issue….or is this something that should be working at this point? Thanks again, Shain > On Mar 15, 2017, at 2:08 PM, Shain Miley <smi...@npr.org> wrote: > > This sender failed our fraud d

Re: [ceph-users] Ceph-deploy and git.ceph.com

t; Nope, Jewel is LTS, Kraken is not. >>> >>> Kraken will only receive updates until the next stable release. Jewel >>> will receive updates for longer. >>> >>> John >>> >>>> On Thu, Mar 16, 2017 at 1:48 AM, Shain Miley <smi...@npr.

Re: [ceph-users] Ceph-deploy and git.ceph.com

ogle.com/+Cephstorage/posts/HuCaTi7Egg3> > > On Thu, Mar 16, 2017 at 1:05 AM, Shain Miley <smi...@npr.org > <mailto:smi...@npr.org>> wrote: > > Hello, > > I am trying to deploy ceph to a new server using ceph-deply which I have > > done in the past many tim

[ceph-users] Ceph-deploy and git.ceph.com

Hello, I am trying to deploy ceph to a new server using ceph-deply which I have done in the past many times without issue. Right now I am seeing a timeout trying to connect to git.ceph.com : [hqosd6][INFO ] Running command: env DEBIAN_FRONTEND=noninteractive apt-get -q

[ceph-users] noout, nodown and blocked requests

Hello, One of the nodes in our 14 node cluster is offline and before I totally commit to fully removing the node from the cluster (there is a chance I can get the node back in working order in the next few days) I would like to run the cluster with that single node out for a few days.

Re: [ceph-users] Ubuntu latest ceph-deploy fails to install hammer

Alex, I ran into this issue yesterday as well. I ended up just installing ceph via apt-get locally on the new server. I have not been able to get an actual osd added to the cluster at this point though (see my emails over the last 2 days or so). Please let me know if you end up able to add an

Re: [ceph-users] Ceph-deploy not creating osd's

is not present in either the current crush map or the 'osd tree'. Thanks, Shain > On Sep 8, 2016, at 10:27 PM, Shain Miley <smi...@npr.org> wrote: > > I ended up starting from scratch and doing a purge and purgedata on that host > using ceph-deploy, after that things seemed to go bette

Re: [ceph-users] Ceph-deploy not creating osd's

map: http://pastebin.com/PMk3xZ0a as you can see from the entry for osd number 108 (the last osd to be added to the cluster)...the crush map does not contain a host entry for hqosd10...which is the host for osd #108. Any ideas on how to resolve this? Thanks, Shain On 9/8/16 2:20 PM, Shain

[ceph-users] Ceph-deploy not creating osd's

Hello, I am trying to use ceph-deploy to add some new osd's to our cluster. I have used this method over the last few years to add all of our 107 osd's and things have seemed to work quite well. One difference this time is that we are going to use a pci nvme card to journal the 16 disks in

Re: [ceph-users] 2 osd failures

wrote: Hello, Too late I see, but still... On Tue, 6 Sep 2016 22:17:05 -0400 Shain Miley wrote: Hello, It looks like we had 2 osd's fail at some point earlier today, here is the current status of the cluster: You will really want to find out how and why that happened, because while

Re: [ceph-users] 2 osd failures

I restarted both osd daemons and things are back to normal. I'm not sure why they failed in the first place but I'll keep looking. Thanks! Shain Sent from my iPhone > On Sep 6, 2016, at 10:39 PM, lyt_yudi <lyt_y...@icloud.com> wrote: > > hi, > >> 在 2016年9月7日，

[ceph-users] 2 osd failures

Hello, It looks like we had 2 osd's fail at some point earlier today, here is the current status of the cluster: root@rbd1:~# ceph -s cluster 504b5794-34bd-44e7-a8c3-0494cf800c23 health HEALTH_WARN 2 pgs backfill 5 pgs backfill_toofull 69 pgs

[ceph-users] Ceph 0.94.8 Hammer upgrade on Ubuntu 14.04

is, at this point for example if I run 'ceph -v' it will tell me I'm running 0.94.7, even though from my perspective I should be running 0.94.8. Thanks, Shain -- NPR | Shain Miley | Manager of Infrastructure, Digital Media | smi...@npr.org | 202.513.3649

Re: [ceph-users] Ceph 0.94.8 Hammer upgrade on Ubuntu 14.04

s prohibited. ---- *From:* Shain Miley [smi...@npr.org] *Sent:* Tuesday, August 30, 2016 11:12 AM *To:* Brian ::; David Turner *Cc:* ceph-users@lists.ceph.com *Subject:* Re: [ceph-users] Ceph 0.94.8 Hammer upgra

Re: [ceph-users] Ceph 0.94.8 Hammer upgrade on Ubuntu 14.04

h-users [ceph-users-boun...@lists.ceph.com <mailto:ceph-users-boun...@lists.ceph.com>] on behalf of Shain Miley [smi...@npr.org <mailto:smi...@npr.org>] Sent: Tuesday, August 30, 2016 10:59 AM To: ceph-users@lists.ceph.com <mailto:ceph-users@lists.ceph.com> Su

Re: [ceph-users] Ceph 0.94.8 Hammer upgrade on Ubuntu 14.04

run 'ceph -v' it will tell me I'm running 0.94.7, even though from my perspective I should be running 0.94.8. Thanks, Shain -- NPR | Shain Miley | Manager of Infrastructure, Digital Media | smi...@npr.org | 202.513.3649 ___ ceph-users mailing list

[ceph-users] Ceph 0.94.8 Hammer upgrade on Ubuntu 14.04

is, at this point for example if I run 'ceph -v' it will tell me I'm running 0.94.7, even though from my perspective I should be running 0.94.8. Thanks, Shain -- NPR | Shain Miley | Manager of Infrastructure, Digital Media | smi...@npr.org | 202.513.3649

Re: [ceph-users] rbd-mirror questions

t;> >>> Op 4 augustus 2016 om 18:17 schreef Shain Miley <smi...@npr.org>: >>> >>> >>> Hello, >>> >>> I am thinking about setting up a second Ceph cluster in the near future, >>> and I was wondering about the cur

[ceph-users] rbd-mirror questions

have some rather large rbd images at this point..several in the 90TB range...would there be any concern using rbd-mirror given the size of our images? Thanks, Shain -- NPR | Shain Miley | Manager of Infrastructure, Digital Media | smi...@npr.org | 202.513.3649

Re: [ceph-users] is it time already to move from hammer to jewel?

-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com -- NPR | Shain Miley | Manager of Infrastructure, Digital Media | smi...@npr.org | 202.513.3649 ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com

[ceph-users] Ceph upgrade questions

upgrade to hammer then move onto ubuntu upgrades from there? Thanks in advance for any advice, Shain *NPR* | Shain Miley | Manager of Infrastructure, Digital Media | smi...@npr.org | 202.513.3649 ___ ceph-users mailing list ceph-users@lists.ceph.com

Re: [ceph-users] rbd directory listing performance issues

instead of samba to see if that provides better stability moving forward. Thanks, Shain Shain Miley | Manager of Systems and Infrastructure, Digital Media | smi...@npr.org | 202.513.3649 From: Christian Balzer [ch...@gol.com] Sent: Monday, January 12, 2015

Re: [ceph-users] reset osd perf counters

...@gregs42.com wrote: perf reset on the admin socket. I'm not sure what version it went in to; you can check the release logs if it doesn't work on whatever you have installed. :) -Greg On Mon, Jan 12, 2015 at 2:26 PM, Shain Miley smi...@npr.org wrote: Is there a way to 'reset' the osd

Re: [ceph-users] rbd directory listing performance issues

Hi, I am just wondering if anyone has any thoughts on the questions below...I would like to order some additional hardware ASAP...and the order that I place may change depending on the feedback that I receive. Thanks again, Shain Sent from my iPhone On Jan 9, 2015, at 2:45 PM, Shain Miley

[ceph-users] reset osd perf counters

Is there a way to 'reset' the osd perf counters? The numbers for osd 73 though osd 83 look really high compared to the rest of the numbers I see here. I was wondering if I could clear the counters out, so that I have a fresh set of data to work with. root@cephmount1:/var/log/samba# ceph

Re: [ceph-users] rbd directory listing performance issues

directories at this point. Also...one thing I just noticed is that the 'ls |wc' returns right away...even in cases when right after that I do an 'ls -l' and it takes a while. Thanks, Shain Shain Miley | Manager of Systems and Infrastructure, Digital Media | smi...@npr.org | 202.513.3649

[ceph-users] rbd directory listing performance issues

a a fix to resolve this issue? Any suggestions would be greatly appreciated. Thanks, Shain Shain Miley | Manager of Systems and Infrastructure, Digital Media | smi...@npr.org | 202.513.3649 ___ ceph-users mailing list ceph-users@lists.ceph.com http

Re: [ceph-users] rbd directory listing performance issues

anything for us. On Tue, Jan 6, 2015 at 12:18 PM, Shain Miley smi...@npr.org wrote: It does seem like the entries get cached for a certain period of time. Here is the memory listing for the rbd client server: root@cephmount1:~# free -m total used free shared

Re: [ceph-users] rbd directory listing performance issues

at the graphs in Calamari, we see around 20ms latency (await) for our OSD's however there are lots of times where we see (via the graphs) spikes of 250ms to 400ms as well. Thanks again, Shain Shain Miley | Manager of Systems and Infrastructure, Digital Media | smi...@npr.org | 202.513.3649

Re: [ceph-users] rbd directory listing performance issues

Robert, xfs on the rbd image as well: /dev/rbd0 on /mnt/ceph-block-device-archive type xfs (rw) However looking at the mount options...it does not look like I've enabled anything special in terms of mount options. Thanks, Shain Shain Miley | Manager of Systems and Infrastructure, Digital

[ceph-users] Calamari install issues

.npr.org: 'ceph.get_heartbeats' is not available. hqceph2.npr.org: 'ceph.get_heartbeats' is not available. hqosd1.npr.org: 'ceph.get_heartbeats' is not available. Any help trying to move forward would be great! Thanks in advance, Shain -- _NPR | Shain Miley| Manager of Systems

Re: [ceph-users] Calamari install issues

': to_bytes(get_latest_graphite(df_path('total_space'))), 'free_bytes': to_bytes(get_latest_graphite(df_path('total_avail'))) Michael Kuriger mk7...@yp.com 818-649-7235 MikeKuriger (IM) From: Shain Miley smi...@npr.org mailto:smi...@npr.org Date: Friday, November 21, 2014 at 8

Re: [ceph-users] osd down

--zap-disk should take care of you. On Thu, Nov 6, 2014 at 5:06 PM, Shain Miley smi...@npr.orgmailto:smi...@npr.org wrote: I tried restarting all the osd's on that node, osd.70 was the only ceph process that did not come back online. There is nothing in the ceph-osd log for osd.70. However I do

Re: [ceph-users] osd down

): xfs_log_force: error 5 returned. Does anyone have any suggestions on how I might be able to get this HD back in the cluster (or whether or not it is worth even trying). Thanks, Shain Shain Miley | Manager of Systems and Infrastructure, Digital Media | smi...@npr.org | 202.513.3649

Re: [ceph-users] cephfs survey results

definitely interested in hearing feedback on this strategy, and on peoples' experiences with giant so far... sage ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com -- Shain Miley | Manager

Re: [ceph-users] Ceph / Dell hardware recommendation

be looking into the new ceph 'ssd tiering' options, or a little bit less dense Dell nodes for osd's using ssd's for the journals, in order to maximize performance. Shain Shain Miley | Manager of Systems and Infrastructure, Digital Media | smi...@npr.org | 202.513.3649

Re: [ceph-users] [Rados Bench Result] Maximum bandwidth only 380MB/sec with 10Gb Network.

test3 30 write' Shain Shain Miley | Manager of Systems and Infrastructure, Digital Media | smi...@npr.org | 202.513.3649 From: ceph-users-boun...@lists.ceph.com [ceph-users-boun...@lists.ceph.com] on behalf of Kuo Hugo [tonyt...@gmail.com] Sent

[ceph-users] rbd ext4 vs xfs

was just wondering, if there is was any major known pros or cons when it comes to using xfs instead of ext4 (or vise versa) for the rbd filesystem. Thanks in advance, Shain Shain Miley | Manager of Systems and Infrastructure, Digital Media | smi...@npr.org | 202.513.3649

Re: [ceph-users] Full OSD with 29% free

Bryan, We are setting up a cluster using xfs and have been a bit concerned about running into similar issues to the ones you described below. I am just wondering if you came across any potential downsides to using a 2K block size with xfs on your osd's. Thanks, Shain Shain Miley | Manager

Re: [ceph-users] Radosgw and large files

root 10737418240 Oct 29 10:36 root.img Thanks again, Shain Shain Miley | Manager of Systems and Infrastructure, Digital Media | smi...@npr.org | 202.513.3649 From: Mark Kirkwood [mark.kirkw...@catalyst.net.nz] Sent: Wednesday, October 30, 2013 8:29 PM

Re: [ceph-users] Radosgw and large files

PS...I tested the cancel script it worked like a charm (the script found quite a few partial files that needed to be removed). Thanks again, Shain Shain Miley | Manager of Systems and Infrastructure, Digital Media | smi...@npr.org | 202.513.3649 From

Re: [ceph-users] Radosgw and large files

Mark, Thanks a lot for the info. Hopefully this will resolve our issues going forward! Shain Shain Miley | Manager of Systems and Infrastructure, Digital Media | smi...@npr.org | 202.513.3649 From: Mark Kirkwood [mark.kirkw...@catalyst.net.nz] Sent

[ceph-users] Radosgw and large files

(and it appears that they do as files larger than 15MB gets split into multiple pieces) however I can only upload files smaller than 5GB up to this point. I am using 0.67.4 and Ubuntu 12.04. Thanks in advance, Shain Shain Miley | Manager of Systems and Infrastructure, Digital Media | smi

Re: [ceph-users] Radosgw and large files

this working soon. Thanks agin for the help already. Shain Shain Miley | Manager of Systems and Infrastructure, Digital Media | smi...@npr.org | 202.513.3649 From: Derek Yarnell [de...@umiacs.umd.edu] Sent: Saturday, October 26, 2013 6:46 PM To: Shain Miley

Re: [ceph-users] Radosgw and large files

ceph.com related to radosgw? Thanks again, Shain Shain Miley | Manager of Systems and Infrastructure, Digital Media | smi...@npr.org | 202.513.3649 From: ceph-users-boun...@lists.ceph.com [ceph-users-boun...@lists.ceph.com] on behalf of Shain Miley

Re: [ceph-users] Radosgw and large files

for testing. I have tested uploading small files that do not have an extension and I do not seem to be running into the same issue. This may be a bug of some sort so I am going to open a bug report when I get a chance. Thanks, Shain Shain Miley | Manager of Systems and Infrastructure, Digital Media

Re: [ceph-users] OSD journal size

Alfredo, Do you know what version of ceph-deploy has this updated functionality I just updated to 1.2.7 and it does not appear to include it. Thanks, Shain Shain Miley | Manager of Systems and Infrastructure, Digital Media | smi...@npr.org | 202.513.3649

Re: [ceph-users] OSD journal size

...or am I doing something wrong? Thanks, Shain Shain Miley | Manager of Systems and Infrastructure, Digital Media | smi...@npr.org | 202.513.3649 From: ceph-users-boun...@lists.ceph.com [ceph-users-boun...@lists.ceph.com] on behalf of Shain Miley [smi

[ceph-users] OSD journal size

/disk/by-path' for their OSD's. Does ceph-deploy currently support setting up OSD's using this method? Thanks, Shain Shain Miley | Manager of Systems and Infrastructure, Digital Media | smi...@npr.org | 202.513.3649 ___ ceph-users mailing list ceph

Re: [ceph-users] OSD journal size

Alfredo, Thanks a lot for the info. I'll make sure I have an updated version of ceph-deploy and give it another shot. Shain Shain Miley | Manager of Systems and Infrastructure, Digital Media | smi...@npr.org | 202.513.3649 From: Alfredo Deza

Re: [ceph-users] snapshots on CephFS

Shain Shain Miley | Manager of Systems and Infrastructure, Digital Media | smi...@npr.org | 202.513.3649 From: ceph-users-boun...@lists.ceph.com [ceph-users-boun...@lists.ceph.com] on behalf of Kasper Dieter [dieter.kas...@ts.fujitsu.com] Sent: Wednesday

Re: [ceph-users] cephforum.com

again (on both the ML and irc), something like this could help eliminate a bit of the unnecessary traffic as well as help new users get answers to some of their most basic questions right away. Just a thought. Shain Shain Miley | Manager of Systems and Infrastructure, Digital Media | smi

[ceph-users] Hardware recommendations

Good morning, I am in the process of deciding what hardware we are going to purchase for our new ceph based storage cluster. I have been informed that I must submit my purchase needs by the end of this week in order to meet our FY13 budget requirements (which does not leave me much time).

Re: [ceph-users] Hardware recommendations

Martin, Thank you very much for sharing your insight on hardware options. This will be very useful for us going forward. Shain Shain Miley | Manager of Systems and Infrastructure, Digital Media | smi...@npr.org | 202.513.3649 From: Martin B Nielsen [mar

[ceph-users] Radosgw auth and 0.65-1

Is there any issue with me trying to test radosgw on a node that is already being used as a mon and an osd? I may end up dropping back to the stable release...however I wanted to send a quick email in case there was an easy fix...or this was a bug of some sort. Thanks in advance, Shain Shain Miley

Re: [ceph-users] Radosgw auth and 0.65-1

Shain Miley | Manager of Systems and Infrastructure, Digital Media | smi...@npr.org | 202.513.3649 From: Gregory Farnum [g...@inktank.com] Sent: Tuesday, July 02, 2013 12:16 PM To: Shain Miley Cc: ceph-us...@ceph.com Subject: Re: [ceph-users] Radosgw