Re: [ceph-users] Help rebalancing OSD usage, Luminus 1.2.2

2018-02-21 Thread David Turner
s...@jumptrading.com> > *Date: *Tuesday, February 20, 2018 at 2:53 PM > *To: *David Turner <drakonst...@gmail.com> > *Cc: *Ceph Users <ceph-users@lists.ceph.com> > > > *Subject: *Re: [ceph-users] Help rebalancing OSD usage, Luminus 1.2.2 > > > > HI David

Re: [ceph-users] Help rebalancing OSD usage, Luminus 1.2.2

2018-02-21 Thread Bryan Stillwell
s <ceph-users@lists.ceph.com> Subject: Re: [ceph-users] Help rebalancing OSD usage, Luminus 1.2.2 HI David [Resending with smaller message size], I tried setting the OSDs down and that does clear the blocked requests momentarily but they just return back to the same state. Not sure how

Re: [ceph-users] Help rebalancing OSD usage, Luminus 1.2.2

2018-02-20 Thread Bryan Banister
...@gmail.com<mailto:icepic...@gmail.com>>; Ceph Users <ceph-users@lists.ceph.com<mailto:ceph-users@lists.ceph.com>> Subject: Re: [ceph-users] Help rebalancing OSD usage, Luminus 1.2.2 Note: External Email That sounds like a good next step.

Re: [ceph-users] Help rebalancing OSD usage, Luminus 1.2.2

2018-02-16 Thread David Turner
; > > > Should I try just marking OSDs with stuck requests down to see if that > will re-assert them? > > > > Thanks!! > > -Bryan > > > > *From:* David Turner [mailto:drakonst...@gmail.com] > *Sent:* Friday, February 16, 2018 2:51 PM > > > *To:* Bryan B

Re: [ceph-users] Help rebalancing OSD usage, Luminus 1.2.2

2018-02-16 Thread Bryan Banister
er <bbanis...@jumptrading.com> Cc: Bryan Stillwell <bstillw...@godaddy.com>; Janne Johansson <icepic...@gmail.com>; Ceph Users <ceph-users@lists.ceph.com> Subject: Re: [ceph-users] Help rebalancing OSD usage, Luminus 1.2.2 Note: External Email __

Re: [ceph-users] Help rebalancing OSD usage, Luminus 1.2.2

2018-02-16 Thread David Turner
d": 0, > > "scrubber.waiting_on": 0, > > "scrubber.waiting_on_whom": [] > > } > > }, > > { > > "name": "Started", > > "enter_time

Re: [ceph-users] Help rebalancing OSD usage, Luminus 1.2.2

2018-02-16 Thread Bryan Banister
t;Started", "enter_time": "2018-02-13 14:33:17.491148" } ], Sorry for all the hand holding, but how do I determine if I need to set an OSD as ‘down’ to fix the issues, and how does it go about re-asserting itself?

Re: [ceph-users] Help rebalancing OSD usage, Luminus 1.2.2

2018-02-16 Thread David Turner
t; At this point we do not know to proceed with recovery efforts. I tried > looking at the ceph docs and mail list archives but wasn’t able to > determine the right path forward here. > > > > Any help is appreciated, > > -Bryan > > > > > > *From:* Bryan Stillwell

Re: [ceph-users] Help rebalancing OSD usage, Luminus 1.2.2

2018-02-16 Thread Bryan Banister
, 2018 2:27 PM To: Bryan Banister <bbanis...@jumptrading.com>; Janne Johansson <icepic...@gmail.com> Cc: Ceph Users <ceph-users@lists.ceph.com> Subject: Re: [ceph-users] Help rebalancing OSD usage, Luminus 1.2.2 Note: External Email It may work fin

Re: [ceph-users] Help rebalancing OSD usage, Luminus 1.2.2

2018-02-13 Thread Bryan Stillwell
l.com> Cc: Ceph Users <ceph-users@lists.ceph.com> Subject: RE: [ceph-users] Help rebalancing OSD usage, Luminus 1.2.2 Thanks for the response Bryan! Would it be good to go ahead and do the increase up to 4096 PGs for thee pool given that it's only at 52% done with the rebalance backfil

Re: [ceph-users] Help rebalancing OSD usage, Luminus 1.2.2

2018-02-13 Thread Bryan Banister
] Sent: Tuesday, February 13, 2018 12:43 PM To: Bryan Banister <bbanis...@jumptrading.com>; Janne Johansson <icepic...@gmail.com> Cc: Ceph Users <ceph-users@lists.ceph.com> Subject: Re: [ceph-users] Help rebalancing OSD usage, Luminus 1.2.2 No

Re: [ceph-users] Help rebalancing OSD usage, Luminus 1.2.2

2018-02-13 Thread Bryan Stillwell
ry 12, 2018 at 2:19 PM To: Janne Johansson <icepic...@gmail.com> Cc: Ceph Users <ceph-users@lists.ceph.com> Subject: Re: [ceph-users] Help rebalancing OSD usage, Luminus 1.2.2 Hi Janne and others,   We used the “ceph osd reweight-by-utilization “ command to move a small amount of data off o

Re: [ceph-users] Help rebalancing OSD usage, Luminus 1.2.2

2018-02-12 Thread Bryan Banister
[mailto:icepic...@gmail.com] Sent: Wednesday, January 31, 2018 9:34 AM To: Bryan Banister <bbanis...@jumptrading.com> Cc: Ceph Users <ceph-users@lists.ceph.com> Subject: Re: [ceph-users] Help rebalancing OSD usage, Luminus 1.2.2 Note: External Email 2018-01-31

Re: [ceph-users] Help rebalancing OSD usage, Luminus 1.2.2

2018-01-31 Thread Janne Johansson
2018-01-31 15:58 GMT+01:00 Bryan Banister : > > > > Given that this will move data around (I think), should we increase the > pg_num and pgp_num first and then see how it looks? > > > I guess adding pgs and pgps will move stuff around too, but if the PGCALC formula

Re: [ceph-users] Help rebalancing OSD usage, Luminus 1.2.2

2018-01-31 Thread Bryan Banister
m and pgp_num first and then see how it looks? Thanks, -Bryan From: Janne Johansson [mailto:icepic...@gmail.com] Sent: Wednesday, January 31, 2018 7:53 AM To: Bryan Banister <bbanis...@jumptrading.com> Cc: Ceph Users <ceph-users@lists.ceph.com> Subject: Re: [ceph-users] Help rebalan

Re: [ceph-users] Help rebalancing OSD usage, Luminus 1.2.2

2018-01-31 Thread Janne Johansson
2018-01-30 17:24 GMT+01:00 Bryan Banister : > Hi all, > > > > We are still very new to running a Ceph cluster and have run a RGW cluster > for a while now (6-ish mo), it mainly holds large DB backups (Write once, > read once, delete after N days). The system is now

[ceph-users] Help rebalancing OSD usage, Luminus 1.2.2

2018-01-30 Thread Bryan Banister
Hi all, We are still very new to running a Ceph cluster and have run a RGW cluster for a while now (6-ish mo), it mainly holds large DB backups (Write once, read once, delete after N days). The system is now warning us about an OSD that is near_full and so we went to look at the usage across