[ceph-users] New EC pool undersized

2015-03-04 Thread Kyle Hutson
Last night I blew away my previous ceph configuration (this environment is pre-production) and have 0.87.1 installed. I've manually edited the crushmap so it down looks like https://dpaste.de/OLEa I currently have 144 OSDs on 8 nodes. After increasing pg_num and pgp_num to a more suitable 1024

Re: [ceph-users] New EC pool undersized

2015-03-04 Thread Don Doerner
Oh duh… OK, then given a 4+4 erasure coding scheme, 14400/8 is 1800, so try 2048. -don- From: ceph-users [mailto:ceph-users-boun...@lists.ceph.com] On Behalf Of Don Doerner Sent: 04 March, 2015 12:14 To: Kyle Hutson; Ceph Users Subject: Re: [ceph-users] New EC pool undersized In this case

Re: [ceph-users] New EC pool undersized

2015-03-04 Thread Don Doerner
issue, I believe. -don- From: ceph-users [mailto:ceph-users-boun...@lists.ceph.com] On Behalf Of Don Doerner Sent: 04 March, 2015 12:49 To: Kyle Hutson Cc: ceph-users@lists.ceph.com Subject: Re: [ceph-users] New EC pool undersized Hmmm, I just struggled through this myself. How many racks do you

Re: [ceph-users] New EC pool undersized

2015-03-04 Thread Kyle Hutson
:* ceph-users [mailto:ceph-users-boun...@lists.ceph.com] *On Behalf Of *Don Doerner *Sent:* 04 March, 2015 12:49 *To:* Kyle Hutson *Cc:* ceph-users@lists.ceph.com *Subject:* Re: [ceph-users] New EC pool undersized Hmmm, I just struggled through this myself. How many racks do you have

Re: [ceph-users] New EC pool undersized

2015-03-04 Thread Kyle Hutson
That did it. 'step set_choose_tries 200' fixed the problem right away. Thanks Yann! On Wed, Mar 4, 2015 at 2:59 PM, Yann Dupont y...@objoo.org wrote: Le 04/03/2015 21:48, Don Doerner a écrit : Hmmm, I just struggled through this myself. How many racks do you have? If not more than 8,

Re: [ceph-users] New EC pool undersized

2015-03-04 Thread Kyle Hutson
, so try 2048. -don- *From:* ceph-users [mailto:ceph-users-boun...@lists.ceph.com] *On Behalf Of *Don Doerner *Sent:* 04 March, 2015 12:14 *To:* Kyle Hutson; Ceph Users *Subject:* Re: [ceph-users] New EC pool undersized In this case, that number means that there is not an OSD that can

Re: [ceph-users] New EC pool undersized

2015-03-04 Thread Don Doerner
12:43 To: Don Doerner Cc: Ceph Users Subject: Re: [ceph-users] New EC pool undersized It wouldn't let me simply change the pg_num, giving Error EEXIST: specified pg_num 2048 = current 8192 But that's not a big deal, I just deleted the pool and recreated with 'ceph osd pool create ec44pool 2048

Re: [ceph-users] New EC pool undersized

2015-03-04 Thread Yann Dupont
Le 04/03/2015 21:48, Don Doerner a écrit : Hmmm, I just struggled through this myself.How many racks do you have?If not more than 8, you might want to make your failure domain smaller?I.e., maybe host?That, at least, would allow you to debug the situation… -don- Hello, I think I

Re: [ceph-users] New EC pool undersized

2015-03-04 Thread Kyle Hutson
, 2015 12:43 *To:* Don Doerner *Cc:* Ceph Users *Subject:* Re: [ceph-users] New EC pool undersized It wouldn't let me simply change the pg_num, giving Error EEXIST: specified pg_num 2048 = current 8192 But that's not a big deal, I just deleted the pool and recreated with 'ceph osd

Re: [ceph-users] New EC pool undersized

2015-03-04 Thread Don Doerner
, 2015 13:15 To: Don Doerner Cc: ceph-users@lists.ceph.com Subject: Re: [ceph-users] New EC pool undersized So it sounds like I should figure out at 'how many nodes' do I need to increase pg_num to 4096, and again for 8192, and increase those incrementally when as I add more hosts, correct? On Wed