Hi Vadim,
Is every pool also using your custom crush_ruleset ("step chooseleaf firstn
0 type osd")?
Otherwise Ceph will use the default rule to replicate data on separate
hosts, which, in your case of a single host, cannot work.
You can check it with
ceph osd dump --format=json-pretty
And in case apply the rule with
ceph osd pool set <pool_name> crush_ruleset <rulesetId>
And you can check your custom rulseteid with
ceph osd crush dump
Hope this helps!
Regards,
Vincenzo.
2014-06-06 8:24 GMT+02:00 Vadim Kimlaychuk <[email protected]>:
> Michael, indeed I have pool size = 3. I changed it to 2. After that I
> have recompiled crush map to reflect different sizes of hard drives and put
> 1.0 to 1Tb drive and 0.75 for 750Gb.
>
>
>
> Now I have all my PG-s at status "active". It should be „active+clean“
> isn’t it ?
>
> I put object into the cluster and have
>
>
>
> health HEALTH_WARN 192 pgs stuck unclean; recovery 1/2 objects
> degraded (50.000%)
>
> monmap e1: 1 mons at {storage=172.16.3.2:6789/0}, election epoch 2,
> quorum 0 storage
>
> osdmap e19: 2 osds: 2 up, 2 in
>
> pgmap v42: 192 pgs, 3 pools, 414 bytes data, 1 objects
>
> 75584 kB used, 1619 GB / 1619 GB avail
>
> 1/2 objects degraded (50.000%)
>
> 192 active
>
>
>
> Does that mean object is stored at wrong place? Or set up is still
> incomplete?
>
>
>
> Thanks.
>
>
>
> -----Original Message-----
> From: ceph-users [mailto:[email protected]] On Behalf Of
> Jeremy Hanmer
> Sent: Thursday, June 05, 2014 9:41 PM
> To: [email protected]
> Subject: Re: [ceph-users] Hard drives of different sizes.
>
>
>
> You'll also want to change the crush weights of your OSDs to reflect the
> different sizes so that the smaller disks don't get filled up prematurely.
> See "weighting bucket items" here:
>
> http://ceph.com/docs/master/rados/operations/crush-map/
>
>
>
> On Thu, Jun 5, 2014 at 10:14 AM, Michael <[email protected]>
> wrote:
>
> > ceph osd dump | grep size
>
> >
>
> > Check that all pools are size 2, min size 2 or 1.
>
> >
>
> > If not you can change on the fly with:
>
> > ceph osd pool set #poolname size/min_size #size
>
> >
>
> > See docs http://ceph.com/docs/master/rados/operations/pools/ for
>
> > alterations to pool attributes.
>
> >
>
> > -Michael
>
> >
>
> >
>
> > On 05/06/2014 17:29, Vadim Kimlaychuk wrote:
>
> >>
>
> >> ____________________________
>
> >>
>
> >> I have
>
> >> osd pool default size = 2
>
> >> at my ceph.conf. Shouldn' it tell ceph to use 2 OSDs ? Or it is
>
> >> somewhere in CRUSH map?
>
> >>
>
> >> Vadim
>
> >> ____________
>
> >> From: Christian Balzer [[email protected]]
>
> >> Sent: Thursday, June 05, 2014 18:26
>
> >> To: Vadim Kimlaychuk
>
> >> Cc: [email protected]
>
> >> Subject: Re: [ceph-users] Hard drives of different sizes.
>
> >>
>
> >> Hello,
>
> >>
>
> >> On Thu, 5 Jun 2014 14:11:47 +0000 Vadim Kimlaychuk wrote:
>
> >>
>
> >>> Hello,
>
> >>>
>
> >>> Probably this is anti-pattern, but I have to get answer
>
> >>> how this will work / not work. Input:
>
> >>> I have single host for tests with ceph 0.80.1 and 2 OSD:
>
> >>> OSD.0 – 1000 Gb
>
> >>> OSD.1 – 750 Gb
>
> >>>
>
> >>> Recompiled CRUSH map to set „step chooseleaf firstn 0
>
> >>> type osd“
>
> >>>
>
> >> You got it half right.
>
> >>
>
> >> Version .8x aka Firefly has a default replication of 3, so you would
>
> >> need
>
> >> 3 OSDs at least.
>
> >>
>
> >> Christian
>
> >>>
>
> >>> I am expecting, that part of PG-s will have status
>
> >>> „active+clean“ (with size of ~750Gb) another part of PG-s will have
>
> >>> „active+degradated“ (with size of ~250Gb), because there is not
>
> >>> enough place to replicate data on the second OSD.
>
> >>>
>
> >>> Instead I have ALL PG-s „active + degradated“
>
> >>>
>
> >>> Output:
>
> >>> health HEALTH_WARN 192 pgs degraded; 192 pgs stuck unclean
>
> >>> monmap e1: 1 mons at {storage=172.16.3.2:6789/0}, election
>
> >>> epoch 2, quorum 0 storage osdmap e15: 2 osds: 2 up, 2 in
>
> >>> pgmap v29: 192 pgs, 3 pools, 0 bytes data, 0 objects
>
> >>> 71496 kB used, 1619 GB / 1619 GB avail
>
> >>> 192 active+degraded
>
> >>>
>
> >>> What is the logic behind this?? Can I use different
>
> >>> hard drives successfully? If yes – how?
>
> >>>
>
> >>> Thank you for explanation,
>
> >>>
>
> >>> Vadim
>
> >>>
>
> >>
>
> >> --
>
> >> Christian Balzer Network/Systems Engineer
>
> >> [email protected] Global OnLine Japan/Fusion Communications
>
> >> http://www.gol.com/
>
> >> _______________________________________________
>
> >> ceph-users mailing list
>
> >> [email protected]
>
> >> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
>
> >
>
> >
>
> > _______________________________________________
>
> > ceph-users mailing list
>
> > [email protected]
>
> > http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
>
> _______________________________________________
>
> ceph-users mailing list
>
> [email protected]
>
> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
>
--
Vincenzo Pii
Researcher, InIT Cloud Computing Lab
Zurich University of Applied Sciences (ZHAW)
http://www.cloudcomp.ch/
_______________________________________________
ceph-users mailing list
[email protected]
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
