Hello Sebastien,
I am configuring ceph with 3 node storage cluster + one ceph admin.
I have few questions.
I have created pool name ' storage' with the replication size 3 on it and I
have set the CRUSH rule .
root@node1:/home/oss# ceph osd dump | grep -E 'storage'
pool 9 'storage' replicated size 3 min_size 1 crush_ruleset 3 object_hash
rjenkins pg_num 8 pgp_num 8 last_change 160 flags hashpspool stripe_width 0
Note: command used to set replication size , #ceph osd pool set stotage size 3.
Even after setting replication size 3 , my data is not getting replicated on
all the 3 nodes.
Example:
root@Cephadmin:/home/oss# ceph osd map storage check1
osdmap e122 pool 'storage' (9) object 'check1' -> pg 9.7c9c5619 (9.1) -> up
([0,2,1], p0) acting ([0,2,1], p0)
but here if I shutdown my 2 nodes I will be unable to access data. In actual
scenario I should be able to access / write data as my other 3rd node is up (if
my understanding is correct). Please let me know where I am wrong.
Crush Map:
# begin crush map
tunable choose_local_tries 0
tunable choose_local_fallback_tries 0
tunable choose_total_tries 50
tunable chooseleaf_descend_once 1
# devices
device 0 osd.0
device 1 osd.1
device 2 osd.2
# types
type 0 osd
type 1 host
type 2 chassis
type 3 rack
type 4 row
type 5 pdu
type 6 pod
type 7 room
type 8 datacenter
type 9 region
type 10 root
type 11 pool
# buckets
host node2 {
id -2 # do not change unnecessarily
# weight 0.030
alg straw
hash 0 # rjenkins1
item osd.0 weight 0.030
}
host node3 {
id -3 # do not change unnecessarily
# weight 0.030
alg straw
hash 0 # rjenkins1
item osd.1 weight 0.030
}
host node1 {
id -4 # do not change unnecessarily
# weight 0.030
alg straw
hash 0 # rjenkins1
item osd.2 weight 0.030
}
root default {
id -1 # do not change unnecessarily
# weight 0.090
alg straw
hash 0 # rjenkins1
item node2 weight 0.030
item node3 weight 0.030
item node1 weight 0.030
}
pool storage {
id -5 # do not change unnecessarily
# weight 0.090
alg straw
hash 0 # rjenkins1
item node2 weight 0.030
item node3 weight 0.030
item node1 weight 0.030
}
# rules
rule replicated_ruleset {
ruleset 0
type replicated
min_size 1
max_size 10
step take default
step chooseleaf firstn 0 type host
step emit
}
rule storage {
ruleset 3
type replicated
min_size 1
max_size 10
step take storage
step choose firstn 0 type osd
step emit
}
# end crush map
root@node1:/home/oss# ceph osd tree
# id weight type name up/down reweight
-5 0.09 pool storage
-2 0.03 host node2
0 0.03 osd.0 up 1
-3 0.03 host node3
1 0.03 osd.1 up 1
-4 0.03 host node1
2 0.03 osd.2 up 1
-1 0.09 root default
-2 0.03 host node2
0 0.03 osd.0 up 1
-3 0.03 host node3
1 0.03 osd.1 up 1
-4 0.03 host node1
2 0.03 osd.2 up 1
Refernce:
http://www.sebastien-han.fr/blog/2012/12/07/ceph-2-speed-storage-with-crush/
-----Original Message-----
From: Sebastien Han [mailto:[email protected]]
Sent: Tuesday, September 16, 2014 7:43 PM
To: Channappa Negalur, M.
Cc: [email protected]
Subject: Re: [ceph-users] vdb busy error when attaching to instance
Did you follow this ceph.com/docs/master/rbd/rbd-openstack/ to configure your
env?
On 12 Sep 2014, at 14:38, [email protected] wrote:
> Hello Team,
>
> I have configured ceph as a multibackend for openstack.
>
> I have created 2 pools .
> 1. Volumes (replication size =3 )
> 2. poolb (replication size =2 )
>
> Below is the details from /etc/cinder/cinder.conf
>
> enabled_backends=rbd-ceph,rbd-cephrep
> [rbd-ceph]
> volume_driver=cinder.volume.drivers.rbd.RBDDriver
> rbd_pool=volumes
> volume_backend_name=ceph
> rbd_user=volumes
> rbd_secret_uuid=34c88ed2-1cf6-446d-8564-f888934eec35
> volumes_dir=/var/lib/cinder/volumes
> [rbd-cephrep]
> volume_driver=cinder.volume.drivers.rbd.RBDDriver
> rbd_pool=poolb
> volume_backend_name=ceph1
> rbd_user=poolb
> rbd_secret_uuid=d62b0df6-ee26-46f0-8d90-4ef4d55caa5b
> volumes_dir=/var/lib/cinder/volumes1
>
> when I am attaching a volume to a instance I am getting "DeviceIsBusy: The
> supplied device (vdb) is busy" error.
>
> Please let me know how to correct this..
>
> Regards,
> Malleshi CN
>
>
> This message is for the designated recipient only and may contain privileged,
> proprietary, or otherwise confidential information. If you have received it
> in error, please notify the sender immediately and delete the original. Any
> other use of the e-mail by you is prohibited. Where allowed by local law,
> electronic communications with Accenture and its affiliates, including e-mail
> and instant messaging (including content), may be scanned by our systems for
> the purposes of information security and assessment of internal compliance
> with Accenture policy.
> ______________________________________________________________________
> ________________
>
> www.accenture.com
> _______________________________________________
> ceph-users mailing list
> [email protected]
> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
Cheers.
----
Sébastien Han
Cloud Architect
"Always give 100%. Unless you're giving blood."
Phone: +33 (0)1 49 70 99 72
Mail: [email protected]
Address : 11 bis, rue Roquépine - 75008 Paris Web : www.enovance.com - Twitter
: @enovance
_______________________________________________
ceph-users mailing list
[email protected]
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
