Hi everyone, After reading all the research papers and docs over the last few months and waiting for Cuttlefish, I finally deployed a test cluster of 18 osds across 6 hosts. It's performing better than I expected so far, all on the default single interface.
I was also surprised by the minimal ceph.conf and non-functioning "service ceph" too, so perhaps worth mentioning in the doc? ceph-deploy also failed to start one of the osds (with activate), but running ceph-osd manually worked. I now want to enter a secondary interface for the cluster-side network, so I assume I have to enter all the OSD's in ceph.conf manually anyway? Very impressed so far! Danny On 9 May 2013 19:29, Sage Weil <[email protected]> wrote: > On Thu, 9 May 2013, Greg Chavez wrote: > > So I feel like I'm missing something. I just deployed 3 storage nodes > > with ceph-deploy, each with a monitor daemon and 6-8 osd's. All of > > them seem to be active with health OK. However, it doesn't seem that > > I ended up with a useful ceph.conf. > > > > ( running 0.61-113-g61354b2-1raring ) > > > > This is all I get: > > > > [global] > > fsid = af1581c1-8c45-4e24-b4f1-9a56e8a62aeb > > mon_initial_members = kvm-cs-sn-10i, kvm-cs-sn-14i, kvm-cs-sn-15i > > mon_host = 192.168.241.110,192.168.241.114,192.168.241.115 > > auth_supported = cephx > > osd_journal_size = 1024 > > filestore_xattr_use_omap = true > > This is correct. It is no longer necessary to enumerate daemons in > ceph.conf when using the default locations (/var/lib/ceph/osd/*). > > > When you run "service ceph status", it returns nothing because it > > can't find any osd stanzas. > > 'service' invokes the sysvinit scripts, and on Ubuntu we are using > upstart. Try > > initctl list | grep ceph > > to see what is running. > > sage > > > > Based on the directory names in > > /var/lib/ceph/mon and the parsing of the mounted storage volumes I > > wrote this out: > > > > http://pastebin.com/DLgtiC23 > > > > And then pushed it out. > > > > The monitor id names seem odd to me, with the hostname instead of a, > > b, and c, but whatever. > > > > Now I get this output: > > > > root@kvm-cs-sn-10i:/etc/ceph# service ceph -a status > > === mon.kvm-cs-sn-10i === > > mon.kvm-cs-sn-10i: not running. > > === mon.kvm-cs-sn-14i === > > mon.kvm-cs-sn-14i: not running. > > === mon.kvm-cs-sn-15i === > > mon.kvm-cs-sn-15i: not running. > > === osd.0 === > > osd.0: not running. > > === osd.1 === > > osd.1: not running. > > === osd.10 === > > osd.10: not running. > > ...etc.... > > > > Not true! Even worse, when I try to run "service ceph -a start", It > > freaks and complains about missing keys. So now I have this process > > hanging around: > > > > /usr/bin/python /usr/sbin/ceph-create-keys -i kvm-cs-sn-10i > > > > Here's the output from that attempt: > > > > root@kvm-cs-sn-10i:/tmp# service ceph -a start > > === mon.kvm-cs-sn-10i === > > Starting Ceph mon.kvm-cs-sn-10i on kvm-cs-sn-10i... > > failed: 'ulimit -n 8192; /usr/bin/ceph-mon -i kvm-cs-sn-10i > > --pid-file /var/run/ceph/mon.kvm-cs-sn-10i.pid -c /etc/ceph/ceph.conf > > ' > > Starting ceph-create-keys on kvm-cs-sn-10i... > > > > Luckily I hadn't set up my ssh keys yet, so that's as far as I got. > > > > Would dearly love some guidance. Thanks in advance! > > > > --Greg Chavez > > _______________________________________________ > > ceph-users mailing list > > [email protected] > > http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com > > > > > _______________________________________________ > ceph-users mailing list > [email protected] > http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com > -- Mean Trading Systems LLP http://www.meantradingsystems.com
_______________________________________________ ceph-users mailing list [email protected] http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
