Update 2 Running
* ceph auth del client.bootstrap-osd* Then *ceph auth add client.bootstrap-osd -i /var/lib/ceph/bootstrap-osd/ceph.keyring* Allows me to run puppet and get an osd working On Mon, Mar 2, 2015 at 10:37 PM, Stuart Fox <[email protected]> wrote: > Sorry again ..... its been a trying week at work and I've only just had > time to pick this up > > So, progress. I can now get the mons up and running. Took some fiddling > with, the $cluster param breaks just about everything so had to stick with > the default cluster name. I can live with that! I havent had to patch > anything this time round. > > Next problem seems to be registering the osd_bootstrap key, the error is > > Notice: > /Stage[main]/Cloud::Roles::Ofceph/Ceph::Key[client.bootstrap-osd]/Exec[ceph-injectkey-client.bootstrap-osd]/returns: > + ceph --name mon. --keyring /var/lib/ceph/mon/ceph-ceph-001/keyring auth > add client.bootstrap-osd --in-file=/var/lib/ceph/bootstrap-osd/ceph.keyring > > Notice: > /Stage[main]/Cloud::Roles::Ofceph/Ceph::Key[client.bootstrap-osd]/Exec[ceph-injectkey-client.bootstrap-osd]/returns: > libust[28614/28614]: Warning: HOME environment variable not set. Disabling > LTTng-UST per-user tracing. (in setup_local_apps() at lttng-ust-comm.c:305) > > Notice: > /Stage[main]/Cloud::Roles::Ofceph/Ceph::Key[client.bootstrap-osd]/Exec[ceph-injectkey-client.bootstrap-osd]/returns: > Error EINVAL: entity client.bootstrap-osd exists but key does not match > > Error: /bin/true # comment to satisfy puppet syntax requirements > > set -ex > > ceph --name 'mon.' --keyring > '/var/lib/ceph/mon/ceph-ceph-001/keyring' auth add client.bootstrap-osd > --in-file=/var/lib/ceph/bootstrap-osd/ceph.keyring returned 22 instead of > one of [0] > > Error: > /Stage[main]/Openfish::Roles::Ofceph/Ceph::Key[client.bootstrap-osd]/Exec[ceph-injectkey-client.bootstrap-osd]/returns: > change from notrun to 0 failed: /bin/true # comment to satisfy puppet > syntax requirements > > set -ex > > ceph --name 'mon.' --keyring > '/var/lib/ceph/mon/ceph-ceph-001/keyring' auth add client.bootstrap-osd > --in-file=/var/lib/ceph/bootstrap-osd/ceph.keyring returned 22 instead of > one of [0] > > > I havent gotten as far as the osd's or gateways yet > > Help greatly appreciated! > > > > On Tue, Feb 24, 2015 at 10:31 AM, David Gurtner <[email protected]> > wrote: > >> Hi again >> >> Just to test if the modules generally work on Trusty I set up a three >> node cluster with shared mon/osd using the configuration below and >> that worked. I was using the master branch of the module and had >> firewall disabled. >> >> Cheers, >> David >> >> >> $admin_key = 'AQCTg71RsNIHORAAW+O6FCMZWBjmVfMIPk3MhQ==' >> $mon_key = 'AQDesGZSsC7KJBAAw+W/Z4eGSQGAIbxWjxjvfw==' >> $bootstrap_osd_key = 'AQABsWZSgEDmJhAAkAGSOOAJwrMHrM5Pz5On1A==' >> $fsid = '066F558C-6789-4A93-AAF1-5AF1BA01A3AD' >> >> node /first|second|third/ { >> class { 'ceph::repo': } >> class { 'ceph': >> fsid => $fsid, >> mon_initial_members => 'first,second,third', >> mon_host => '10.11.12.2,10.11.12.3,10.11.12.4', >> } >> ceph::mon { $::hostname: >> key => $mon_key, >> } >> Ceph::Key { >> inject => true, >> inject_as_id => 'mon.', >> inject_keyring => "/var/lib/ceph/mon/ceph-${::hostname}/keyring", >> } >> ceph::key { 'client.admin': >> secret => $admin_key, >> cap_mon => 'allow *', >> cap_osd => 'allow *', >> cap_mds => 'allow', >> } >> ceph::key { 'client.bootstrap-osd': >> secret => $bootstrap_osd_key, >> keyring_path => '/var/lib/ceph/bootstrap-osd/ceph.keyring', >> cap_mon => 'allow profile bootstrap-osd', >> } >> ceph::osd { '/dev/sdb': } >> } >> >> >> On Mon, Feb 23, 2015 at 11:39 PM, David Gurtner <[email protected]> >> wrote: >> > Hi Stuart >> > >> > osds can only come up once at least the initial mon services are >> > running and formed a quorum, so lets concentrate at getting the mons >> > up first. >> > If using cephx authentication all mon services need to be started at >> > the same time (a similar time), translated to puppet this means all >> > the puppet runs on the mons need to be started within a certain time >> > frame (before the timeout of the mons trying to connect each other). >> > The reason being that the mons first need to form a quorum before most >> > of the further actions can happen. >> > >> > Can you provide some additional information about your setup? >> Specifically: >> > - what's your exact puppet modules configuration >> > - what version are you running? >> > - please be aware that running on Ubuntu 14.04 is currently not >> > getting integration tested (currently only Ubuntu 12.04 and CentOS 6 >> > are tested, support for CentOS 7 is in the works). >> > - how many mons are you running >> > - are the osds running on the same hosts as the mons >> > - do you use a single network, or do you use separate cluster/public >> networks >> > - do you have a firewall running, if yes, can you share your firewall >> > configuration (iptables -vnL) >> > - what exactly did you need to patch? >> > >> > Cheers, >> > David >> > >> > On Mon, Feb 23, 2015 at 8:16 PM, Stuart Fox <[email protected]> >> wrote: >> >> Sorry to ask a question and run, I had a busy weekend. I'll be back on >> the >> >> ceph POC later today. >> >> >> >> I'll paste some outputs from the puppet runs shortly but essentially, >> no mon >> >> or osd comes up. >> >> I had to patch several manifests to explicitly reference >> /etc/ceph/ceph.conf >> >> and given that the puppet module calls a reasonable amount of shell >> code, >> >> getting debug info is not easy. The puppet runs exit correctly but no >> >> service comes up. >> >> I'm using an almost cut'n'paste from USECASES.md, even down to keeping >> the >> >> keys the same. >> >> >> >> BR, >> >> Stuart >> >> >> >> On Sun, Feb 22, 2015 at 6:57 AM, David Moreau Simard < >> [email protected]> >> >> wrote: >> >>> >> >>> Hey Stuart, >> >>> >> >>> You might want to look at (and use, if you want) the built-in roles >> and >> >>> profiles layer [1] which gives you a good idea of how the module is >> used. >> >>> >> >>> It leverages ceph::profile::params to pass values [2] to other classes >> >>> such as ceph::profile::osd [3]. >> >>> >> >>> Another good place to look at how to use the module would be the >> >>> integration tests [4]. >> >>> Since the module is integration tested, this means we actually use the >> >>> module to deploy a virtual Ceph cluster and we test that the cluster >> works - >> >>> see for example the tests for ceph::profile::osd [4]. >> >>> >> >>> I'm dmsimard on #puppet-openstack and #openstack-operators if you >> need a >> >>> hand to get this to work. >> >>> This is still a relatively new module in comparison to the likes of >> >>> puppet-nova and such. Feedback and contributions are appreciated! >> >>> >> >>> [1] >> >>> >> https://github.com/stackforge/puppet-ceph/tree/master/manifests/profile >> >>> [2] >> >>> >> https://github.com/stackforge/puppet-ceph/blob/master/manifests/profile/params.pp >> >>> [3] >> >>> >> https://github.com/stackforge/puppet-ceph/blob/master/manifests/profile/osd.pp >> >>> [4] >> >>> >> https://github.com/stackforge/puppet-ceph/blob/master/spec/system/ceph_profile_osd_spec.rb >> >>> -- >> >>> David Moreau Simard >> >>> >> >>> From: Andrew Woodward <[email protected]> >> >>> Date: Saturday, February 21, 2015 at 4:39 PM >> >>> To: Stuart Fox <[email protected]> >> >>> Cc: "[email protected]" >> >>> <[email protected]>, " >> [email protected]" >> >>> <[email protected]> >> >>> Subject: Re: [Openstack-operators] Ceph puppet module >> >>> >> >>> [+puppet-openstack ml] >> >>> >> >>> Stuart, >> >>> >> >>> Please review the USECASES.md such as [1]. You should find usable >> examples >> >>> there. If you are still having problems please reach out with a more >> detail >> >>> about what you configuration you are attempting to deploy. The module >> >>> authors can be found in #puppet-openstack and are usually lurking on >> the >> >>> puppet-openstack mailing list (CC'd). >> >>> >> >>> >> >>> [1] >> https://github.com/stackforge/puppet-ceph/blob/master/USECASES.md#i-want-to-operate-a-production-cluster >> >>> >> >>> >> >>> >> >>> On Sat, Feb 21, 2015 at 12:04 AM, Stuart Fox <[email protected]> >> wrote: >> >>>> >> >>>> Hey all >> >>>> >> >>>> Im having a complete nightmare trying to get ceph deployed using the >> >>>> https://github.com/stackforge/puppet-ceph module. Scant >> documentation isn't >> >>>> helping my cause! >> >>>> >> >>>> Is anybody else using this module? Im attempting to deploy ceph as a >> >>>> cinder backend in Juno on Ubuntu 14.04 although I haven't gotten as >> far as >> >>>> integration yet. >> >>>> >> >>>> Does anyone have working example's I could look at? >> >>>> >> >>>> -- >> >>>> BR, >> >>>> Stuart >> >>>> >> >>>> _______________________________________________ >> >>>> OpenStack-operators mailing list >> >>>> [email protected] >> >>>> >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators >> >>>> >> >>> >> >>> >> >>> >> >>> -- >> >>> Andrew >> >>> Mirantis >> >>> Fuel community ambassador >> >>> Ceph community >> >> >> >> >> >> >> >> >> >> -- >> >> BR, >> >> Stuart >> >> >> >> _______________________________________________ >> >> OpenStack-operators mailing list >> >> [email protected] >> >> >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators >> >> >> > > > > -- > BR, > Stuart > -- BR, Stuart
_______________________________________________ OpenStack-operators mailing list [email protected] http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators
