Sorry again ..... its been a trying week at work and I've only just had time to pick this up
So, progress. I can now get the mons up and running. Took some fiddling with, the $cluster param breaks just about everything so had to stick with the default cluster name. I can live with that! I havent had to patch anything this time round. Next problem seems to be registering the osd_bootstrap key, the error is Notice: /Stage[main]/Cloud::Roles::Ofceph/Ceph::Key[client.bootstrap-osd]/Exec[ceph-injectkey-client.bootstrap-osd]/returns: + ceph --name mon. --keyring /var/lib/ceph/mon/ceph-ceph-001/keyring auth add client.bootstrap-osd --in-file=/var/lib/ceph/bootstrap-osd/ceph.keyring Notice: /Stage[main]/Cloud::Roles::Ofceph/Ceph::Key[client.bootstrap-osd]/Exec[ceph-injectkey-client.bootstrap-osd]/returns: libust[28614/28614]: Warning: HOME environment variable not set. Disabling LTTng-UST per-user tracing. (in setup_local_apps() at lttng-ust-comm.c:305) Notice: /Stage[main]/Cloud::Roles::Ofceph/Ceph::Key[client.bootstrap-osd]/Exec[ceph-injectkey-client.bootstrap-osd]/returns: Error EINVAL: entity client.bootstrap-osd exists but key does not match Error: /bin/true # comment to satisfy puppet syntax requirements set -ex ceph --name 'mon.' --keyring '/var/lib/ceph/mon/ceph-ceph-001/keyring' auth add client.bootstrap-osd --in-file=/var/lib/ceph/bootstrap-osd/ceph.keyring returned 22 instead of one of [0] Error: /Stage[main]/Openfish::Roles::Ofceph/Ceph::Key[client.bootstrap-osd]/Exec[ceph-injectkey-client.bootstrap-osd]/returns: change from notrun to 0 failed: /bin/true # comment to satisfy puppet syntax requirements set -ex ceph --name 'mon.' --keyring '/var/lib/ceph/mon/ceph-ceph-001/keyring' auth add client.bootstrap-osd --in-file=/var/lib/ceph/bootstrap-osd/ceph.keyring returned 22 instead of one of [0] I havent gotten as far as the osd's or gateways yet Help greatly appreciated! On Tue, Feb 24, 2015 at 10:31 AM, David Gurtner <[email protected]> wrote: > Hi again > > Just to test if the modules generally work on Trusty I set up a three > node cluster with shared mon/osd using the configuration below and > that worked. I was using the master branch of the module and had > firewall disabled. > > Cheers, > David > > > $admin_key = 'AQCTg71RsNIHORAAW+O6FCMZWBjmVfMIPk3MhQ==' > $mon_key = 'AQDesGZSsC7KJBAAw+W/Z4eGSQGAIbxWjxjvfw==' > $bootstrap_osd_key = 'AQABsWZSgEDmJhAAkAGSOOAJwrMHrM5Pz5On1A==' > $fsid = '066F558C-6789-4A93-AAF1-5AF1BA01A3AD' > > node /first|second|third/ { > class { 'ceph::repo': } > class { 'ceph': > fsid => $fsid, > mon_initial_members => 'first,second,third', > mon_host => '10.11.12.2,10.11.12.3,10.11.12.4', > } > ceph::mon { $::hostname: > key => $mon_key, > } > Ceph::Key { > inject => true, > inject_as_id => 'mon.', > inject_keyring => "/var/lib/ceph/mon/ceph-${::hostname}/keyring", > } > ceph::key { 'client.admin': > secret => $admin_key, > cap_mon => 'allow *', > cap_osd => 'allow *', > cap_mds => 'allow', > } > ceph::key { 'client.bootstrap-osd': > secret => $bootstrap_osd_key, > keyring_path => '/var/lib/ceph/bootstrap-osd/ceph.keyring', > cap_mon => 'allow profile bootstrap-osd', > } > ceph::osd { '/dev/sdb': } > } > > > On Mon, Feb 23, 2015 at 11:39 PM, David Gurtner <[email protected]> > wrote: > > Hi Stuart > > > > osds can only come up once at least the initial mon services are > > running and formed a quorum, so lets concentrate at getting the mons > > up first. > > If using cephx authentication all mon services need to be started at > > the same time (a similar time), translated to puppet this means all > > the puppet runs on the mons need to be started within a certain time > > frame (before the timeout of the mons trying to connect each other). > > The reason being that the mons first need to form a quorum before most > > of the further actions can happen. > > > > Can you provide some additional information about your setup? > Specifically: > > - what's your exact puppet modules configuration > > - what version are you running? > > - please be aware that running on Ubuntu 14.04 is currently not > > getting integration tested (currently only Ubuntu 12.04 and CentOS 6 > > are tested, support for CentOS 7 is in the works). > > - how many mons are you running > > - are the osds running on the same hosts as the mons > > - do you use a single network, or do you use separate cluster/public > networks > > - do you have a firewall running, if yes, can you share your firewall > > configuration (iptables -vnL) > > - what exactly did you need to patch? > > > > Cheers, > > David > > > > On Mon, Feb 23, 2015 at 8:16 PM, Stuart Fox <[email protected]> > wrote: > >> Sorry to ask a question and run, I had a busy weekend. I'll be back on > the > >> ceph POC later today. > >> > >> I'll paste some outputs from the puppet runs shortly but essentially, > no mon > >> or osd comes up. > >> I had to patch several manifests to explicitly reference > /etc/ceph/ceph.conf > >> and given that the puppet module calls a reasonable amount of shell > code, > >> getting debug info is not easy. The puppet runs exit correctly but no > >> service comes up. > >> I'm using an almost cut'n'paste from USECASES.md, even down to keeping > the > >> keys the same. > >> > >> BR, > >> Stuart > >> > >> On Sun, Feb 22, 2015 at 6:57 AM, David Moreau Simard <[email protected] > > > >> wrote: > >>> > >>> Hey Stuart, > >>> > >>> You might want to look at (and use, if you want) the built-in roles and > >>> profiles layer [1] which gives you a good idea of how the module is > used. > >>> > >>> It leverages ceph::profile::params to pass values [2] to other classes > >>> such as ceph::profile::osd [3]. > >>> > >>> Another good place to look at how to use the module would be the > >>> integration tests [4]. > >>> Since the module is integration tested, this means we actually use the > >>> module to deploy a virtual Ceph cluster and we test that the cluster > works - > >>> see for example the tests for ceph::profile::osd [4]. > >>> > >>> I'm dmsimard on #puppet-openstack and #openstack-operators if you need > a > >>> hand to get this to work. > >>> This is still a relatively new module in comparison to the likes of > >>> puppet-nova and such. Feedback and contributions are appreciated! > >>> > >>> [1] > >>> > https://github.com/stackforge/puppet-ceph/tree/master/manifests/profile > >>> [2] > >>> > https://github.com/stackforge/puppet-ceph/blob/master/manifests/profile/params.pp > >>> [3] > >>> > https://github.com/stackforge/puppet-ceph/blob/master/manifests/profile/osd.pp > >>> [4] > >>> > https://github.com/stackforge/puppet-ceph/blob/master/spec/system/ceph_profile_osd_spec.rb > >>> -- > >>> David Moreau Simard > >>> > >>> From: Andrew Woodward <[email protected]> > >>> Date: Saturday, February 21, 2015 at 4:39 PM > >>> To: Stuart Fox <[email protected]> > >>> Cc: "[email protected]" > >>> <[email protected]>, " > [email protected]" > >>> <[email protected]> > >>> Subject: Re: [Openstack-operators] Ceph puppet module > >>> > >>> [+puppet-openstack ml] > >>> > >>> Stuart, > >>> > >>> Please review the USECASES.md such as [1]. You should find usable > examples > >>> there. If you are still having problems please reach out with a more > detail > >>> about what you configuration you are attempting to deploy. The module > >>> authors can be found in #puppet-openstack and are usually lurking on > the > >>> puppet-openstack mailing list (CC'd). > >>> > >>> > >>> [1] > https://github.com/stackforge/puppet-ceph/blob/master/USECASES.md#i-want-to-operate-a-production-cluster > >>> > >>> > >>> > >>> On Sat, Feb 21, 2015 at 12:04 AM, Stuart Fox <[email protected]> > wrote: > >>>> > >>>> Hey all > >>>> > >>>> Im having a complete nightmare trying to get ceph deployed using the > >>>> https://github.com/stackforge/puppet-ceph module. Scant > documentation isn't > >>>> helping my cause! > >>>> > >>>> Is anybody else using this module? Im attempting to deploy ceph as a > >>>> cinder backend in Juno on Ubuntu 14.04 although I haven't gotten as > far as > >>>> integration yet. > >>>> > >>>> Does anyone have working example's I could look at? > >>>> > >>>> -- > >>>> BR, > >>>> Stuart > >>>> > >>>> _______________________________________________ > >>>> OpenStack-operators mailing list > >>>> [email protected] > >>>> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators > >>>> > >>> > >>> > >>> > >>> -- > >>> Andrew > >>> Mirantis > >>> Fuel community ambassador > >>> Ceph community > >> > >> > >> > >> > >> -- > >> BR, > >> Stuart > >> > >> _______________________________________________ > >> OpenStack-operators mailing list > >> [email protected] > >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators > >> > -- BR, Stuart
_______________________________________________ OpenStack-operators mailing list [email protected] http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators
