Thank you for the comments Emilien! Updated as per your suggestions: puppet-veritas-hyperscale: https://github.com/abhishek-kane/puppet-veritas-hyperscale
Also, please find the inline replies to your comments. We had been able to call the old puppet modules (which had scripts) using the generic puppet way (puppet agent --test). But we are unable to deploy the new puppet modules (which use existing puppet classes) the tripleo way. Following paste shows a sample puppet module which we are trying to deploy on an overcloud which is already deployed. We see that our resources are in “CREATE_COMPLETE” status, but the actual operation hasn’t happened. http://paste.openstack.org/show/612537/ Regards, Abhishek On 6/14/17, 1:18 AM, "Emilien Macchi" <emil...@redhat.com> wrote: On Wed, Jun 7, 2017 at 10:38 AM, Abhishek Kane <abhishek.k...@veritas.com> wrote: > Hi, > > On cinder node- we need to modify the cinder.conf. We don’t change any config apart from this. We want to keep the config changes in heat template, package installation in puppet, and trigger rest of the operations via Horizon (as it’s done today). We are also trying to get rid of the nova.conf file changes. Once the approach for cinder is sorted, will get back on this. > > If this is correct approach for cinder, I will raise review requests for the following projects: > puppet-tripleo: http://paste.openstack.org/show/611697/ > puppet-cinder: http://paste.openstack.org/show/611698/ > tripleo-heat-templates: http://paste.openstack.org/show/611700/ > > Also, I am not sure which TripleO repos need to be patched for the controller components. > > We have decomposed the controller bin installer into idempotent modules/scripts. Now, the installer is not a black box operation: > https://github.com/abhishek-kane/puppet-veritas-hyperscale > The inline replies below are w.r.t. this project. The product installer bin currently works in atomic fashion. One issue which we see in puppet is the error handling and rollback operations. > > Thanks, > Abhishek > > On 6/1/17, 8:41 PM, "Emilien Macchi" <emil...@redhat.com> wrote: > > On Thu, Jun 1, 2017 at 3:47 PM, Abhishek Kane <abhishek.k...@veritas.com> wrote: > > Hi Emilien, > > > > The bin does following things on controller: > > 1. Install core HyperScale packages. > > Should be done by Puppet, with Package resource. > Ak> It’s done. > > > 2. Start HyperScale API server > > Should be done by Puppet, with Service resource. > AK> It’s done. > > > 3. Install UI packages. This will add new files to and modify some existing files of Horison. > > Should be done by Puppet, with Package resource and also some changes > in puppet-horizon maybe if you need to change Horizon config. > Ak> We have got rid of the horizon dependency right now. Our GUI components get installed via separate package. > > > 4. Create HyperScale user in mysql db. Create database and dump config. Add permissions of nova and cinder DB to HyperScale user. > > We have puppet-openstacklib which already manages DBs, you could > easily re-use it. Please look at puppet-nova for example to see how > things works in nova::db::mysql, etc. > AK> TBD > > > 5. Add ceilometer pollsters for additional stats and modify ceilometer files. > > puppet-ceilometer I guess. What do you mean by "files"? Config files? > Ak> We are trying to get rid of this dependency as well. TBD. > > > 6. Change OpenStack configuration: > > a. Create rabbitmq exchanges > > puppet-* modules already does it. > AK> It’s done via script. Do we need to patch any module? Everything that touch *.conf files of OpenStack services need to be done by existing openstack/puppet-* modules. AK> https://github.com/abhishek-kane/puppet-veritas-hyperscale/blob/master/manifests/hs_rabbitmq.pp We are still doing some operation via script, but it’s idempotent. > > > b. Create keystone user > > puppet-keystone already does it. > AK> It’s done via script. Do we need to patch keystone module? No, you'll need to create the right user/roles/endpoints/... in puppet-tripleo, in a new profile, most probably. You'll probably need to read a bit about: https://github.com/openstack/puppet-keystone#setup Let me know if you need more help on this thing. AK> I have already. https://github.com/abhishek-kane/puppet-veritas-hyperscale/blob/master/manifests/hs_keystone.pp > > > c. Define new flavors > > puppet-nova can manage flavors. > AK> It’s done via script. Do we need to patch nova module? Same as Keystone. See: https://github.com/openstack/puppet-openstack-integration/blob/master/manifests/provision.pp#L7-L13 You'll probably need that thing in your module or in puppet-tripleo (composition layer for tripleo). AK> Already done that. https://github.com/abhishek-kane/puppet-veritas-hyperscale/blob/master/manifests/hs_flavors.pp > > > d. Create HyperScale volume type and set default volume type to HyperScale in cinder.conf. > > we already support multi backends in tripleo, HyperScale would just be > a new addition. Re-use the bits please: puppet-cinder and > puppet-tripleo will need to be patched. > AK> It’s done via script. Do we need to patch cinder module? Same, please look how other backends are done in TripleO, we have a bunch of examples in puppet-tripleo. Ak> The paste links mentioned in last email. Is it correct approach? puppet-cinder: http://paste.openstack.org/show/612493/ puppet-tripleo: http://paste.openstack.org/show/611697/ tripleo-heat-templates: http://paste.openstack.org/show/611700/ > > e. Restart openstack’s services > > Already done by openstack/puppet-* modules. > AK> We are trying to get rid of all OpenStack config file changes that we used to do. TBD. > > > 7. Configure HyperScale services > > Should be done by your module, (you can either write a _config > provider if it's ini standard otherwise just do a template that you > ship in the module, like puppet-horizon). > AK> It’s done. > > > Once the controller is configured, we use HyperScale’s CLI to configure data and compute nodes- > > > > On data node (cinder): > > 1. Install HyperScale data node packages. > > Should be done by Puppet, with Package resource. > Ak> We have written a puppet module. https://github.com/abhishek-kane/puppet-veritas-hyperscale/blob/master/manifests/datanode_pkg_inst.pp > > 2. Change cinder.conf to add backend and change rpc_backend. > > puppet-cinder Ak > Paste links mentioned above. > > > 3. Give the raw data disks and meta disks to HyperScale storage layer for usage. > > what does it means? Do you run a CLI for that? I didn't get reply for that FYI ^ Ak> We need disk names as user input. We show the filtered (exclude mounted, partitioned, etc disks) disks on the GUI and let user choose disks. We create LVM volumes and create XFS mount points which we use later as backend for instance volumes. Since this operation is not done at the time of deployment, we don’t have any puppet module for this. > > 4. Configure HyperScale services. > > Should be done by Puppet, with Service resource. > AK> We have added a puppet module for this. https://github.com/abhishek-kane/puppet-veritas-hyperscale/blob/master/manifests/datanode_service_start.pp > > 5. Dump config in the HyperScale database. > > can be done by a script maybe? ditto. Ak> Work in progress to use the openstacklib mysql component. Currently we are running .sql files. Once we finalize it I will upload it on the github and update. > > > > On compute (nova): > > 1. Install HyperScale compute packages. > > Should be done by Puppet, with Package resource. > Ak> https://github.com/abhishek-kane/puppet-veritas-hyperscale/blob/master/manifests/compute_pkg_inst.pp > > 2. Configure cgroup. > > we don't manage cgroups in TripleO AFIK yet but it's something we > could add, maybe with a puppet module. > AK> TODO > > 3. Disable selinux. > > Please don't do that. Disabling SElinux is a NOGO when adding new > features (sorry to care about Security). > Ak> We have now removed this change. We don’t change any selinux setting now. > > 4. Add ceilometer pollsters for additional stats and modify ceilometer files. > > puppet-ceilometer > Ak> We are trying to get rid of the ceilometer changes. Will update on this later. > > 5. Modify qemu.conf to relax ACS checks. > > puppet-nova maybe, but not sure we really want to do that: > https://vfio.blogspot.fr/2014/08/iommu-groups-inside-and-out.html > > Any details on why you're doing it? > AK> TODO > > 6. Modify libvirt.conf and libvirtd to allow live migration. > Ak> TODO > It's already supported by puppet-nova. > > > 7. Change network settings. > > Should be done by os-net-config in TripleO. > Ak> TODO > > 8. Configure HyperScale services. > > Done by your module (again). > Ak> https://github.com/abhishek-kane/puppet-veritas-hyperscale/blob/master/manifests/compute_service_start.pp > > 9. Dump config in the HyperScale database. > > same as before. > Ak> Work in progress to use the openstacklib mysql component. Currently we are running .sql files. Once we finalize it I will upload it on the github and update. > > > > We assume that we will not require steps to install packages if we put packages in the overcloud image. We have started to convert the bin and the CLI into puppet modules. > > > > > > Regards, > > Abhishek > > Hope it helped. > > > On 6/1/17, 4:24 AM, "Emilien Macchi" <emil...@redhat.com> wrote: > > > > On Wed, May 31, 2017 at 6:29 PM, Dnyaneshwar Pawar > > <dnyaneshwar.pa...@veritas.com> wrote: > > > Hi Alex, > > > > > > Currently we have puppet modules[0] to configure our software which has > > > components on Openstack Controller, Cinder node and Nova node. > > > As per document[1] we successfully tried out role specific configuration[2]. > > > > > > So, does it mean that if we have an overcloud image with our packages > > > inbuilt and we call our configuration scripts using role specific > > > configuration, we may not need puppet modules[0] ? Is it acceptable > > > deployment method? > > > > So running a binary from Puppet, to make configuration management is > > not something we recommend. > > Puppet has been good at managing configuration files and services for > > example. In your module, you just manage a file and execute it. The > > problem with that workflow is we have no idea what happens in backend. > > Also we have no way to make Puppet run idempotent, which is one > > important aspect in TripleO. > > > > Please tell us what does the binary, and maybe we can convert the > > tasks into Puppet resources that could be managed by your module. Also > > make the resources by class (service), so we can plug it into the > > composable services in TripleO. > > > > Thanks, > > > > > [0] https://github.com/abhishek-kane/puppet-veritas-hyperscale > > > [1] > > > https://docs.openstack.org/developer/tripleo-docs/advanced_deployment/node_config.html > > > [2] http://paste.openstack.org/show/611116/ > > > > > > Thanks, > > > Dnyaneshwar > > > > > > On 5/30/17, 6:52 PM, "Alex Schultz" <aschu...@redhat.com> wrote: > > > > > > On Mon, May 29, 2017 at 5:05 AM, Dnyaneshwar Pawar > > > <dnyaneshwar.pa...@veritas.com> wrote: > > > > > > Hi, > > > > > > I am tying to deploy a software on openstack controller on the overcloud. > > > One way to do this is by modifying ‘overcloud image’ so that all packages of > > > our software are added to image and then run overcloud deploy. > > > Other way is to write heat template and puppet module which will deploy the > > > required packages. > > > > > > Question: Which of above two approaches is better? > > > > > > Note: Configuration part of the software will be done via separate heat > > > template and puppet module. > > > > > > > > > Usually you do both. Depending on how the end user is expected to > > > deploy, if they are using the TripleoPackages service[0] in their > > > role, the puppet installation of the package won't actually work (we > > > override the package provider to noop) so it needs to be in the > > > images. That being said, usually there is also a bit of puppet that > > > needs to be written to configure the end service and as a best > > > practice (and for development purposes), it's a good idea to also > > > capture the package in the manifest. > > > > > > Thanks, > > > -Alex > > > > > > [0] > > > https://github.com/openstack/tripleo-heat-templates/blob/master/puppet/services/tripleo-packages.yaml > > > > > > > > > Thanks and Regards, > > > Dnyaneshwar > > > > > > __________________________________________________________________________ > > > OpenStack Development Mailing List (not for usage questions) > > > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > > > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > > > > > > > > > __________________________________________________________________________ > > > OpenStack Development Mailing List (not for usage questions) > > > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > > > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > > > > > > > > > __________________________________________________________________________ > > > OpenStack Development Mailing List (not for usage questions) > > > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > > > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > > > > > > > > > > > -- > > Emilien Macchi > > > > __________________________________________________________________________ > > OpenStack Development Mailing List (not for usage questions) > > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > > > > > > > > > > __________________________________________________________________________ > > OpenStack Development Mailing List (not for usage questions) > > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > > > > -- > Emilien Macchi > > __________________________________________________________________________ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > > > __________________________________________________________________________ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev -- Emilien Macchi __________________________________________________________________________ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __________________________________________________________________________ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
