Re: [openstack-dev] [tripleo] mismatch of user/group id between ovs (baremetal) and libvirt (container)
On 23 August 2017 at 07:18, Saravanan KR wrote: > On Wed, Aug 23, 2017 at 4:28 AM, Oliver Walsh wrote: >> Hi, >> >>> sed -i 's/Group=qemu/Group=42427/' >>> /usr/lib/systemd/system/ovs-vswitchd.service >> >> Can't this be overridden via /etc/systemd/system/ovs-vswitchd.service? >> > Yes. I just provided the changes done on a existing deployment to make > it work. I will incorporate it to templates. > >>> This change basically runs ovs with group id of kolla's qemu user >>> (42427). For the solution, my opinion is that we don't require host's >>> qemu (107) user in a containerized deployment. I am planning to ensure >>> that kolla's user id (42427) is updated to the host via the host prep >>> tasks. Let me know if there is any other aspects to be considered. >>> >> >> Might be worth considering overriding the qemu uid/gid to 107 in the >> kolla config file instead. > > Definitely it will be good for upgrade to keep the same uid/gid so > that existing DPDK based VMs continue to work after upgrade. > > But any other specific reasons for sticking with same qemu uid/gid in > kolla containers? Do you foresee any particular cases related to it? > I don't expect it would cause any issues. Many of the uids/gids created during rpm installs are dynamic so they could vary depending on the order that uids are created. This could result in inconsistent uids/gids across the docker images. I assume this is the reason that kolla is setting the uids/gids in the base image - the numeric values are not significant, they just need to be consistent. Thanks, Ollie > Regards, > Saravanan KR > >> >> Regards, >> Ollie >> >> __ >> OpenStack Development Mailing List (not for usage questions) >> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > > __ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [tripleo] mismatch of user/group id between ovs (baremetal) and libvirt (container)
Hi, > sed -i 's/Group=qemu/Group=42427/' > /usr/lib/systemd/system/ovs-vswitchd.service Can't this be overridden via /etc/systemd/system/ovs-vswitchd.service? > This change basically runs ovs with group id of kolla's qemu user > (42427). For the solution, my opinion is that we don't require host's > qemu (107) user in a containerized deployment. I am planning to ensure > that kolla's user id (42427) is updated to the host via the host prep > tasks. Let me know if there is any other aspects to be considered. > Might be worth considering overriding the qemu uid/gid to 107 in the kolla config file instead. Regards, Ollie __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [tripleo] [ironic] Need to update kernel parameters on local boot
Hi Saravanan, Ah, ok. I'd assumed ironic didn't touch the bootloader as the docs show a mkconfig & reboot to customize grub for localboot: http://docs.openstack.org/project-install-guide/baremetal/draft/advanced.html#appending-kernel-parameters-to-boot-instances. Thanks, Ollie On 13 December 2016 at 11:03, Saravanan KR wrote: > Hi Oliver, > > During the deployment, Ironic will start the node with IPA ramdisk, > which will copy the overcloud image to the node's disk, then configure > the grub cfg [1], set the node to local boot and reboot the node. > After which the node will be boot with the overcloud image. So no > reboot required in the overcloud image, as the "deploy steps" will be > run by the IPA itself. > > Regards, > Saravanan KR > > [1] > https://github.com/openstack/ironic-python-agent/blob/master/ironic_python_agent/extensions/image.py#L136 > > > On Tue, Dec 13, 2016 at 4:18 PM, Oliver Walsh wrote: >> Hi Yolanda, >> >>> these changes will be created by ironic before the image is deployed >> >> The question I'm asking is how are the changed created without a reboot? >> >> Typically when setting this via a manual change or via tuned the >> process is to modify /etc/default/grub, run grub2-mkconfig, and then >> reboot. Are you suggesting we drop in a pre-build grub cfg before >> deployment? >> >> Thanks, >> Ollie >> >> On 13 December 2016 at 10:33, Yolanda Robla Mota wrote: >>> It won't need a reboot, because these changes will be created by ironic >>> before the image is deployed. So it will boot with the right parameters. The >>> alternative of doing with puppet after the image was deployed, needed a >>> reboot, because the changes were done post-deploy. >>> So ironic build steps are pre-deploy without reboot, puppet changes are >>> post-deploy with a reboot. >>> >>> On Tue, Dec 13, 2016 at 11:24 AM, Oliver Walsh wrote: >>>> >>>> Hi, >>>> >>>> Saravanan wrote: >>>> > If ironic "deploy steps" can configure this "tuned" setting and run the >>>> > command >>>> >>>> How does this avoid the reboot? >>>> >>>> Yolanda wrote: >>>> > The idea will be to define custom deployment steps for ironic, like >>>> > including the kernel boot parameters. >>>> >>>> Again, is this avoiding the reboot or just moving it? >>>> >>>> Thanks, >>>> Ollie >>>> >>>> On 13 December 2016 at 09:02, Saravanan KR wrote: >>>> > Hi Yolanda, >>>> > >>>> > The flow for "tuned" will be like set the "tuned" configuration files, >>>> > and then activate the profile by running the command "tuned-adm >>>> > tuned-profile-nfv". This command will actually write the required >>>> > configuration files for tuning the host. If ironic "deploy steps" can >>>> > configure this "tuned" setting and run the command, then it is good >>>> > enough. >>>> > >>>> > Regards, >>>> > Saravanan KR >>>> > >>>> > On Tue, Dec 13, 2016 at 1:04 PM, Yolanda Robla Mota >>>> > wrote: >>>> >> Hi Saravanan >>>> >> Thanks for your comments. With this new module, I guess a reboot is >>>> >> still >>>> >> needed after os-host-config ? >>>> >> Right now we have been guided by TripleO and Ironic people to start >>>> >> using >>>> >> what in Ironic is called "custom deployment steps". An initial spec is >>>> >> reflected here: >>>> >> https://review.openstack.org/#/c/382091 >>>> >> >>>> >> The idea will be to define custom deployment steps for ironic, like >>>> >> including the kernel boot parameters. Can that be a solution for your >>>> >> "tuned" needs as well? >>>> >> >>>> >> Best >>>> >> Yolanda >>>> >> >>>> >> On Tue, Dec 13, 2016 at 7:59 AM, Saravanan KR >>>> >> wrote: >>>> >>> >>>> >>> Hello, >>>> >>> >>>> >>> Thanks Yolanda for starting the thread. The list of requirements in >>>> >>> the host configuration, related to boot pa
Re: [openstack-dev] [tripleo] [ironic] Need to update kernel parameters on local boot
Hi Yolanda, > these changes will be created by ironic before the image is deployed The question I'm asking is how are the changed created without a reboot? Typically when setting this via a manual change or via tuned the process is to modify /etc/default/grub, run grub2-mkconfig, and then reboot. Are you suggesting we drop in a pre-build grub cfg before deployment? Thanks, Ollie On 13 December 2016 at 10:33, Yolanda Robla Mota wrote: > It won't need a reboot, because these changes will be created by ironic > before the image is deployed. So it will boot with the right parameters. The > alternative of doing with puppet after the image was deployed, needed a > reboot, because the changes were done post-deploy. > So ironic build steps are pre-deploy without reboot, puppet changes are > post-deploy with a reboot. > > On Tue, Dec 13, 2016 at 11:24 AM, Oliver Walsh wrote: >> >> Hi, >> >> Saravanan wrote: >> > If ironic "deploy steps" can configure this "tuned" setting and run the >> > command >> >> How does this avoid the reboot? >> >> Yolanda wrote: >> > The idea will be to define custom deployment steps for ironic, like >> > including the kernel boot parameters. >> >> Again, is this avoiding the reboot or just moving it? >> >> Thanks, >> Ollie >> >> On 13 December 2016 at 09:02, Saravanan KR wrote: >> > Hi Yolanda, >> > >> > The flow for "tuned" will be like set the "tuned" configuration files, >> > and then activate the profile by running the command "tuned-adm >> > tuned-profile-nfv". This command will actually write the required >> > configuration files for tuning the host. If ironic "deploy steps" can >> > configure this "tuned" setting and run the command, then it is good >> > enough. >> > >> > Regards, >> > Saravanan KR >> > >> > On Tue, Dec 13, 2016 at 1:04 PM, Yolanda Robla Mota >> > wrote: >> >> Hi Saravanan >> >> Thanks for your comments. With this new module, I guess a reboot is >> >> still >> >> needed after os-host-config ? >> >> Right now we have been guided by TripleO and Ironic people to start >> >> using >> >> what in Ironic is called "custom deployment steps". An initial spec is >> >> reflected here: >> >> https://review.openstack.org/#/c/382091 >> >> >> >> The idea will be to define custom deployment steps for ironic, like >> >> including the kernel boot parameters. Can that be a solution for your >> >> "tuned" needs as well? >> >> >> >> Best >> >> Yolanda >> >> >> >> On Tue, Dec 13, 2016 at 7:59 AM, Saravanan KR >> >> wrote: >> >>> >> >>> Hello, >> >>> >> >>> Thanks Yolanda for starting the thread. The list of requirements in >> >>> the host configuration, related to boot parameters and reboot are: >> >>> >> >>> * DPDK - For vfio-pci driver binding, iommu support on kernel args is >> >>> mandatory, which has to be configured before os-net-config runs >> >>> * DPDK & RealTime - Enabling "tuned" profile for nfv or rt, will >> >>> update the boot parameters and a reboot is required >> >>> * Other items mentioned by Yolanda >> >>> >> >>> If it is configuring only, the boot parameters, then ironic's deploy >> >>> feature may help, but there are other requirement to enable the >> >>> "tuned" profile which tunes the host for the required configuration, >> >>> which also requires reboot, as it will alter the boot parameters. If >> >>> we can collate the all the configurations which requires reboot >> >>> together, we will improve the reboot time. And if we reboot before the >> >>> actual openstack services are started, then the reboot time _may_ >> >>> improve. >> >>> >> >>> Can I propose a *new* module for TripleO deployments, like > >> >>> os-host-config <, which will run after os-collect-config and before >> >>> os-net-config, then we can collate all the host specific configuration >> >>> inside this module. This module can be a set of ansible scripts, which >> >>> will only configure the host. Ofcource the parameter to this module >> >>> should be provide
Re: [openstack-dev] [tripleo] [ironic] Need to update kernel parameters on local boot
Hi, Saravanan wrote: > If ironic "deploy steps" can configure this "tuned" setting and run the > command How does this avoid the reboot? Yolanda wrote: > The idea will be to define custom deployment steps for ironic, like including > the kernel boot parameters. Again, is this avoiding the reboot or just moving it? Thanks, Ollie On 13 December 2016 at 09:02, Saravanan KR wrote: > Hi Yolanda, > > The flow for "tuned" will be like set the "tuned" configuration files, > and then activate the profile by running the command "tuned-adm > tuned-profile-nfv". This command will actually write the required > configuration files for tuning the host. If ironic "deploy steps" can > configure this "tuned" setting and run the command, then it is good > enough. > > Regards, > Saravanan KR > > On Tue, Dec 13, 2016 at 1:04 PM, Yolanda Robla Mota > wrote: >> Hi Saravanan >> Thanks for your comments. With this new module, I guess a reboot is still >> needed after os-host-config ? >> Right now we have been guided by TripleO and Ironic people to start using >> what in Ironic is called "custom deployment steps". An initial spec is >> reflected here: >> https://review.openstack.org/#/c/382091 >> >> The idea will be to define custom deployment steps for ironic, like >> including the kernel boot parameters. Can that be a solution for your >> "tuned" needs as well? >> >> Best >> Yolanda >> >> On Tue, Dec 13, 2016 at 7:59 AM, Saravanan KR wrote: >>> >>> Hello, >>> >>> Thanks Yolanda for starting the thread. The list of requirements in >>> the host configuration, related to boot parameters and reboot are: >>> >>> * DPDK - For vfio-pci driver binding, iommu support on kernel args is >>> mandatory, which has to be configured before os-net-config runs >>> * DPDK & RealTime - Enabling "tuned" profile for nfv or rt, will >>> update the boot parameters and a reboot is required >>> * Other items mentioned by Yolanda >>> >>> If it is configuring only, the boot parameters, then ironic's deploy >>> feature may help, but there are other requirement to enable the >>> "tuned" profile which tunes the host for the required configuration, >>> which also requires reboot, as it will alter the boot parameters. If >>> we can collate the all the configurations which requires reboot >>> together, we will improve the reboot time. And if we reboot before the >>> actual openstack services are started, then the reboot time _may_ >>> improve. >>> >>> Can I propose a *new* module for TripleO deployments, like > >>> os-host-config <, which will run after os-collect-config and before >>> os-net-config, then we can collate all the host specific configuration >>> inside this module. This module can be a set of ansible scripts, which >>> will only configure the host. Ofcource the parameter to this module >>> should be provided via os-collect-config. Separating the host >>> configuration will help in the containerized TripleO deployment also. >>> >>> Or any other better alternatives are welcome. >>> >>> Please pour in your views if you think for/against it. >>> >>> Regards, >>> Saravanan KR >>> >>> >>> On Fri, Dec 2, 2016 at 9:31 PM, Yolanda Robla Mota >>> wrote: >>> > Hi , Dmitry >>> > That's what i didn't get very clear. If all the deployment steps are >>> > pre-imaging as that statement says, or every deploy step could be isolated >>> > and configured somehow. >>> > I'm also a bit confused with that spec, because it mixes the concept of >>> > "deployment steps", will all the changes needed for runtime RAID. Could it >>> > be possible to separate into two separate ones? >>> > >>> > - Original Message - >>> > From: "Dmitry Tantsur" >>> > To: openstack-dev@lists.openstack.org >>> > Sent: Friday, December 2, 2016 3:51:30 PM >>> > Subject: Re: [openstack-dev] [tripleo] [ironic] Need to update kernel >>> > parameters on local boot >>> > >>> > On 12/02/2016 01:28 PM, Yolanda Robla Mota wrote: >>> >> Hi Dmitry >>> >> >>> >> So we've been looking at that spec you suggested, but we are wondering >>> >> if that will be useful for our use case. As the text says: >>> >> >>> >> The ``ironic-python-agent`` project and ``agent`` driver will be >>> >> adjusted to >>> >> support ``get_deploy_steps``. That way, ``ironic-python-agent`` will be >>> >> able >>> >> to declare deploy steps to run prior to disk imaging, and operators >>> >> will be >>> >> able to extend ``ironic-python-agent`` to add any custom step. >>> >> >>> >> Our needs are different, actually we need to create a deployment step >>> >> after imaging. We'd need an step that drops config on /etc/default/grub , >>> >> and updates it. This is a post-imaging deploy step, that modifies the >>> >> base >>> >> image. Could ironic support these kind of steps, if there is a base >>> >> system >>> >> to just define per-user steps? >>> > >>> > I thought that all deployment operations are converted to steps, with >>> > partitioning, writing the image, writing the configdrive and installing >>> > the boot >>>
Re: [openstack-dev] [tripleo] [ironic] Need to update kernel parameters on local boot
Hi Yolanda, I've the same requirements for a real-time compute role. I'm current using #4. Puppet sets the kernel cmdline in /etc/defaults/grub, then touches a file that triggers a reboot in an os-refresh-config post-configure script. How much of an issue is the reboot? In my dev env it doesn't make a significant difference to the overall deployment time of a node. I'm thinking we could use #4 as the general solution. In cases where a reboot is too expensive then #1/#2/#3 could be used to prime the image with same /etc/default/grub that puppet would create. As puppet doesn't change it doesn't trigger a reboot. Re custom images. Support in python-tripleoclient could be improved but for now this is what I'm doing: Uploading a custom image: OS_IMAGE=custom-overcloud-full.qcow2 openstack overcloud image upload Set the Image for the custom role in param_defaults: parameter_defaults: CustomRoleImage: custom-overcloud-full Thanks, Ollie On 28 November 2016 at 15:36, Yolanda Robla Mota wrote: > Hi, good afternoon > > I wanted to start an email thread about how to properly setup kernel > parameters on local boot, for our overcloud images on TripleO. > These parameters may vary depending on the needs of our end users, and even > can be different ( for different roles ) per deployment. As an example, we > need it for: > - enable FIPS kernel in terms of security > (https://bugs.launchpad.net/tripleo/+bug/1640235) > - enable functionality for DPDK/SR-IOV > (https://review.openstack.org/#/c/331564/) > - enable rd.iscsi.firmware=1 flag (this for the ramdisk image) > - etc.. > > So far, the solutions we got were on several directions: > > 1. Update the golden overcloud-full image with virt-customize, modifying > /etc/default/grub settings according to our needs: this is a manual process, > not really driven by TripleO. End users will want to avoid manual steps as > much as possible. Also if we announce that OpenStack ships features in > TripleO like DPDK, SR-IOV... doesn't make sense to tell end users that if > they want to consume that feature, they need to do manual updates on the > image. It shall be natively supported, or configurable per TripleO > environments. > > 2. Create our own images using diskimage-builder and custom elements: in this > case, we have the problem that the partners will loose support, as building > their own images is good for upstream, but not accepted into the OSP > environment. Also the combination of images needed can be huge, that can be a > blocker for QA. > > 3. Add Ironic support for it. Images can be uploaded to glance, and some > properties can be set on metadata, like a json with kernel parameters. Ironic > will modify these kernel parameters when deploying the image (in a similar > way that when it installs bootloader, or generates partitions). > > 4. Configure it post-deployment: there can be some puppet element that > updates kernel parameters. But it will need a node reboot to be applied, and > it's very far from being optimal and acceptable for the end users. Reboots > are slow, they can be a problem depending on the number of nodes/hardware, > and also the timing of reboot shall be totally controlled (after all puppet > has been applied properly). > > > In the first three cases, we also hit the problem that TripleO only accepts > one single overcloud image for all deployments - there is no way to instruct > TripleO to upload and use several images, depending on the node type > (although Ironic supports it). Also, we are worried about upgrade paths if we > do image customizations. We need a clear way to move forward on it. > > So, we'd like to discuss the possible options there and the action items to > take (raise bugs, create some blueprints...). To summarize, our end goal is > the following: > > - need to map overcloud-full images to roles > - need to be done in an automated way, no manual steps enforced, and in a way > that can pass properly quality controls > - reboots are sub-optimal > > What are your thoughts there? > > Best, > > > Yolanda Robla > yrobl...@redhat.com > Principal Software Engineer - NFV Partner Engineer > > > __ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [tripleo][spec] Feedback wanted on TripleO real-time compute node proposal
Hi, I'd like to post a link to a blueprint/spec that I'm hoping to discuss at the upcoming summit. https://blueprints.launchpad.net/tripleo/+spec/tripleo-realtime https://review.openstack.org/388162 I'm new to both TripleO and OpenStack development. Any advice/hints/criticism is very much appreciated. Thanks, Ollie __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
