Hi Yolanda, > these changes will be created by ironic before the image is deployed
The question I'm asking is how are the changed created without a reboot? Typically when setting this via a manual change or via tuned the process is to modify /etc/default/grub, run grub2-mkconfig, and then reboot. Are you suggesting we drop in a pre-build grub cfg before deployment? Thanks, Ollie On 13 December 2016 at 10:33, Yolanda Robla Mota <[email protected]> wrote: > It won't need a reboot, because these changes will be created by ironic > before the image is deployed. So it will boot with the right parameters. The > alternative of doing with puppet after the image was deployed, needed a > reboot, because the changes were done post-deploy. > So ironic build steps are pre-deploy without reboot, puppet changes are > post-deploy with a reboot. > > On Tue, Dec 13, 2016 at 11:24 AM, Oliver Walsh <[email protected]> wrote: >> >> Hi, >> >> Saravanan wrote: >> > If ironic "deploy steps" can configure this "tuned" setting and run the >> > command >> >> How does this avoid the reboot? >> >> Yolanda wrote: >> > The idea will be to define custom deployment steps for ironic, like >> > including the kernel boot parameters. >> >> Again, is this avoiding the reboot or just moving it? >> >> Thanks, >> Ollie >> >> On 13 December 2016 at 09:02, Saravanan KR <[email protected]> wrote: >> > Hi Yolanda, >> > >> > The flow for "tuned" will be like set the "tuned" configuration files, >> > and then activate the profile by running the command "tuned-adm >> > tuned-profile-nfv". This command will actually write the required >> > configuration files for tuning the host. If ironic "deploy steps" can >> > configure this "tuned" setting and run the command, then it is good >> > enough. >> > >> > Regards, >> > Saravanan KR >> > >> > On Tue, Dec 13, 2016 at 1:04 PM, Yolanda Robla Mota >> > <[email protected]> wrote: >> >> Hi Saravanan >> >> Thanks for your comments. With this new module, I guess a reboot is >> >> still >> >> needed after os-host-config ? >> >> Right now we have been guided by TripleO and Ironic people to start >> >> using >> >> what in Ironic is called "custom deployment steps". An initial spec is >> >> reflected here: >> >> https://review.openstack.org/#/c/382091 >> >> >> >> The idea will be to define custom deployment steps for ironic, like >> >> including the kernel boot parameters. Can that be a solution for your >> >> "tuned" needs as well? >> >> >> >> Best >> >> Yolanda >> >> >> >> On Tue, Dec 13, 2016 at 7:59 AM, Saravanan KR <[email protected]> >> >> wrote: >> >>> >> >>> Hello, >> >>> >> >>> Thanks Yolanda for starting the thread. The list of requirements in >> >>> the host configuration, related to boot parameters and reboot are: >> >>> >> >>> * DPDK - For vfio-pci driver binding, iommu support on kernel args is >> >>> mandatory, which has to be configured before os-net-config runs >> >>> * DPDK & RealTime - Enabling "tuned" profile for nfv or rt, will >> >>> update the boot parameters and a reboot is required >> >>> * Other items mentioned by Yolanda >> >>> >> >>> If it is configuring only, the boot parameters, then ironic's deploy >> >>> feature may help, but there are other requirement to enable the >> >>> "tuned" profile which tunes the host for the required configuration, >> >>> which also requires reboot, as it will alter the boot parameters. If >> >>> we can collate the all the configurations which requires reboot >> >>> together, we will improve the reboot time. And if we reboot before the >> >>> actual openstack services are started, then the reboot time _may_ >> >>> improve. >> >>> >> >>> Can I propose a *new* module for TripleO deployments, like > >> >>> os-host-config <, which will run after os-collect-config and before >> >>> os-net-config, then we can collate all the host specific configuration >> >>> inside this module. This module can be a set of ansible scripts, which >> >>> will only configure the host. Ofcource the parameter to this module >> >>> should be provided via os-collect-config. Separating the host >> >>> configuration will help in the containerized TripleO deployment also. >> >>> >> >>> Or any other better alternatives are welcome. >> >>> >> >>> Please pour in your views if you think for/against it. >> >>> >> >>> Regards, >> >>> Saravanan KR >> >>> >> >>> >> >>> On Fri, Dec 2, 2016 at 9:31 PM, Yolanda Robla Mota >> >>> <[email protected]> >> >>> wrote: >> >>> > Hi , Dmitry >> >>> > That's what i didn't get very clear. If all the deployment steps are >> >>> > pre-imaging as that statement says, or every deploy step could be >> >>> > isolated >> >>> > and configured somehow. >> >>> > I'm also a bit confused with that spec, because it mixes the concept >> >>> > of >> >>> > "deployment steps", will all the changes needed for runtime RAID. >> >>> > Could it >> >>> > be possible to separate into two separate ones? >> >>> > >> >>> > ----- Original Message ----- >> >>> > From: "Dmitry Tantsur" <[email protected]> >> >>> > To: [email protected] >> >>> > Sent: Friday, December 2, 2016 3:51:30 PM >> >>> > Subject: Re: [openstack-dev] [tripleo] [ironic] Need to update >> >>> > kernel >> >>> > parameters on local boot >> >>> > >> >>> > On 12/02/2016 01:28 PM, Yolanda Robla Mota wrote: >> >>> >> Hi Dmitry >> >>> >> >> >>> >> So we've been looking at that spec you suggested, but we are >> >>> >> wondering >> >>> >> if that will be useful for our use case. As the text says: >> >>> >> >> >>> >> The ``ironic-python-agent`` project and ``agent`` driver will be >> >>> >> adjusted to >> >>> >> support ``get_deploy_steps``. That way, ``ironic-python-agent`` >> >>> >> will be >> >>> >> able >> >>> >> to declare deploy steps to run prior to disk imaging, and operators >> >>> >> will be >> >>> >> able to extend ``ironic-python-agent`` to add any custom step. >> >>> >> >> >>> >> Our needs are different, actually we need to create a deployment >> >>> >> step >> >>> >> after imaging. We'd need an step that drops config on >> >>> >> /etc/default/grub , >> >>> >> and updates it. This is a post-imaging deploy step, that modifies >> >>> >> the base >> >>> >> image. Could ironic support these kind of steps, if there is a base >> >>> >> system >> >>> >> to just define per-user steps? >> >>> > >> >>> > I thought that all deployment operations are converted to steps, >> >>> > with >> >>> > partitioning, writing the image, writing the configdrive and >> >>> > installing >> >>> > the boot >> >>> > loader being four default ones (as you see, two steps actually >> >>> > happen >> >>> > after the >> >>> > image is written). >> >>> > >> >>> >> >> >>> >> The idea we had on mind is: >> >>> >> - from tripleo, add a property to each flavor, that defines the >> >>> >> boot >> >>> >> parameters: openstack flavor set compute --property >> >>> >> os:kernel_boot_params='abc' >> >>> >> - define a "ironic post-imaging deploy step", that will grab this >> >>> >> property from the flavor, drop it on /etc/default/grub and >> >>> >> regenerate it >> >>> >> - then on local boot, the proper kernel parameters will be applied >> >>> >> >> >>> >> What is your feedback there? >> >>> >> >> >>> >> ----- Original Message ----- >> >>> >> From: "Dmitry Tantsur" <[email protected]> >> >>> >> To: [email protected] >> >>> >> Sent: Friday, December 2, 2016 12:44:29 PM >> >>> >> Subject: Re: [openstack-dev] [tripleo] [ironic] Need to update >> >>> >> kernel >> >>> >> parameters on local boot >> >>> >> >> >>> >> On 11/28/2016 04:46 PM, Jay Faulkner wrote: >> >>> >>> >> >>> >>>> On Nov 28, 2016, at 7:36 AM, Yolanda Robla Mota >> >>> >>>> <[email protected]> >> >>> >>>> wrote: >> >>> >>>> >> >>> >>>> Hi, good afternoon >> >>> >>>> >> >>> >>>> I wanted to start an email thread about how to properly setup >> >>> >>>> kernel >> >>> >>>> parameters on local boot, for our overcloud images on TripleO. >> >>> >>>> These parameters may vary depending on the needs of our end >> >>> >>>> users, >> >>> >>>> and even can be different ( for different roles ) per deployment. >> >>> >>>> As an >> >>> >>>> example, we need it for: >> >>> >>>> - enable FIPS kernel in terms of security >> >>> >>>> (https://bugs.launchpad.net/tripleo/+bug/1640235) >> >>> >>>> - enable functionality for DPDK/SR-IOV >> >>> >>>> (https://review.openstack.org/#/c/331564/) >> >>> >>>> - enable rd.iscsi.firmware=1 flag (this for the ramdisk image) >> >>> >>>> - etc.. >> >>> >>>> >> >>> >>>> So far, the solutions we got were on several directions: >> >>> >>>> >> >>> >>>> 1. Update the golden overcloud-full image with virt-customize, >> >>> >>>> modifying /etc/default/grub settings according to our needs: this >> >>> >>>> is a >> >>> >>>> manual process, not really driven by TripleO. End users will want >> >>> >>>> to avoid >> >>> >>>> manual steps as much as possible. Also if we announce that >> >>> >>>> OpenStack ships >> >>> >>>> features in TripleO like DPDK, SR-IOV... doesn't make sense to >> >>> >>>> tell end >> >>> >>>> users that if they want to consume that feature, they need to do >> >>> >>>> manual >> >>> >>>> updates on the image. It shall be natively supported, or >> >>> >>>> configurable per >> >>> >>>> TripleO environments. >> >>> >>>> >> >>> >>>> 2. Create our own images using diskimage-builder and custom >> >>> >>>> elements: >> >>> >>>> in this case, we have the problem that the partners will loose >> >>> >>>> support, as >> >>> >>>> building their own images is good for upstream, but not accepted >> >>> >>>> into the >> >>> >>>> OSP environment. Also the combination of images needed can be >> >>> >>>> huge, that can >> >>> >>>> be a blocker for QA. >> >>> >>>> >> >>> >>>> 3. Add Ironic support for it. Images can be uploaded to glance, >> >>> >>>> and >> >>> >>>> some properties can be set on metadata, like a json with kernel >> >>> >>>> parameters. >> >>> >>>> Ironic will modify these kernel parameters when deploying the >> >>> >>>> image (in a >> >>> >>>> similar way that when it installs bootloader, or generates >> >>> >>>> partitions). >> >>> >>>> >> >>> >>> >> >>> >>> This has been proposed before in ironic-specs >> >>> >>> (https://review.openstack.org/#/c/331564/) and was rejected, as it >> >>> >>> would >> >>> >>> require Ironic to reach out and modify image contents, which >> >>> >>> traditionally >> >>> >>> has been considered out of scope for Ironic. I would personally >> >>> >>> recommend >> >>> >>> #4, as post-boot automation is the safest way to configure >> >>> >>> node-specific >> >>> >>> options inside an image. >> >>> >> >> >>> >> I'm still a bit divided about our decision back then.. On one hand, >> >>> >> this does >> >>> >> seem somewhat out of scope. On the other, I quite understand why >> >>> >> reboot >> >>> >> is >> >>> >> suboptimal. I wonder if the ongoing deploy steps work will actually >> >>> >> solve it by >> >>> >> allowing hardware managers to provide additional deploy steps. >> >>> >> >> >>> >> Yolanda, you may want to check the spec >> >>> >> https://review.openstack.org/#/c/382091/ >> >>> >> as it lays the foundation for the deploy steps idea. >> >>> >> >> >>> >>> >> >>> >>> Thanks, >> >>> >>> Jay Faulkner >> >>> >>> OSIC >> >>> >>> >> >>> >>> >> >>> >>>> 4. Configure it post-deployment: there can be some puppet element >> >>> >>>> that updates kernel parameters. But it will need a node reboot to >> >>> >>>> be >> >>> >>>> applied, and it's very far from being optimal and acceptable for >> >>> >>>> the end >> >>> >>>> users. Reboots are slow, they can be a problem depending on the >> >>> >>>> number of >> >>> >>>> nodes/hardware, and also the timing of reboot shall be totally >> >>> >>>> controlled >> >>> >>>> (after all puppet has been applied properly). >> >>> >>>> >> >>> >>>> >> >>> >>>> In the first three cases, we also hit the problem that TripleO >> >>> >>>> only >> >>> >>>> accepts one single overcloud image for all deployments - there is >> >>> >>>> no way to >> >>> >>>> instruct TripleO to upload and use several images, depending on >> >>> >>>> the node >> >>> >>>> type (although Ironic supports it). Also, we are worried about >> >>> >>>> upgrade paths >> >>> >>>> if we do image customizations. We need a clear way to move >> >>> >>>> forward on it. >> >>> >>>> >> >>> >>>> So, we'd like to discuss the possible options there and the >> >>> >>>> action >> >>> >>>> items to take (raise bugs, create some blueprints...). To >> >>> >>>> summarize, our end >> >>> >>>> goal is the following: >> >>> >>>> >> >>> >>>> - need to map overcloud-full images to roles >> >>> >>>> - need to be done in an automated way, no manual steps enforced, >> >>> >>>> and >> >>> >>>> in a way that can pass properly quality controls >> >>> >>>> - reboots are sub-optimal >> >>> >>>> >> >>> >>>> What are your thoughts there? >> >>> >>>> >> >>> >>>> Best, >> >>> >>>> >> >>> >>>> >> >>> >>>> Yolanda Robla >> >>> >>>> [email protected] >> >>> >>>> Principal Software Engineer - NFV Partner Engineer >> >>> >>>> >> >>> >>>> >> >>> >>>> >> >>> >>>> >> >>> >>>> __________________________________________________________________________ >> >>> >>>> OpenStack Development Mailing List (not for usage questions) >> >>> >>>> Unsubscribe: >> >>> >>>> [email protected]?subject:unsubscribe >> >>> >>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >> >>> >>> >> >>> >>> >> >>> >>> >> >>> >>> >> >>> >>> __________________________________________________________________________ >> >>> >>> OpenStack Development Mailing List (not for usage questions) >> >>> >>> Unsubscribe: >> >>> >>> [email protected]?subject:unsubscribe >> >>> >>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >> >>> >>> >> >>> >> >> >>> >> >> >>> >> >> >>> >> >> >>> >> __________________________________________________________________________ >> >>> >> OpenStack Development Mailing List (not for usage questions) >> >>> >> Unsubscribe: >> >>> >> [email protected]?subject:unsubscribe >> >>> >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >> >>> >> >> >>> >> >> >>> >> >> >>> >> __________________________________________________________________________ >> >>> >> OpenStack Development Mailing List (not for usage questions) >> >>> >> Unsubscribe: >> >>> >> [email protected]?subject:unsubscribe >> >>> >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >> >>> >> >> >>> > >> >>> > >> >>> > >> >>> > >> >>> > __________________________________________________________________________ >> >>> > OpenStack Development Mailing List (not for usage questions) >> >>> > Unsubscribe: >> >>> > [email protected]?subject:unsubscribe >> >>> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >> >>> > >> >>> > >> >>> > >> >>> > __________________________________________________________________________ >> >>> > OpenStack Development Mailing List (not for usage questions) >> >>> > Unsubscribe: >> >>> > [email protected]?subject:unsubscribe >> >>> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >> >>> >> >>> >> >>> __________________________________________________________________________ >> >>> OpenStack Development Mailing List (not for usage questions) >> >>> Unsubscribe: >> >>> [email protected]?subject:unsubscribe >> >>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >> >> >> >> >> >> >> >> >> >> -- >> >> Yolanda Robla Mota >> >> NFV Partner Engineer >> >> [email protected] >> >> +34 605641639 >> >> >> >> >> >> __________________________________________________________________________ >> >> OpenStack Development Mailing List (not for usage questions) >> >> Unsubscribe: >> >> [email protected]?subject:unsubscribe >> >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >> >> >> > >> > >> > __________________________________________________________________________ >> > OpenStack Development Mailing List (not for usage questions) >> > Unsubscribe: >> > [email protected]?subject:unsubscribe >> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >> >> __________________________________________________________________________ >> OpenStack Development Mailing List (not for usage questions) >> Unsubscribe: [email protected]?subject:unsubscribe >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > > > > > -- > Yolanda Robla Mota > NFV Partner Engineer > [email protected] > +34 605641639 > > __________________________________________________________________________ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: [email protected]?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > __________________________________________________________________________ OpenStack Development Mailing List (not for usage questions) Unsubscribe: [email protected]?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
