Re: [openstack-dev] [tripleo] [ironic] Need to update kernel parameters on local boot

[Yolanda Robla Mota]

;> needed after os-host-config ?
> >>>> >> Right now we have been guided by TripleO and Ironic people to start
> >>>> >> using
> >>>> >> what in Ironic is called "custom deployment steps". An initial
> spec is
> >>>> >> reflected here:
> >>>> >> https://review.openstack.org/#/c/382091
> >>>> >>
> >>>> >> The idea will be to define custom deployment steps for ironic, like
> >>>> >> including the kernel boot parameters. Can that be a solution for
> your
> >>>> >> "tuned" needs as well?
> >>>> >>
> >>>> >> Best
> >>>> >> Yolanda
> >>>> >>
> >>>> >> On Tue, Dec 13, 2016 at 7:59 AM, Saravanan KR <skram...@redhat.com
> >
> >>>> >> wrote:
> >>>> >>>
> >>>> >>> Hello,
> >>>> >>>
> >>>> >>> Thanks Yolanda for starting the thread. The list of requirements
> in
> >>>> >>> the host configuration, related to boot parameters and reboot are:
> >>>> >>>
> >>>> >>> * DPDK - For vfio-pci driver binding, iommu support on kernel
> args is
> >>>> >>> mandatory, which has to be configured before os-net-config runs
> >>>> >>> * DPDK & RealTime - Enabling "tuned" profile for nfv or rt, will
> >>>> >>> update the boot parameters and a reboot is required
> >>>> >>> * Other items mentioned by Yolanda
> >>>> >>>
> >>>> >>> If it is configuring only, the boot parameters, then ironic's
> deploy
> >>>> >>> feature may help, but there are other requirement to enable the
> >>>> >>> "tuned" profile which tunes the host for the required
> configuration,
> >>>> >>> which also requires reboot, as it will alter the boot parameters.
> If
> >>>> >>> we can collate the all the configurations which requires reboot
> >>>> >>> together, we will improve the reboot time. And if we reboot
> before the
> >>>> >>> actual openstack services are started, then the reboot time _may_
> >>>> >>> improve.
> >>>> >>>
> >>>> >>> Can I propose a *new* module for TripleO deployments, like >
> >>>> >>> os-host-config <, which will run after os-collect-config and
> before
> >>>> >>> os-net-config, then we can collate all the host specific
> configuration
> >>>> >>> inside this module. This module can be a set of ansible scripts,
> which
> >>>> >>> will only configure the host. Ofcource the parameter to this
> module
> >>>> >>> should be provided via os-collect-config. Separating the host
> >>>> >>> configuration will help in the containerized TripleO deployment
> also.
> >>>> >>>
> >>>> >>> Or any other better alternatives are welcome.
> >>>> >>>
> >>>> >>> Please pour in your views if you think for/against it.
> >>>> >>>
> >>>> >>> Regards,
> >>>> >>> Saravanan KR
> >>>> >>>
> >>>> >>>
> >>>> >>> On Fri, Dec 2, 2016 at 9:31 PM, Yolanda Robla Mota
> >>>> >>> <yrobl...@redhat.com>
> >>>> >>> wrote:
> >>>> >>> > Hi , Dmitry
> >>>> >>> > That's what i didn't get very clear. If all the deployment
> steps are
> >>>> >>> > pre-imaging as that statement says, or every deploy step could
> be
> >>>> >>> > isolated
> >>>> >>> > and configured somehow.
> >>>> >>> > I'm also a bit confused with that spec, because it mixes the
> concept
> >>>> >>> > of
> >>>> >>> > "deployment steps", will all the changes needed for runtime
> RAID.
> >>>> >>> > Could it
> >>>> >>> > be possible to separate into two separate ones?
> >>>> >>> >
> >>>> >>> > - Original Message -
> >>&

Re: [openstack-dev] [tripleo] [ironic] Need to update kernel parameters on local boot

[Oliver Walsh]

 >>> Thanks Yolanda for starting the thread. The list of requirements in
>>>> >>> the host configuration, related to boot parameters and reboot are:
>>>> >>>
>>>> >>> * DPDK - For vfio-pci driver binding, iommu support on kernel args is
>>>> >>> mandatory, which has to be configured before os-net-config runs
>>>> >>> * DPDK & RealTime - Enabling "tuned" profile for nfv or rt, will
>>>> >>> update the boot parameters and a reboot is required
>>>> >>> * Other items mentioned by Yolanda
>>>> >>>
>>>> >>> If it is configuring only, the boot parameters, then ironic's deploy
>>>> >>> feature may help, but there are other requirement to enable the
>>>> >>> "tuned" profile which tunes the host for the required configuration,
>>>> >>> which also requires reboot, as it will alter the boot parameters. If
>>>> >>> we can collate the all the configurations which requires reboot
>>>> >>> together, we will improve the reboot time. And if we reboot before the
>>>> >>> actual openstack services are started, then the reboot time _may_
>>>> >>> improve.
>>>> >>>
>>>> >>> Can I propose a *new* module for TripleO deployments, like >
>>>> >>> os-host-config <, which will run after os-collect-config and before
>>>> >>> os-net-config, then we can collate all the host specific configuration
>>>> >>> inside this module. This module can be a set of ansible scripts, which
>>>> >>> will only configure the host. Ofcource the parameter to this module
>>>> >>> should be provided via os-collect-config. Separating the host
>>>> >>> configuration will help in the containerized TripleO deployment also.
>>>> >>>
>>>> >>> Or any other better alternatives are welcome.
>>>> >>>
>>>> >>> Please pour in your views if you think for/against it.
>>>> >>>
>>>> >>> Regards,
>>>> >>> Saravanan KR
>>>> >>>
>>>> >>>
>>>> >>> On Fri, Dec 2, 2016 at 9:31 PM, Yolanda Robla Mota
>>>> >>> <yrobl...@redhat.com>
>>>> >>> wrote:
>>>> >>> > Hi , Dmitry
>>>> >>> > That's what i didn't get very clear. If all the deployment steps are
>>>> >>> > pre-imaging as that statement says, or every deploy step could be
>>>> >>> > isolated
>>>> >>> > and configured somehow.
>>>> >>> > I'm also a bit confused with that spec, because it mixes the concept
>>>> >>> > of
>>>> >>> > "deployment steps", will all the changes needed for runtime RAID.
>>>> >>> > Could it
>>>> >>> > be possible to separate into two separate ones?
>>>> >>> >
>>>> >>> > - Original Message -
>>>> >>> > From: "Dmitry Tantsur" <dtant...@redhat.com>
>>>> >>> > To: openstack-dev@lists.openstack.org
>>>> >>> > Sent: Friday, December 2, 2016 3:51:30 PM
>>>> >>> > Subject: Re: [openstack-dev] [tripleo] [ironic] Need to update
>>>> >>> > kernel
>>>> >>> > parameters on local boot
>>>> >>> >
>>>> >>> > On 12/02/2016 01:28 PM, Yolanda Robla Mota wrote:
>>>> >>> >> Hi Dmitry
>>>> >>> >>
>>>> >>> >> So we've been looking at that spec you suggested, but we are
>>>> >>> >> wondering
>>>> >>> >> if that will be useful for our use case. As the text says:
>>>> >>> >>
>>>> >>> >> The ``ironic-python-agent`` project and ``agent`` driver will be
>>>> >>> >> adjusted to
>>>> >>> >> support ``get_deploy_steps``. That way, ``ironic-python-agent``
>>>> >>> >> will be
>>>> >>> >> able
>>>> >>> >> to declare deploy steps to run prior to disk imaging, and operators
>>>> >>> >> will be
>>>> &g

Re: [openstack-dev] [tripleo] [ironic] Need to update kernel parameters on local boot

[Saravanan KR]

;>> >>> feature may help, but there are other requirement to enable the
>>> >>> "tuned" profile which tunes the host for the required configuration,
>>> >>> which also requires reboot, as it will alter the boot parameters. If
>>> >>> we can collate the all the configurations which requires reboot
>>> >>> together, we will improve the reboot time. And if we reboot before the
>>> >>> actual openstack services are started, then the reboot time _may_
>>> >>> improve.
>>> >>>
>>> >>> Can I propose a *new* module for TripleO deployments, like >
>>> >>> os-host-config <, which will run after os-collect-config and before
>>> >>> os-net-config, then we can collate all the host specific configuration
>>> >>> inside this module. This module can be a set of ansible scripts, which
>>> >>> will only configure the host. Ofcource the parameter to this module
>>> >>> should be provided via os-collect-config. Separating the host
>>> >>> configuration will help in the containerized TripleO deployment also.
>>> >>>
>>> >>> Or any other better alternatives are welcome.
>>> >>>
>>> >>> Please pour in your views if you think for/against it.
>>> >>>
>>> >>> Regards,
>>> >>> Saravanan KR
>>> >>>
>>> >>>
>>> >>> On Fri, Dec 2, 2016 at 9:31 PM, Yolanda Robla Mota
>>> >>> <yrobl...@redhat.com>
>>> >>> wrote:
>>> >>> > Hi , Dmitry
>>> >>> > That's what i didn't get very clear. If all the deployment steps are
>>> >>> > pre-imaging as that statement says, or every deploy step could be
>>> >>> > isolated
>>> >>> > and configured somehow.
>>> >>> > I'm also a bit confused with that spec, because it mixes the concept
>>> >>> > of
>>> >>> > "deployment steps", will all the changes needed for runtime RAID.
>>> >>> > Could it
>>> >>> > be possible to separate into two separate ones?
>>> >>> >
>>> >>> > - Original Message -
>>> >>> > From: "Dmitry Tantsur" <dtant...@redhat.com>
>>> >>> > To: openstack-dev@lists.openstack.org
>>> >>> > Sent: Friday, December 2, 2016 3:51:30 PM
>>> >>> > Subject: Re: [openstack-dev] [tripleo] [ironic] Need to update
>>> >>> > kernel
>>> >>> > parameters on local boot
>>> >>> >
>>> >>> > On 12/02/2016 01:28 PM, Yolanda Robla Mota wrote:
>>> >>> >> Hi Dmitry
>>> >>> >>
>>> >>> >> So we've been looking at that spec you suggested, but we are
>>> >>> >> wondering
>>> >>> >> if that will be useful for our use case. As the text says:
>>> >>> >>
>>> >>> >> The ``ironic-python-agent`` project and ``agent`` driver will be
>>> >>> >> adjusted to
>>> >>> >> support ``get_deploy_steps``. That way, ``ironic-python-agent``
>>> >>> >> will be
>>> >>> >> able
>>> >>> >> to declare deploy steps to run prior to disk imaging, and operators
>>> >>> >> will be
>>> >>> >> able to extend ``ironic-python-agent`` to add any custom step.
>>> >>> >>
>>> >>> >> Our needs are different, actually we need to create a deployment
>>> >>> >> step
>>> >>> >> after imaging. We'd need an step that drops config on
>>> >>> >> /etc/default/grub ,
>>> >>> >> and updates it. This is a post-imaging deploy step, that modifies
>>> >>> >> the base
>>> >>> >> image. Could ironic support these kind of steps, if there is a base
>>> >>> >> system
>>> >>> >> to just define per-user steps?
>>> >>> >
>>> >>> > I thought that all deployment operations are converted to steps,
>>> >>> > with
>>> >>> > partitioning, writing the image, writing the configdrive and
>>> >>> &

Re: [openstack-dev] [tripleo] [ironic] Need to update kernel parameters on local boot

[Oliver Walsh]

gt;>> will only configure the host. Ofcource the parameter to this module
>> >>> should be provided via os-collect-config. Separating the host
>> >>> configuration will help in the containerized TripleO deployment also.
>> >>>
>> >>> Or any other better alternatives are welcome.
>> >>>
>> >>> Please pour in your views if you think for/against it.
>> >>>
>> >>> Regards,
>> >>> Saravanan KR
>> >>>
>> >>>
>> >>> On Fri, Dec 2, 2016 at 9:31 PM, Yolanda Robla Mota
>> >>> <yrobl...@redhat.com>
>> >>> wrote:
>> >>> > Hi , Dmitry
>> >>> > That's what i didn't get very clear. If all the deployment steps are
>> >>> > pre-imaging as that statement says, or every deploy step could be
>> >>> > isolated
>> >>> > and configured somehow.
>> >>> > I'm also a bit confused with that spec, because it mixes the concept
>> >>> > of
>> >>> > "deployment steps", will all the changes needed for runtime RAID.
>> >>> > Could it
>> >>> > be possible to separate into two separate ones?
>> >>> >
>> >>> > - Original Message -
>> >>> > From: "Dmitry Tantsur" <dtant...@redhat.com>
>> >>> > To: openstack-dev@lists.openstack.org
>> >>> > Sent: Friday, December 2, 2016 3:51:30 PM
>> >>> > Subject: Re: [openstack-dev] [tripleo] [ironic] Need to update
>> >>> > kernel
>> >>> > parameters on local boot
>> >>> >
>> >>> > On 12/02/2016 01:28 PM, Yolanda Robla Mota wrote:
>> >>> >> Hi Dmitry
>> >>> >>
>> >>> >> So we've been looking at that spec you suggested, but we are
>> >>> >> wondering
>> >>> >> if that will be useful for our use case. As the text says:
>> >>> >>
>> >>> >> The ``ironic-python-agent`` project and ``agent`` driver will be
>> >>> >> adjusted to
>> >>> >> support ``get_deploy_steps``. That way, ``ironic-python-agent``
>> >>> >> will be
>> >>> >> able
>> >>> >> to declare deploy steps to run prior to disk imaging, and operators
>> >>> >> will be
>> >>> >> able to extend ``ironic-python-agent`` to add any custom step.
>> >>> >>
>> >>> >> Our needs are different, actually we need to create a deployment
>> >>> >> step
>> >>> >> after imaging. We'd need an step that drops config on
>> >>> >> /etc/default/grub ,
>> >>> >> and updates it. This is a post-imaging deploy step, that modifies
>> >>> >> the base
>> >>> >> image. Could ironic support these kind of steps, if there is a base
>> >>> >> system
>> >>> >> to just define per-user steps?
>> >>> >
>> >>> > I thought that all deployment operations are converted to steps,
>> >>> > with
>> >>> > partitioning, writing the image, writing the configdrive and
>> >>> > installing
>> >>> > the boot
>> >>> > loader being four default ones (as you see, two steps actually
>> >>> > happen
>> >>> > after the
>> >>> > image is written).
>> >>> >
>> >>> >>
>> >>> >> The idea we had on mind is:
>> >>> >> - from tripleo, add a property to each flavor, that defines the
>> >>> >> boot
>> >>> >> parameters:  openstack flavor set compute --property
>> >>> >> os:kernel_boot_params='abc'
>> >>> >> - define a "ironic post-imaging deploy step", that will grab this
>> >>> >> property from the flavor, drop it on /etc/default/grub and
>> >>> >> regenerate it
>> >>> >> - then on local boot, the proper kernel parameters will be applied
>> >>> >>
>> >>> >> What is your feedback there?
>> >>> >>
>> >>> >> - Original Message -
>> >>> >> From: "Dmitry Tantsur" <dtant...@redhat.com>
>

Re: [openstack-dev] [tripleo] [ironic] Need to update kernel parameters on local boot

[Yolanda Robla Mota]

It won't need a reboot, because these changes will be created by ironic
before the image is deployed. So it will boot with the right parameters.
The alternative of doing with puppet after the image was deployed, needed a
reboot, because the changes were done post-deploy.
So ironic build steps are pre-deploy without reboot, puppet changes are
post-deploy with a reboot.

On Tue, Dec 13, 2016 at 11:24 AM, Oliver Walsh <owa...@redhat.com> wrote:

> Hi,
>
> Saravanan wrote:
> > If ironic "deploy steps" can configure this "tuned" setting and run the
> command
>
> How does this avoid the reboot?
>
> Yolanda wrote:
> > The idea will be to define custom deployment steps for ironic, like
> including the kernel boot parameters.
>
> Again, is this avoiding the reboot or just moving it?
>
> Thanks,
> Ollie
>
> On 13 December 2016 at 09:02, Saravanan KR <skram...@redhat.com> wrote:
> > Hi Yolanda,
> >
> > The flow for "tuned" will be like set the "tuned" configuration files,
> > and then activate the profile by running the command "tuned-adm
> > tuned-profile-nfv". This command will actually write the required
> > configuration files for tuning the host. If ironic "deploy steps" can
> > configure this "tuned" setting and run the command, then it is good
> > enough.
> >
> > Regards,
> > Saravanan KR
> >
> > On Tue, Dec 13, 2016 at 1:04 PM, Yolanda Robla Mota <yrobl...@redhat.com>
> wrote:
> >> Hi Saravanan
> >> Thanks for your comments. With this new module, I guess a reboot is
> still
> >> needed after os-host-config ?
> >> Right now we have been guided by TripleO and Ironic people to start
> using
> >> what in Ironic is called "custom deployment steps". An initial spec is
> >> reflected here:
> >> https://review.openstack.org/#/c/382091
> >>
> >> The idea will be to define custom deployment steps for ironic, like
> >> including the kernel boot parameters. Can that be a solution for your
> >> "tuned" needs as well?
> >>
> >> Best
> >> Yolanda
> >>
> >> On Tue, Dec 13, 2016 at 7:59 AM, Saravanan KR <skram...@redhat.com>
> wrote:
> >>>
> >>> Hello,
> >>>
> >>> Thanks Yolanda for starting the thread. The list of requirements in
> >>> the host configuration, related to boot parameters and reboot are:
> >>>
> >>> * DPDK - For vfio-pci driver binding, iommu support on kernel args is
> >>> mandatory, which has to be configured before os-net-config runs
> >>> * DPDK & RealTime - Enabling "tuned" profile for nfv or rt, will
> >>> update the boot parameters and a reboot is required
> >>> * Other items mentioned by Yolanda
> >>>
> >>> If it is configuring only, the boot parameters, then ironic's deploy
> >>> feature may help, but there are other requirement to enable the
> >>> "tuned" profile which tunes the host for the required configuration,
> >>> which also requires reboot, as it will alter the boot parameters. If
> >>> we can collate the all the configurations which requires reboot
> >>> together, we will improve the reboot time. And if we reboot before the
> >>> actual openstack services are started, then the reboot time _may_
> >>> improve.
> >>>
> >>> Can I propose a *new* module for TripleO deployments, like >
> >>> os-host-config <, which will run after os-collect-config and before
> >>> os-net-config, then we can collate all the host specific configuration
> >>> inside this module. This module can be a set of ansible scripts, which
> >>> will only configure the host. Ofcource the parameter to this module
> >>> should be provided via os-collect-config. Separating the host
> >>> configuration will help in the containerized TripleO deployment also.
> >>>
> >>> Or any other better alternatives are welcome.
> >>>
> >>> Please pour in your views if you think for/against it.
> >>>
> >>> Regards,
> >>> Saravanan KR
> >>>
> >>>
> >>> On Fri, Dec 2, 2016 at 9:31 PM, Yolanda Robla Mota <
> yrobl...@redhat.com>
> >>> wrote:
> >>> > Hi , Dmitry
> >>> > That's what i didn't get very clear. If all the deployment steps are
> >>> > pre-imaging as that statement says, or every deploy step cou

Re: [openstack-dev] [tripleo] [ironic] Need to update kernel parameters on local boot

[Oliver Walsh]

Hi,

Saravanan wrote:
> If ironic "deploy steps" can configure this "tuned" setting and run the 
> command

How does this avoid the reboot?

Yolanda wrote:
> The idea will be to define custom deployment steps for ironic, like including 
> the kernel boot parameters.

Again, is this avoiding the reboot or just moving it?

Thanks,
Ollie

On 13 December 2016 at 09:02, Saravanan KR <skram...@redhat.com> wrote:
> Hi Yolanda,
>
> The flow for "tuned" will be like set the "tuned" configuration files,
> and then activate the profile by running the command "tuned-adm
> tuned-profile-nfv". This command will actually write the required
> configuration files for tuning the host. If ironic "deploy steps" can
> configure this "tuned" setting and run the command, then it is good
> enough.
>
> Regards,
> Saravanan KR
>
> On Tue, Dec 13, 2016 at 1:04 PM, Yolanda Robla Mota <yrobl...@redhat.com> 
> wrote:
>> Hi Saravanan
>> Thanks for your comments. With this new module, I guess a reboot is still
>> needed after os-host-config ?
>> Right now we have been guided by TripleO and Ironic people to start using
>> what in Ironic is called "custom deployment steps". An initial spec is
>> reflected here:
>> https://review.openstack.org/#/c/382091
>>
>> The idea will be to define custom deployment steps for ironic, like
>> including the kernel boot parameters. Can that be a solution for your
>> "tuned" needs as well?
>>
>> Best
>> Yolanda
>>
>> On Tue, Dec 13, 2016 at 7:59 AM, Saravanan KR <skram...@redhat.com> wrote:
>>>
>>> Hello,
>>>
>>> Thanks Yolanda for starting the thread. The list of requirements in
>>> the host configuration, related to boot parameters and reboot are:
>>>
>>> * DPDK - For vfio-pci driver binding, iommu support on kernel args is
>>> mandatory, which has to be configured before os-net-config runs
>>> * DPDK & RealTime - Enabling "tuned" profile for nfv or rt, will
>>> update the boot parameters and a reboot is required
>>> * Other items mentioned by Yolanda
>>>
>>> If it is configuring only, the boot parameters, then ironic's deploy
>>> feature may help, but there are other requirement to enable the
>>> "tuned" profile which tunes the host for the required configuration,
>>> which also requires reboot, as it will alter the boot parameters. If
>>> we can collate the all the configurations which requires reboot
>>> together, we will improve the reboot time. And if we reboot before the
>>> actual openstack services are started, then the reboot time _may_
>>> improve.
>>>
>>> Can I propose a *new* module for TripleO deployments, like >
>>> os-host-config <, which will run after os-collect-config and before
>>> os-net-config, then we can collate all the host specific configuration
>>> inside this module. This module can be a set of ansible scripts, which
>>> will only configure the host. Ofcource the parameter to this module
>>> should be provided via os-collect-config. Separating the host
>>> configuration will help in the containerized TripleO deployment also.
>>>
>>> Or any other better alternatives are welcome.
>>>
>>> Please pour in your views if you think for/against it.
>>>
>>> Regards,
>>> Saravanan KR
>>>
>>>
>>> On Fri, Dec 2, 2016 at 9:31 PM, Yolanda Robla Mota <yrobl...@redhat.com>
>>> wrote:
>>> > Hi , Dmitry
>>> > That's what i didn't get very clear. If all the deployment steps are
>>> > pre-imaging as that statement says, or every deploy step could be isolated
>>> > and configured somehow.
>>> > I'm also a bit confused with that spec, because it mixes the concept of
>>> > "deployment steps", will all the changes needed for runtime RAID. Could it
>>> > be possible to separate into two separate ones?
>>> >
>>> > - Original Message -
>>> > From: "Dmitry Tantsur" <dtant...@redhat.com>
>>> > To: openstack-dev@lists.openstack.org
>>> > Sent: Friday, December 2, 2016 3:51:30 PM
>>> > Subject: Re: [openstack-dev] [tripleo] [ironic] Need to update kernel
>>> > parameters on local boot
>>> >
>>> > On 12/02/2016 01:28 PM, Yolanda Robla Mota wrote:
>>> >> Hi Dmitry
>>> >>
>>> >> So we've been looking at th

Re: [openstack-dev] [tripleo] [ironic] Need to update kernel parameters on local boot

[Saravanan KR]

Hi Yolanda,

The flow for "tuned" will be like set the "tuned" configuration files,
and then activate the profile by running the command "tuned-adm
tuned-profile-nfv". This command will actually write the required
configuration files for tuning the host. If ironic "deploy steps" can
configure this "tuned" setting and run the command, then it is good
enough.

Regards,
Saravanan KR

On Tue, Dec 13, 2016 at 1:04 PM, Yolanda Robla Mota <yrobl...@redhat.com> wrote:
> Hi Saravanan
> Thanks for your comments. With this new module, I guess a reboot is still
> needed after os-host-config ?
> Right now we have been guided by TripleO and Ironic people to start using
> what in Ironic is called "custom deployment steps". An initial spec is
> reflected here:
> https://review.openstack.org/#/c/382091
>
> The idea will be to define custom deployment steps for ironic, like
> including the kernel boot parameters. Can that be a solution for your
> "tuned" needs as well?
>
> Best
> Yolanda
>
> On Tue, Dec 13, 2016 at 7:59 AM, Saravanan KR <skram...@redhat.com> wrote:
>>
>> Hello,
>>
>> Thanks Yolanda for starting the thread. The list of requirements in
>> the host configuration, related to boot parameters and reboot are:
>>
>> * DPDK - For vfio-pci driver binding, iommu support on kernel args is
>> mandatory, which has to be configured before os-net-config runs
>> * DPDK & RealTime - Enabling "tuned" profile for nfv or rt, will
>> update the boot parameters and a reboot is required
>> * Other items mentioned by Yolanda
>>
>> If it is configuring only, the boot parameters, then ironic's deploy
>> feature may help, but there are other requirement to enable the
>> "tuned" profile which tunes the host for the required configuration,
>> which also requires reboot, as it will alter the boot parameters. If
>> we can collate the all the configurations which requires reboot
>> together, we will improve the reboot time. And if we reboot before the
>> actual openstack services are started, then the reboot time _may_
>> improve.
>>
>> Can I propose a *new* module for TripleO deployments, like >
>> os-host-config <, which will run after os-collect-config and before
>> os-net-config, then we can collate all the host specific configuration
>> inside this module. This module can be a set of ansible scripts, which
>> will only configure the host. Ofcource the parameter to this module
>> should be provided via os-collect-config. Separating the host
>> configuration will help in the containerized TripleO deployment also.
>>
>> Or any other better alternatives are welcome.
>>
>> Please pour in your views if you think for/against it.
>>
>> Regards,
>> Saravanan KR
>>
>>
>> On Fri, Dec 2, 2016 at 9:31 PM, Yolanda Robla Mota <yrobl...@redhat.com>
>> wrote:
>> > Hi , Dmitry
>> > That's what i didn't get very clear. If all the deployment steps are
>> > pre-imaging as that statement says, or every deploy step could be isolated
>> > and configured somehow.
>> > I'm also a bit confused with that spec, because it mixes the concept of
>> > "deployment steps", will all the changes needed for runtime RAID. Could it
>> > be possible to separate into two separate ones?
>> >
>> > - Original Message -
>> > From: "Dmitry Tantsur" <dtant...@redhat.com>
>> > To: openstack-dev@lists.openstack.org
>> > Sent: Friday, December 2, 2016 3:51:30 PM
>> > Subject: Re: [openstack-dev] [tripleo] [ironic] Need to update kernel
>> > parameters on local boot
>> >
>> > On 12/02/2016 01:28 PM, Yolanda Robla Mota wrote:
>> >> Hi Dmitry
>> >>
>> >> So we've been looking at that spec you suggested, but we are wondering
>> >> if that will be useful for our use case. As the text says:
>> >>
>> >> The ``ironic-python-agent`` project and ``agent`` driver will be
>> >> adjusted to
>> >> support ``get_deploy_steps``. That way, ``ironic-python-agent`` will be
>> >> able
>> >> to declare deploy steps to run prior to disk imaging, and operators
>> >> will be
>> >> able to extend ``ironic-python-agent`` to add any custom step.
>> >>
>> >> Our needs are different, actually we need to create a deployment step
>> >> after imaging. We'd need an step that drops config on /etc/default/grub ,
>> >> and updates it. This is a post-imagin

Re: [openstack-dev] [tripleo] [ironic] Need to update kernel parameters on local boot

[Yolanda Robla Mota]

Hi Saravanan
Thanks for your comments. With this new module, I guess a reboot is still
needed after os-host-config ?
Right now we have been guided by TripleO and Ironic people to start using
what in Ironic is called "custom deployment steps". An initial spec is
reflected here:
https://review.openstack.org/#/c/382091

The idea will be to define custom deployment steps for ironic, like
including the kernel boot parameters. Can that be a solution for your
"tuned" needs as well?

Best
Yolanda

On Tue, Dec 13, 2016 at 7:59 AM, Saravanan KR <skram...@redhat.com> wrote:

> Hello,
>
> Thanks Yolanda for starting the thread. The list of requirements in
> the host configuration, related to boot parameters and reboot are:
>
> * DPDK - For vfio-pci driver binding, iommu support on kernel args is
> mandatory, which has to be configured before os-net-config runs
> * DPDK & RealTime - Enabling "tuned" profile for nfv or rt, will
> update the boot parameters and a reboot is required
> * Other items mentioned by Yolanda
>
> If it is configuring only, the boot parameters, then ironic's deploy
> feature may help, but there are other requirement to enable the
> "tuned" profile which tunes the host for the required configuration,
> which also requires reboot, as it will alter the boot parameters. If
> we can collate the all the configurations which requires reboot
> together, we will improve the reboot time. And if we reboot before the
> actual openstack services are started, then the reboot time _may_
> improve.
>
> Can I propose a *new* module for TripleO deployments, like >
> os-host-config <, which will run after os-collect-config and before
> os-net-config, then we can collate all the host specific configuration
> inside this module. This module can be a set of ansible scripts, which
> will only configure the host. Ofcource the parameter to this module
> should be provided via os-collect-config. Separating the host
> configuration will help in the containerized TripleO deployment also.
>
> Or any other better alternatives are welcome.
>
> Please pour in your views if you think for/against it.
>
> Regards,
> Saravanan KR
>
>
> On Fri, Dec 2, 2016 at 9:31 PM, Yolanda Robla Mota <yrobl...@redhat.com>
> wrote:
> > Hi , Dmitry
> > That's what i didn't get very clear. If all the deployment steps are
> pre-imaging as that statement says, or every deploy step could be isolated
> and configured somehow.
> > I'm also a bit confused with that spec, because it mixes the concept of
> "deployment steps", will all the changes needed for runtime RAID. Could it
> be possible to separate into two separate ones?
> >
> > - Original Message -
> > From: "Dmitry Tantsur" <dtant...@redhat.com>
> > To: openstack-dev@lists.openstack.org
> > Sent: Friday, December 2, 2016 3:51:30 PM
> > Subject: Re: [openstack-dev] [tripleo] [ironic] Need to update kernel
> parameters on local boot
> >
> > On 12/02/2016 01:28 PM, Yolanda Robla Mota wrote:
> >> Hi Dmitry
> >>
> >> So we've been looking at that spec you suggested, but we are wondering
> if that will be useful for our use case. As the text says:
> >>
> >> The ``ironic-python-agent`` project and ``agent`` driver will be
> adjusted to
> >> support ``get_deploy_steps``. That way, ``ironic-python-agent`` will be
> able
> >> to declare deploy steps to run prior to disk imaging, and operators
> will be
> >> able to extend ``ironic-python-agent`` to add any custom step.
> >>
> >> Our needs are different, actually we need to create a deployment step
> after imaging. We'd need an step that drops config on /etc/default/grub ,
> and updates it. This is a post-imaging deploy step, that modifies the base
> image. Could ironic support these kind of steps, if there is a base system
> to just define per-user steps?
> >
> > I thought that all deployment operations are converted to steps, with
> > partitioning, writing the image, writing the configdrive and installing
> the boot
> > loader being four default ones (as you see, two steps actually happen
> after the
> > image is written).
> >
> >>
> >> The idea we had on mind is:
> >> - from tripleo, add a property to each flavor, that defines the boot
> parameters:  openstack flavor set compute --property
> os:kernel_boot_params='abc'
> >> - define a "ironic post-imaging deploy step", that will grab this
> property from the flavor, drop it on /etc/default/grub and regenerate it
> >> - then on local boot, the proper kernel parameters will be applied
> >>
> 

Re: [openstack-dev] [tripleo] [ironic] Need to update kernel parameters on local boot

[Saravanan KR]

Hello,

Thanks Yolanda for starting the thread. The list of requirements in
the host configuration, related to boot parameters and reboot are:

* DPDK - For vfio-pci driver binding, iommu support on kernel args is
mandatory, which has to be configured before os-net-config runs
* DPDK & RealTime - Enabling "tuned" profile for nfv or rt, will
update the boot parameters and a reboot is required
* Other items mentioned by Yolanda

If it is configuring only, the boot parameters, then ironic's deploy
feature may help, but there are other requirement to enable the
"tuned" profile which tunes the host for the required configuration,
which also requires reboot, as it will alter the boot parameters. If
we can collate the all the configurations which requires reboot
together, we will improve the reboot time. And if we reboot before the
actual openstack services are started, then the reboot time _may_
improve.

Can I propose a *new* module for TripleO deployments, like >
os-host-config <, which will run after os-collect-config and before
os-net-config, then we can collate all the host specific configuration
inside this module. This module can be a set of ansible scripts, which
will only configure the host. Ofcource the parameter to this module
should be provided via os-collect-config. Separating the host
configuration will help in the containerized TripleO deployment also.

Or any other better alternatives are welcome.

Please pour in your views if you think for/against it.

Regards,
Saravanan KR


On Fri, Dec 2, 2016 at 9:31 PM, Yolanda Robla Mota <yrobl...@redhat.com> wrote:
> Hi , Dmitry
> That's what i didn't get very clear. If all the deployment steps are 
> pre-imaging as that statement says, or every deploy step could be isolated 
> and configured somehow.
> I'm also a bit confused with that spec, because it mixes the concept of 
> "deployment steps", will all the changes needed for runtime RAID. Could it be 
> possible to separate into two separate ones?
>
> - Original Message -
> From: "Dmitry Tantsur" <dtant...@redhat.com>
> To: openstack-dev@lists.openstack.org
> Sent: Friday, December 2, 2016 3:51:30 PM
> Subject: Re: [openstack-dev] [tripleo] [ironic] Need to update kernel 
> parameters on local boot
>
> On 12/02/2016 01:28 PM, Yolanda Robla Mota wrote:
>> Hi Dmitry
>>
>> So we've been looking at that spec you suggested, but we are wondering if 
>> that will be useful for our use case. As the text says:
>>
>> The ``ironic-python-agent`` project and ``agent`` driver will be adjusted to
>> support ``get_deploy_steps``. That way, ``ironic-python-agent`` will be able
>> to declare deploy steps to run prior to disk imaging, and operators will be
>> able to extend ``ironic-python-agent`` to add any custom step.
>>
>> Our needs are different, actually we need to create a deployment step after 
>> imaging. We'd need an step that drops config on /etc/default/grub , and 
>> updates it. This is a post-imaging deploy step, that modifies the base 
>> image. Could ironic support these kind of steps, if there is a base system 
>> to just define per-user steps?
>
> I thought that all deployment operations are converted to steps, with
> partitioning, writing the image, writing the configdrive and installing the 
> boot
> loader being four default ones (as you see, two steps actually happen after 
> the
> image is written).
>
>>
>> The idea we had on mind is:
>> - from tripleo, add a property to each flavor, that defines the boot 
>> parameters:  openstack flavor set compute --property 
>> os:kernel_boot_params='abc'
>> - define a "ironic post-imaging deploy step", that will grab this property 
>> from the flavor, drop it on /etc/default/grub and regenerate it
>> - then on local boot, the proper kernel parameters will be applied
>>
>> What is your feedback there?
>>
>> - Original Message -
>> From: "Dmitry Tantsur" <dtant...@redhat.com>
>> To: openstack-dev@lists.openstack.org
>> Sent: Friday, December 2, 2016 12:44:29 PM
>> Subject: Re: [openstack-dev] [tripleo] [ironic] Need to update kernel 
>> parameters on local boot
>>
>> On 11/28/2016 04:46 PM, Jay Faulkner wrote:
>>>
>>>> On Nov 28, 2016, at 7:36 AM, Yolanda Robla Mota <yrobl...@redhat.com> 
>>>> wrote:
>>>>
>>>> Hi, good afternoon
>>>>
>>>> I wanted to start an email thread about how to properly setup kernel 
>>>> parameters on local boot, for our overcloud images on TripleO.
>>>> These parameters may vary depending on the needs of our end users, and 
>>>> even ca

Re: [openstack-dev] [tripleo] [ironic] Need to update kernel parameters on local boot

[Yolanda Robla Mota]

Hi , Dmitry
That's what i didn't get very clear. If all the deployment steps are 
pre-imaging as that statement says, or every deploy step could be isolated and 
configured somehow.
I'm also a bit confused with that spec, because it mixes the concept of 
"deployment steps", will all the changes needed for runtime RAID. Could it be 
possible to separate into two separate ones?

- Original Message -
From: "Dmitry Tantsur" <dtant...@redhat.com>
To: openstack-dev@lists.openstack.org
Sent: Friday, December 2, 2016 3:51:30 PM
Subject: Re: [openstack-dev] [tripleo] [ironic] Need to update kernel 
parameters on local boot

On 12/02/2016 01:28 PM, Yolanda Robla Mota wrote:
> Hi Dmitry
>
> So we've been looking at that spec you suggested, but we are wondering if 
> that will be useful for our use case. As the text says:
>
> The ``ironic-python-agent`` project and ``agent`` driver will be adjusted to
> support ``get_deploy_steps``. That way, ``ironic-python-agent`` will be able
> to declare deploy steps to run prior to disk imaging, and operators will be
> able to extend ``ironic-python-agent`` to add any custom step.
>
> Our needs are different, actually we need to create a deployment step after 
> imaging. We'd need an step that drops config on /etc/default/grub , and 
> updates it. This is a post-imaging deploy step, that modifies the base image. 
> Could ironic support these kind of steps, if there is a base system to just 
> define per-user steps?

I thought that all deployment operations are converted to steps, with 
partitioning, writing the image, writing the configdrive and installing the 
boot 
loader being four default ones (as you see, two steps actually happen after the 
image is written).

>
> The idea we had on mind is:
> - from tripleo, add a property to each flavor, that defines the boot 
> parameters:  openstack flavor set compute --property 
> os:kernel_boot_params='abc'
> - define a "ironic post-imaging deploy step", that will grab this property 
> from the flavor, drop it on /etc/default/grub and regenerate it
> - then on local boot, the proper kernel parameters will be applied
>
> What is your feedback there?
>
> - Original Message -
> From: "Dmitry Tantsur" <dtant...@redhat.com>
> To: openstack-dev@lists.openstack.org
> Sent: Friday, December 2, 2016 12:44:29 PM
> Subject: Re: [openstack-dev] [tripleo] [ironic] Need to update kernel 
> parameters on local boot
>
> On 11/28/2016 04:46 PM, Jay Faulkner wrote:
>>
>>> On Nov 28, 2016, at 7:36 AM, Yolanda Robla Mota <yrobl...@redhat.com> wrote:
>>>
>>> Hi, good afternoon
>>>
>>> I wanted to start an email thread about how to properly setup kernel 
>>> parameters on local boot, for our overcloud images on TripleO.
>>> These parameters may vary depending on the needs of our end users, and even 
>>> can be different ( for different roles ) per deployment. As an example, we 
>>> need it for:
>>> - enable FIPS kernel in terms of security 
>>> (https://bugs.launchpad.net/tripleo/+bug/1640235)
>>> - enable functionality for DPDK/SR-IOV 
>>> (https://review.openstack.org/#/c/331564/)
>>> - enable rd.iscsi.firmware=1 flag (this for the ramdisk image)
>>> - etc..
>>>
>>> So far, the solutions we got were on several directions:
>>>
>>> 1. Update the golden overcloud-full image with virt-customize, modifying 
>>> /etc/default/grub settings according to our needs: this is a manual 
>>> process, not really driven by TripleO. End users will want to avoid manual 
>>> steps as much as possible. Also if we announce that OpenStack ships 
>>> features in TripleO like DPDK, SR-IOV... doesn't make sense to tell end 
>>> users that if they want to consume that feature, they need to do manual 
>>> updates on the image. It shall be natively supported, or configurable per 
>>> TripleO environments.
>>>
>>> 2. Create our own images using diskimage-builder and custom elements: in 
>>> this case, we have the problem that the partners will loose support, as 
>>> building their own images is good for upstream, but not accepted into the 
>>> OSP environment. Also the combination of images needed can be huge, that 
>>> can be a blocker for QA.
>>>
>>> 3. Add Ironic support for it. Images can be uploaded to glance, and some 
>>> properties can be set on metadata, like a json with kernel parameters. 
>>> Ironic will modify these kernel parameters when deploying the image (in a 
>>> similar way that when it installs bootloader, or generates partitions).
>>>
>>
&g

Re: [openstack-dev] [tripleo] [ironic] Need to update kernel parameters on local boot

[Jay Faulkner]

> On Dec 2, 2016, at 3:44 AM, Dmitry Tantsur  wrote:
> 
> On 11/28/2016 04:46 PM, Jay Faulkner wrote:
>> 
>>> On Nov 28, 2016, at 7:36 AM, Yolanda Robla Mota  wrote:
>>> 
>>> Hi, good afternoon
>>> 
>>> I wanted to start an email thread about how to properly setup kernel 
>>> parameters on local boot, for our overcloud images on TripleO.
>>> These parameters may vary depending on the needs of our end users, and even 
>>> can be different ( for different roles ) per deployment. As an example, we 
>>> need it for:
>>> - enable FIPS kernel in terms of security 
>>> (https://bugs.launchpad.net/tripleo/+bug/1640235)
>>> - enable functionality for DPDK/SR-IOV 
>>> (https://review.openstack.org/#/c/331564/)
>>> - enable rd.iscsi.firmware=1 flag (this for the ramdisk image)
>>> - etc..
>>> 
>>> So far, the solutions we got were on several directions:
>>> 
>>> 1. Update the golden overcloud-full image with virt-customize, modifying 
>>> /etc/default/grub settings according to our needs: this is a manual 
>>> process, not really driven by TripleO. End users will want to avoid manual 
>>> steps as much as possible. Also if we announce that OpenStack ships 
>>> features in TripleO like DPDK, SR-IOV... doesn't make sense to tell end 
>>> users that if they want to consume that feature, they need to do manual 
>>> updates on the image. It shall be natively supported, or configurable per 
>>> TripleO environments.
>>> 
>>> 2. Create our own images using diskimage-builder and custom elements: in 
>>> this case, we have the problem that the partners will loose support, as 
>>> building their own images is good for upstream, but not accepted into the 
>>> OSP environment. Also the combination of images needed can be huge, that 
>>> can be a blocker for QA.
>>> 
>>> 3. Add Ironic support for it. Images can be uploaded to glance, and some 
>>> properties can be set on metadata, like a json with kernel parameters. 
>>> Ironic will modify these kernel parameters when deploying the image (in a 
>>> similar way that when it installs bootloader, or generates partitions).
>>> 
>> 
>> This has been proposed before in ironic-specs 
>> (https://review.openstack.org/#/c/331564/) and was rejected, as it would 
>> require Ironic to reach out and modify image contents, which traditionally 
>> has been considered out of scope for Ironic. I would personally recommend 
>> #4, as post-boot automation is the safest way to configure node-specific 
>> options inside an image.
> 
> I'm still a bit divided about our decision back then.. On one hand, this does 
> seem somewhat out of scope. On the other, I quite understand why reboot is 
> suboptimal. I wonder if the ongoing deploy steps work will actually solve it 
> by allowing hardware managers to provide additional deploy steps.
> 

I’m not really of two minds on this at all. Modifying the filesystem directly 
would expose Ironic to a whole new world of complexity, including security 
issues, dealing with multiple incompatible filesystems, and the like. I’m 
obviously OK if anyone wants to use a customization point to do stuff that’d 
typically be outside of Ironic’s scope, but I don’t think this is a use case we 
should encourage.

The realm of configuring a machine beyond laying down the image has to lie in 
configuration management software, or else we open up to a huge scope increase 
and get away from our core mission.

-Jay


> Yolanda, you may want to check the spec 
> https://review.openstack.org/#/c/382091/ as it lays the foundation for the 
> deploy steps idea.
> 
>> 
>> Thanks,
>> Jay Faulkner
>> OSIC
>> 
>> 
>>> 4. Configure it post-deployment: there can be some puppet element that 
>>> updates kernel parameters. But it will need a node reboot to be applied, 
>>> and it's very far from being optimal and acceptable for the end users. 
>>> Reboots are slow, they can be a problem depending on the number of 
>>> nodes/hardware, and also the timing of reboot shall be totally controlled 
>>> (after all puppet has been applied properly).
>>> 
>>> 
>>> In the first three cases, we also hit the problem that TripleO only accepts 
>>> one single overcloud image for all deployments - there is no way to 
>>> instruct TripleO to upload and use several images, depending on the node 
>>> type (although Ironic supports it). Also, we are worried about upgrade 
>>> paths if we do image customizations. We need a clear way to move forward on 
>>> it.
>>> 
>>> So, we'd like to discuss the possible options there and the action items to 
>>> take (raise bugs, create some blueprints...). To summarize, our end goal is 
>>> the following:
>>> 
>>> - need to map overcloud-full images to roles
>>> - need to be done in an automated way, no manual steps enforced, and in a 
>>> way that can pass properly quality controls
>>> - reboots are sub-optimal
>>> 
>>> What are your thoughts there?
>>> 
>>> Best,
>>> 
>>> 
>>> Yolanda Robla
>>> yrobl...@redhat.com
>>> Principal 

Re: [openstack-dev] [tripleo] [ironic] Need to update kernel parameters on local boot

[Dmitry Tantsur]

On 12/02/2016 01:28 PM, Yolanda Robla Mota wrote:

Hi Dmitry

So we've been looking at that spec you suggested, but we are wondering if that 
will be useful for our use case. As the text says:

The ``ironic-python-agent`` project and ``agent`` driver will be adjusted to
support ``get_deploy_steps``. That way, ``ironic-python-agent`` will be able
to declare deploy steps to run prior to disk imaging, and operators will be
able to extend ``ironic-python-agent`` to add any custom step.

Our needs are different, actually we need to create a deployment step after 
imaging. We'd need an step that drops config on /etc/default/grub , and updates 
it. This is a post-imaging deploy step, that modifies the base image. Could 
ironic support these kind of steps, if there is a base system to just define 
per-user steps?


I thought that all deployment operations are converted to steps, with 
partitioning, writing the image, writing the configdrive and installing the boot 
loader being four default ones (as you see, two steps actually happen after the 
image is written).




The idea we had on mind is:
- from tripleo, add a property to each flavor, that defines the boot 
parameters:  openstack flavor set compute --property os:kernel_boot_params='abc'
- define a "ironic post-imaging deploy step", that will grab this property from 
the flavor, drop it on /etc/default/grub and regenerate it
- then on local boot, the proper kernel parameters will be applied

What is your feedback there?

- Original Message -
From: "Dmitry Tantsur" <dtant...@redhat.com>
To: openstack-dev@lists.openstack.org
Sent: Friday, December 2, 2016 12:44:29 PM
Subject: Re: [openstack-dev] [tripleo] [ironic] Need to update kernel 
parameters on local boot

On 11/28/2016 04:46 PM, Jay Faulkner wrote:



On Nov 28, 2016, at 7:36 AM, Yolanda Robla Mota <yrobl...@redhat.com> wrote:

Hi, good afternoon

I wanted to start an email thread about how to properly setup kernel parameters 
on local boot, for our overcloud images on TripleO.
These parameters may vary depending on the needs of our end users, and even can 
be different ( for different roles ) per deployment. As an example, we need it 
for:
- enable FIPS kernel in terms of security 
(https://bugs.launchpad.net/tripleo/+bug/1640235)
- enable functionality for DPDK/SR-IOV 
(https://review.openstack.org/#/c/331564/)
- enable rd.iscsi.firmware=1 flag (this for the ramdisk image)
- etc..

So far, the solutions we got were on several directions:

1. Update the golden overcloud-full image with virt-customize, modifying 
/etc/default/grub settings according to our needs: this is a manual process, 
not really driven by TripleO. End users will want to avoid manual steps as much 
as possible. Also if we announce that OpenStack ships features in TripleO like 
DPDK, SR-IOV... doesn't make sense to tell end users that if they want to 
consume that feature, they need to do manual updates on the image. It shall be 
natively supported, or configurable per TripleO environments.

2. Create our own images using diskimage-builder and custom elements: in this 
case, we have the problem that the partners will loose support, as building 
their own images is good for upstream, but not accepted into the OSP 
environment. Also the combination of images needed can be huge, that can be a 
blocker for QA.

3. Add Ironic support for it. Images can be uploaded to glance, and some 
properties can be set on metadata, like a json with kernel parameters. Ironic 
will modify these kernel parameters when deploying the image (in a similar way 
that when it installs bootloader, or generates partitions).



This has been proposed before in ironic-specs 
(https://review.openstack.org/#/c/331564/) and was rejected, as it would 
require Ironic to reach out and modify image contents, which traditionally has 
been considered out of scope for Ironic. I would personally recommend #4, as 
post-boot automation is the safest way to configure node-specific options 
inside an image.


I'm still a bit divided about our decision back then.. On one hand, this does
seem somewhat out of scope. On the other, I quite understand why reboot is
suboptimal. I wonder if the ongoing deploy steps work will actually solve it by
allowing hardware managers to provide additional deploy steps.

Yolanda, you may want to check the spec https://review.openstack.org/#/c/382091/
as it lays the foundation for the deploy steps idea.



Thanks,
Jay Faulkner
OSIC



4. Configure it post-deployment: there can be some puppet element that updates 
kernel parameters. But it will need a node reboot to be applied, and it's very 
far from being optimal and acceptable for the end users. Reboots are slow, they 
can be a problem depending on the number of nodes/hardware, and also the timing 
of reboot shall be totally controlled (after all puppet has been applied 
properly).


In the first three cases, we also hit the problem that TripleO only acc

Re: [openstack-dev] [tripleo] [ironic] Need to update kernel parameters on local boot

[Yolanda Robla Mota]

Yes, we are talking about physical hardware and large clusters. Also in a very 
demanding environment.
So reboots are really suboptimal in that case.

- Original Message -
From: "Alex Schultz" <aschu...@redhat.com>
To: "OpenStack Development Mailing List (not for usage questions)" 
<openstack-dev@lists.openstack.org>
Sent: Friday, December 2, 2016 3:38:53 PM
Subject: Re: [openstack-dev] [tripleo] [ironic] Need to update kernel 
parameters on local boot

On Fri, Dec 2, 2016 at 7:29 AM, Oliver Walsh <owa...@redhat.com> wrote:
> Hi Yolanda,
>
> I've the same requirements for a real-time compute role.
>
> I'm current using #4. Puppet sets the kernel cmdline in
> /etc/defaults/grub, then touches a file that triggers a reboot in an
> os-refresh-config post-configure script.
>
> How much of an issue is the reboot? In my dev env it doesn't make a
> significant difference to the overall deployment time of a node.
>

If this is physical hardware, the reboot can add 5-10+ minutes
depending on the vendor.  VM's aren't a good representation of the
actual cost of a reboot.

Thanks,
-Alex

> I'm thinking we could use #4 as the general solution. In cases where a
> reboot is too expensive then #1/#2/#3 could be used to prime the image
> with same /etc/default/grub that puppet would create. As puppet
> doesn't change it doesn't trigger a reboot.
>
>
> Re custom images. Support in python-tripleoclient could be improved
> but for now this is what I'm doing:
>
> Uploading a custom image:
> OS_IMAGE=custom-overcloud-full.qcow2 openstack overcloud image upload
>
> Set the Image for the custom role in param_defaults:
> parameter_defaults:
>   CustomRoleImage: custom-overcloud-full
>
> Thanks,
> Ollie
>
> On 28 November 2016 at 15:36, Yolanda Robla Mota <yrobl...@redhat.com> wrote:
>> Hi, good afternoon
>>
>> I wanted to start an email thread about how to properly setup kernel 
>> parameters on local boot, for our overcloud images on TripleO.
>> These parameters may vary depending on the needs of our end users, and even 
>> can be different ( for different roles ) per deployment. As an example, we 
>> need it for:
>> - enable FIPS kernel in terms of security 
>> (https://bugs.launchpad.net/tripleo/+bug/1640235)
>> - enable functionality for DPDK/SR-IOV 
>> (https://review.openstack.org/#/c/331564/)
>> - enable rd.iscsi.firmware=1 flag (this for the ramdisk image)
>> - etc..
>>
>> So far, the solutions we got were on several directions:
>>
>> 1. Update the golden overcloud-full image with virt-customize, modifying 
>> /etc/default/grub settings according to our needs: this is a manual process, 
>> not really driven by TripleO. End users will want to avoid manual steps as 
>> much as possible. Also if we announce that OpenStack ships features in 
>> TripleO like DPDK, SR-IOV... doesn't make sense to tell end users that if 
>> they want to consume that feature, they need to do manual updates on the 
>> image. It shall be natively supported, or configurable per TripleO 
>> environments.
>>
>> 2. Create our own images using diskimage-builder and custom elements: in 
>> this case, we have the problem that the partners will loose support, as 
>> building their own images is good for upstream, but not accepted into the 
>> OSP environment. Also the combination of images needed can be huge, that can 
>> be a blocker for QA.
>>
>> 3. Add Ironic support for it. Images can be uploaded to glance, and some 
>> properties can be set on metadata, like a json with kernel parameters. 
>> Ironic will modify these kernel parameters when deploying the image (in a 
>> similar way that when it installs bootloader, or generates partitions).
>>
>> 4. Configure it post-deployment: there can be some puppet element that 
>> updates kernel parameters. But it will need a node reboot to be applied, and 
>> it's very far from being optimal and acceptable for the end users. Reboots 
>> are slow, they can be a problem depending on the number of nodes/hardware, 
>> and also the timing of reboot shall be totally controlled (after all puppet 
>> has been applied properly).
>>
>>
>> In the first three cases, we also hit the problem that TripleO only accepts 
>> one single overcloud image for all deployments - there is no way to instruct 
>> TripleO to upload and use several images, depending on the node type 
>> (although Ironic supports it). Also, we are worried about upgrade paths if 
>> we do image customizations. We need a clear way to move forward on it.
>>
>> So, we'd like to discuss the possible options there 

Re: [openstack-dev] [tripleo] [ironic] Need to update kernel parameters on local boot

[Oliver Walsh]

Hi Yolanda,

I've the same requirements for a real-time compute role.

I'm current using #4. Puppet sets the kernel cmdline in
/etc/defaults/grub, then touches a file that triggers a reboot in an
os-refresh-config post-configure script.

How much of an issue is the reboot? In my dev env it doesn't make a
significant difference to the overall deployment time of a node.

I'm thinking we could use #4 as the general solution. In cases where a
reboot is too expensive then #1/#2/#3 could be used to prime the image
with same /etc/default/grub that puppet would create. As puppet
doesn't change it doesn't trigger a reboot.


Re custom images. Support in python-tripleoclient could be improved
but for now this is what I'm doing:

Uploading a custom image:
OS_IMAGE=custom-overcloud-full.qcow2 openstack overcloud image upload

Set the Image for the custom role in param_defaults:
parameter_defaults:
  CustomRoleImage: custom-overcloud-full

Thanks,
Ollie

On 28 November 2016 at 15:36, Yolanda Robla Mota  wrote:
> Hi, good afternoon
>
> I wanted to start an email thread about how to properly setup kernel 
> parameters on local boot, for our overcloud images on TripleO.
> These parameters may vary depending on the needs of our end users, and even 
> can be different ( for different roles ) per deployment. As an example, we 
> need it for:
> - enable FIPS kernel in terms of security 
> (https://bugs.launchpad.net/tripleo/+bug/1640235)
> - enable functionality for DPDK/SR-IOV 
> (https://review.openstack.org/#/c/331564/)
> - enable rd.iscsi.firmware=1 flag (this for the ramdisk image)
> - etc..
>
> So far, the solutions we got were on several directions:
>
> 1. Update the golden overcloud-full image with virt-customize, modifying 
> /etc/default/grub settings according to our needs: this is a manual process, 
> not really driven by TripleO. End users will want to avoid manual steps as 
> much as possible. Also if we announce that OpenStack ships features in 
> TripleO like DPDK, SR-IOV... doesn't make sense to tell end users that if 
> they want to consume that feature, they need to do manual updates on the 
> image. It shall be natively supported, or configurable per TripleO 
> environments.
>
> 2. Create our own images using diskimage-builder and custom elements: in this 
> case, we have the problem that the partners will loose support, as building 
> their own images is good for upstream, but not accepted into the OSP 
> environment. Also the combination of images needed can be huge, that can be a 
> blocker for QA.
>
> 3. Add Ironic support for it. Images can be uploaded to glance, and some 
> properties can be set on metadata, like a json with kernel parameters. Ironic 
> will modify these kernel parameters when deploying the image (in a similar 
> way that when it installs bootloader, or generates partitions).
>
> 4. Configure it post-deployment: there can be some puppet element that 
> updates kernel parameters. But it will need a node reboot to be applied, and 
> it's very far from being optimal and acceptable for the end users. Reboots 
> are slow, they can be a problem depending on the number of nodes/hardware, 
> and also the timing of reboot shall be totally controlled (after all puppet 
> has been applied properly).
>
>
> In the first three cases, we also hit the problem that TripleO only accepts 
> one single overcloud image for all deployments - there is no way to instruct 
> TripleO to upload and use several images, depending on the node type 
> (although Ironic supports it). Also, we are worried about upgrade paths if we 
> do image customizations. We need a clear way to move forward on it.
>
> So, we'd like to discuss the possible options there and the action items to 
> take (raise bugs, create some blueprints...). To summarize, our end goal is 
> the following:
>
> - need to map overcloud-full images to roles
> - need to be done in an automated way, no manual steps enforced, and in a way 
> that can pass properly quality controls
> - reboots are sub-optimal
>
> What are your thoughts there?
>
> Best,
>
>
> Yolanda Robla
> yrobl...@redhat.com
> Principal Software Engineer - NFV Partner Engineer
>
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [tripleo] [ironic] Need to update kernel parameters on local boot

[Alex Schultz]

On Fri, Dec 2, 2016 at 7:29 AM, Oliver Walsh  wrote:
> Hi Yolanda,
>
> I've the same requirements for a real-time compute role.
>
> I'm current using #4. Puppet sets the kernel cmdline in
> /etc/defaults/grub, then touches a file that triggers a reboot in an
> os-refresh-config post-configure script.
>
> How much of an issue is the reboot? In my dev env it doesn't make a
> significant difference to the overall deployment time of a node.
>

If this is physical hardware, the reboot can add 5-10+ minutes
depending on the vendor.  VM's aren't a good representation of the
actual cost of a reboot.

Thanks,
-Alex

> I'm thinking we could use #4 as the general solution. In cases where a
> reboot is too expensive then #1/#2/#3 could be used to prime the image
> with same /etc/default/grub that puppet would create. As puppet
> doesn't change it doesn't trigger a reboot.
>
>
> Re custom images. Support in python-tripleoclient could be improved
> but for now this is what I'm doing:
>
> Uploading a custom image:
> OS_IMAGE=custom-overcloud-full.qcow2 openstack overcloud image upload
>
> Set the Image for the custom role in param_defaults:
> parameter_defaults:
>   CustomRoleImage: custom-overcloud-full
>
> Thanks,
> Ollie
>
> On 28 November 2016 at 15:36, Yolanda Robla Mota  wrote:
>> Hi, good afternoon
>>
>> I wanted to start an email thread about how to properly setup kernel 
>> parameters on local boot, for our overcloud images on TripleO.
>> These parameters may vary depending on the needs of our end users, and even 
>> can be different ( for different roles ) per deployment. As an example, we 
>> need it for:
>> - enable FIPS kernel in terms of security 
>> (https://bugs.launchpad.net/tripleo/+bug/1640235)
>> - enable functionality for DPDK/SR-IOV 
>> (https://review.openstack.org/#/c/331564/)
>> - enable rd.iscsi.firmware=1 flag (this for the ramdisk image)
>> - etc..
>>
>> So far, the solutions we got were on several directions:
>>
>> 1. Update the golden overcloud-full image with virt-customize, modifying 
>> /etc/default/grub settings according to our needs: this is a manual process, 
>> not really driven by TripleO. End users will want to avoid manual steps as 
>> much as possible. Also if we announce that OpenStack ships features in 
>> TripleO like DPDK, SR-IOV... doesn't make sense to tell end users that if 
>> they want to consume that feature, they need to do manual updates on the 
>> image. It shall be natively supported, or configurable per TripleO 
>> environments.
>>
>> 2. Create our own images using diskimage-builder and custom elements: in 
>> this case, we have the problem that the partners will loose support, as 
>> building their own images is good for upstream, but not accepted into the 
>> OSP environment. Also the combination of images needed can be huge, that can 
>> be a blocker for QA.
>>
>> 3. Add Ironic support for it. Images can be uploaded to glance, and some 
>> properties can be set on metadata, like a json with kernel parameters. 
>> Ironic will modify these kernel parameters when deploying the image (in a 
>> similar way that when it installs bootloader, or generates partitions).
>>
>> 4. Configure it post-deployment: there can be some puppet element that 
>> updates kernel parameters. But it will need a node reboot to be applied, and 
>> it's very far from being optimal and acceptable for the end users. Reboots 
>> are slow, they can be a problem depending on the number of nodes/hardware, 
>> and also the timing of reboot shall be totally controlled (after all puppet 
>> has been applied properly).
>>
>>
>> In the first three cases, we also hit the problem that TripleO only accepts 
>> one single overcloud image for all deployments - there is no way to instruct 
>> TripleO to upload and use several images, depending on the node type 
>> (although Ironic supports it). Also, we are worried about upgrade paths if 
>> we do image customizations. We need a clear way to move forward on it.
>>
>> So, we'd like to discuss the possible options there and the action items to 
>> take (raise bugs, create some blueprints...). To summarize, our end goal is 
>> the following:
>>
>> - need to map overcloud-full images to roles
>> - need to be done in an automated way, no manual steps enforced, and in a 
>> way that can pass properly quality controls
>> - reboots are sub-optimal
>>
>> What are your thoughts there?
>>
>> Best,
>>
>>
>> Yolanda Robla
>> yrobl...@redhat.com
>> Principal Software Engineer - NFV Partner Engineer
>>
>>
>> __
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
> __
> OpenStack Development Mailing List (not for usage questions)
> 

Re: [openstack-dev] [tripleo] [ironic] Need to update kernel parameters on local boot

[Yolanda Robla Mota]

Hi Dmitry

So we've been looking at that spec you suggested, but we are wondering if that 
will be useful for our use case. As the text says:

The ``ironic-python-agent`` project and ``agent`` driver will be adjusted to
support ``get_deploy_steps``. That way, ``ironic-python-agent`` will be able
to declare deploy steps to run prior to disk imaging, and operators will be
able to extend ``ironic-python-agent`` to add any custom step.

Our needs are different, actually we need to create a deployment step after 
imaging. We'd need an step that drops config on /etc/default/grub , and updates 
it. This is a post-imaging deploy step, that modifies the base image. Could 
ironic support these kind of steps, if there is a base system to just define 
per-user steps?

The idea we had on mind is:
- from tripleo, add a property to each flavor, that defines the boot 
parameters:  openstack flavor set compute --property os:kernel_boot_params='abc'
- define a "ironic post-imaging deploy step", that will grab this property from 
the flavor, drop it on /etc/default/grub and regenerate it
- then on local boot, the proper kernel parameters will be applied

What is your feedback there?

- Original Message -
From: "Dmitry Tantsur" <dtant...@redhat.com>
To: openstack-dev@lists.openstack.org
Sent: Friday, December 2, 2016 12:44:29 PM
Subject: Re: [openstack-dev] [tripleo] [ironic] Need to update kernel 
parameters on local boot

On 11/28/2016 04:46 PM, Jay Faulkner wrote:
>
>> On Nov 28, 2016, at 7:36 AM, Yolanda Robla Mota <yrobl...@redhat.com> wrote:
>>
>> Hi, good afternoon
>>
>> I wanted to start an email thread about how to properly setup kernel 
>> parameters on local boot, for our overcloud images on TripleO.
>> These parameters may vary depending on the needs of our end users, and even 
>> can be different ( for different roles ) per deployment. As an example, we 
>> need it for:
>> - enable FIPS kernel in terms of security 
>> (https://bugs.launchpad.net/tripleo/+bug/1640235)
>> - enable functionality for DPDK/SR-IOV 
>> (https://review.openstack.org/#/c/331564/)
>> - enable rd.iscsi.firmware=1 flag (this for the ramdisk image)
>> - etc..
>>
>> So far, the solutions we got were on several directions:
>>
>> 1. Update the golden overcloud-full image with virt-customize, modifying 
>> /etc/default/grub settings according to our needs: this is a manual process, 
>> not really driven by TripleO. End users will want to avoid manual steps as 
>> much as possible. Also if we announce that OpenStack ships features in 
>> TripleO like DPDK, SR-IOV... doesn't make sense to tell end users that if 
>> they want to consume that feature, they need to do manual updates on the 
>> image. It shall be natively supported, or configurable per TripleO 
>> environments.
>>
>> 2. Create our own images using diskimage-builder and custom elements: in 
>> this case, we have the problem that the partners will loose support, as 
>> building their own images is good for upstream, but not accepted into the 
>> OSP environment. Also the combination of images needed can be huge, that can 
>> be a blocker for QA.
>>
>> 3. Add Ironic support for it. Images can be uploaded to glance, and some 
>> properties can be set on metadata, like a json with kernel parameters. 
>> Ironic will modify these kernel parameters when deploying the image (in a 
>> similar way that when it installs bootloader, or generates partitions).
>>
>
> This has been proposed before in ironic-specs 
> (https://review.openstack.org/#/c/331564/) and was rejected, as it would 
> require Ironic to reach out and modify image contents, which traditionally 
> has been considered out of scope for Ironic. I would personally recommend #4, 
> as post-boot automation is the safest way to configure node-specific options 
> inside an image.

I'm still a bit divided about our decision back then.. On one hand, this does 
seem somewhat out of scope. On the other, I quite understand why reboot is 
suboptimal. I wonder if the ongoing deploy steps work will actually solve it by 
allowing hardware managers to provide additional deploy steps.

Yolanda, you may want to check the spec 
https://review.openstack.org/#/c/382091/ 
as it lays the foundation for the deploy steps idea.

>
> Thanks,
> Jay Faulkner
> OSIC
>
>
>> 4. Configure it post-deployment: there can be some puppet element that 
>> updates kernel parameters. But it will need a node reboot to be applied, and 
>> it's very far from being optimal and acceptable for the end users. Reboots 
>> are slow, they can be a problem depending on the number of nodes/hardware, 
>> and also the timing of reboot shall be total

Re: [openstack-dev] [tripleo] [ironic] Need to update kernel parameters on local boot

[Dmitry Tantsur]

On 11/28/2016 04:46 PM, Jay Faulkner wrote:



On Nov 28, 2016, at 7:36 AM, Yolanda Robla Mota  wrote:

Hi, good afternoon

I wanted to start an email thread about how to properly setup kernel parameters 
on local boot, for our overcloud images on TripleO.
These parameters may vary depending on the needs of our end users, and even can 
be different ( for different roles ) per deployment. As an example, we need it 
for:
- enable FIPS kernel in terms of security 
(https://bugs.launchpad.net/tripleo/+bug/1640235)
- enable functionality for DPDK/SR-IOV 
(https://review.openstack.org/#/c/331564/)
- enable rd.iscsi.firmware=1 flag (this for the ramdisk image)
- etc..

So far, the solutions we got were on several directions:

1. Update the golden overcloud-full image with virt-customize, modifying 
/etc/default/grub settings according to our needs: this is a manual process, 
not really driven by TripleO. End users will want to avoid manual steps as much 
as possible. Also if we announce that OpenStack ships features in TripleO like 
DPDK, SR-IOV... doesn't make sense to tell end users that if they want to 
consume that feature, they need to do manual updates on the image. It shall be 
natively supported, or configurable per TripleO environments.

2. Create our own images using diskimage-builder and custom elements: in this 
case, we have the problem that the partners will loose support, as building 
their own images is good for upstream, but not accepted into the OSP 
environment. Also the combination of images needed can be huge, that can be a 
blocker for QA.

3. Add Ironic support for it. Images can be uploaded to glance, and some 
properties can be set on metadata, like a json with kernel parameters. Ironic 
will modify these kernel parameters when deploying the image (in a similar way 
that when it installs bootloader, or generates partitions).



This has been proposed before in ironic-specs 
(https://review.openstack.org/#/c/331564/) and was rejected, as it would 
require Ironic to reach out and modify image contents, which traditionally has 
been considered out of scope for Ironic. I would personally recommend #4, as 
post-boot automation is the safest way to configure node-specific options 
inside an image.


I'm still a bit divided about our decision back then.. On one hand, this does 
seem somewhat out of scope. On the other, I quite understand why reboot is 
suboptimal. I wonder if the ongoing deploy steps work will actually solve it by 
allowing hardware managers to provide additional deploy steps.


Yolanda, you may want to check the spec https://review.openstack.org/#/c/382091/ 
as it lays the foundation for the deploy steps idea.




Thanks,
Jay Faulkner
OSIC



4. Configure it post-deployment: there can be some puppet element that updates 
kernel parameters. But it will need a node reboot to be applied, and it's very 
far from being optimal and acceptable for the end users. Reboots are slow, they 
can be a problem depending on the number of nodes/hardware, and also the timing 
of reboot shall be totally controlled (after all puppet has been applied 
properly).


In the first three cases, we also hit the problem that TripleO only accepts one 
single overcloud image for all deployments - there is no way to instruct 
TripleO to upload and use several images, depending on the node type (although 
Ironic supports it). Also, we are worried about upgrade paths if we do image 
customizations. We need a clear way to move forward on it.

So, we'd like to discuss the possible options there and the action items to 
take (raise bugs, create some blueprints...). To summarize, our end goal is the 
following:

- need to map overcloud-full images to roles
- need to be done in an automated way, no manual steps enforced, and in a way 
that can pass properly quality controls
- reboots are sub-optimal

What are your thoughts there?

Best,


Yolanda Robla
yrobl...@redhat.com
Principal Software Engineer - NFV Partner Engineer


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev




__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [tripleo] [ironic] Need to update kernel parameters on local boot

[Frank Zdarsky]

On Tue, Nov 29, 2016 at 10:48 AM, Yolanda Robla Mota <yrobl...@redhat.com>
wrote:

> The problem we have with that is that for changes to take effect we will
> need a reboot after it.
> This is suboptimal for production environments, where there are large
> amount of nodes and with an slow hardware. And also the reboot needs to be
> carefully synced, to only take place after all puppet changes have been
> applied.
> That's why we wanted to consider some other solutions that are done
> pre-deployment, they will be much more effective in terms of performance.
>

Would it be an option that Ironic provides those kernel params just for the
first boot (when the overcloud is deployed) and TripleO then persists them
by modifying grub.cfg accordingly for later local boots? This would avoid
Ironic having to modify grub.cfg while also avoiding reboots.

However, those kernel params would need to be configurable per node / per
role and ideally without duplicating the overcloud images in Glance.


>
> - Original Message -
> From: "Jay Faulkner" <j...@jvf.cc>
> To: "OpenStack Development Mailing List (not for usage questions)" <
> openstack-dev@lists.openstack.org>
> Sent: Monday, November 28, 2016 4:46:56 PM
> Subject: Re: [openstack-dev] [tripleo] [ironic] Need to update kernel
> parameterson local boot
>
>
> > On Nov 28, 2016, at 7:36 AM, Yolanda Robla Mota <yrobl...@redhat.com>
> wrote:
> >
> > Hi, good afternoon
> >
> > I wanted to start an email thread about how to properly setup kernel
> parameters on local boot, for our overcloud images on TripleO.
> > These parameters may vary depending on the needs of our end users, and
> even can be different ( for different roles ) per deployment. As an
> example, we need it for:
> > - enable FIPS kernel in terms of security (https://bugs.launchpad.net/
> tripleo/+bug/1640235)
> > - enable functionality for DPDK/SR-IOV (https://review.openstack.org/
> #/c/331564/)
> > - enable rd.iscsi.firmware=1 flag (this for the ramdisk image)
> > - etc..
> >
> > So far, the solutions we got were on several directions:
> >
> > 1. Update the golden overcloud-full image with virt-customize, modifying
> /etc/default/grub settings according to our needs: this is a manual
> process, not really driven by TripleO. End users will want to avoid manual
> steps as much as possible. Also if we announce that OpenStack ships
> features in TripleO like DPDK, SR-IOV... doesn't make sense to tell end
> users that if they want to consume that feature, they need to do manual
> updates on the image. It shall be natively supported, or configurable per
> TripleO environments.
> >
> > 2. Create our own images using diskimage-builder and custom elements: in
> this case, we have the problem that the partners will loose support, as
> building their own images is good for upstream, but not accepted into the
> OSP environment. Also the combination of images needed can be huge, that
> can be a blocker for QA.
> >
> > 3. Add Ironic support for it. Images can be uploaded to glance, and some
> properties can be set on metadata, like a json with kernel parameters.
> Ironic will modify these kernel parameters when deploying the image (in a
> similar way that when it installs bootloader, or generates partitions).
> >
>
> This has been proposed before in ironic-specs (
> https://review.openstack.org/#/c/331564/) and was rejected, as it would
> require Ironic to reach out and modify image contents, which traditionally
> has been considered out of scope for Ironic. I would personally recommend
> #4, as post-boot automation is the safest way to configure node-specific
> options inside an image.
>
> Thanks,
> Jay Faulkner
> OSIC
>
>
> > 4. Configure it post-deployment: there can be some puppet element that
> updates kernel parameters. But it will need a node reboot to be applied,
> and it's very far from being optimal and acceptable for the end users.
> Reboots are slow, they can be a problem depending on the number of
> nodes/hardware, and also the timing of reboot shall be totally controlled
> (after all puppet has been applied properly).
> >
> >
> > In the first three cases, we also hit the problem that TripleO only
> accepts one single overcloud image for all deployments - there is no way to
> instruct TripleO to upload and use several images, depending on the node
> type (although Ironic supports it). Also, we are worried about upgrade
> paths if we do image customizations. We need a clear way to move forward on
> it.
> >
> > So, we'd like to discuss the possible options there and the action items
> to take (raise bugs, 

Re: [openstack-dev] [tripleo] [ironic] Need to update kernel parameters on local boot

[Yolanda Robla Mota]

The problem we have with that is that for changes to take effect we will need a 
reboot after it.
This is suboptimal for production environments, where there are large amount of 
nodes and with an slow hardware. And also the reboot needs to be carefully 
synced, to only take place after all puppet changes have been applied.
That's why we wanted to consider some other solutions that are done 
pre-deployment, they will be much more effective in terms of performance.

- Original Message -
From: "Jay Faulkner" <j...@jvf.cc>
To: "OpenStack Development Mailing List (not for usage questions)" 
<openstack-dev@lists.openstack.org>
Sent: Monday, November 28, 2016 4:46:56 PM
Subject: Re: [openstack-dev] [tripleo] [ironic] Need to update kernel 
parameterson local boot


> On Nov 28, 2016, at 7:36 AM, Yolanda Robla Mota <yrobl...@redhat.com> wrote:
> 
> Hi, good afternoon
> 
> I wanted to start an email thread about how to properly setup kernel 
> parameters on local boot, for our overcloud images on TripleO.
> These parameters may vary depending on the needs of our end users, and even 
> can be different ( for different roles ) per deployment. As an example, we 
> need it for:
> - enable FIPS kernel in terms of security 
> (https://bugs.launchpad.net/tripleo/+bug/1640235)
> - enable functionality for DPDK/SR-IOV 
> (https://review.openstack.org/#/c/331564/)
> - enable rd.iscsi.firmware=1 flag (this for the ramdisk image)
> - etc..
> 
> So far, the solutions we got were on several directions:
> 
> 1. Update the golden overcloud-full image with virt-customize, modifying 
> /etc/default/grub settings according to our needs: this is a manual process, 
> not really driven by TripleO. End users will want to avoid manual steps as 
> much as possible. Also if we announce that OpenStack ships features in 
> TripleO like DPDK, SR-IOV... doesn't make sense to tell end users that if 
> they want to consume that feature, they need to do manual updates on the 
> image. It shall be natively supported, or configurable per TripleO 
> environments.
> 
> 2. Create our own images using diskimage-builder and custom elements: in this 
> case, we have the problem that the partners will loose support, as building 
> their own images is good for upstream, but not accepted into the OSP 
> environment. Also the combination of images needed can be huge, that can be a 
> blocker for QA.
> 
> 3. Add Ironic support for it. Images can be uploaded to glance, and some 
> properties can be set on metadata, like a json with kernel parameters. Ironic 
> will modify these kernel parameters when deploying the image (in a similar 
> way that when it installs bootloader, or generates partitions).
> 

This has been proposed before in ironic-specs 
(https://review.openstack.org/#/c/331564/) and was rejected, as it would 
require Ironic to reach out and modify image contents, which traditionally has 
been considered out of scope for Ironic. I would personally recommend #4, as 
post-boot automation is the safest way to configure node-specific options 
inside an image.

Thanks,
Jay Faulkner
OSIC


> 4. Configure it post-deployment: there can be some puppet element that 
> updates kernel parameters. But it will need a node reboot to be applied, and 
> it's very far from being optimal and acceptable for the end users. Reboots 
> are slow, they can be a problem depending on the number of nodes/hardware, 
> and also the timing of reboot shall be totally controlled (after all puppet 
> has been applied properly).
> 
> 
> In the first three cases, we also hit the problem that TripleO only accepts 
> one single overcloud image for all deployments - there is no way to instruct 
> TripleO to upload and use several images, depending on the node type 
> (although Ironic supports it). Also, we are worried about upgrade paths if we 
> do image customizations. We need a clear way to move forward on it.
> 
> So, we'd like to discuss the possible options there and the action items to 
> take (raise bugs, create some blueprints...). To summarize, our end goal is 
> the following:
> 
> - need to map overcloud-full images to roles
> - need to be done in an automated way, no manual steps enforced, and in a way 
> that can pass properly quality controls
> - reboots are sub-optimal
> 
> What are your thoughts there?
> 
> Best,
> 
> 
> Yolanda Robla
> yrobl...@redhat.com
> Principal Software Engineer - NFV Partner Engineer
> 
> 
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [tripleo] [ironic] Need to update kernel parameters on local boot

[Jay Faulkner]

> On Nov 28, 2016, at 7:36 AM, Yolanda Robla Mota  wrote:
> 
> Hi, good afternoon
> 
> I wanted to start an email thread about how to properly setup kernel 
> parameters on local boot, for our overcloud images on TripleO.
> These parameters may vary depending on the needs of our end users, and even 
> can be different ( for different roles ) per deployment. As an example, we 
> need it for:
> - enable FIPS kernel in terms of security 
> (https://bugs.launchpad.net/tripleo/+bug/1640235)
> - enable functionality for DPDK/SR-IOV 
> (https://review.openstack.org/#/c/331564/)
> - enable rd.iscsi.firmware=1 flag (this for the ramdisk image)
> - etc..
> 
> So far, the solutions we got were on several directions:
> 
> 1. Update the golden overcloud-full image with virt-customize, modifying 
> /etc/default/grub settings according to our needs: this is a manual process, 
> not really driven by TripleO. End users will want to avoid manual steps as 
> much as possible. Also if we announce that OpenStack ships features in 
> TripleO like DPDK, SR-IOV... doesn't make sense to tell end users that if 
> they want to consume that feature, they need to do manual updates on the 
> image. It shall be natively supported, or configurable per TripleO 
> environments.
> 
> 2. Create our own images using diskimage-builder and custom elements: in this 
> case, we have the problem that the partners will loose support, as building 
> their own images is good for upstream, but not accepted into the OSP 
> environment. Also the combination of images needed can be huge, that can be a 
> blocker for QA.
> 
> 3. Add Ironic support for it. Images can be uploaded to glance, and some 
> properties can be set on metadata, like a json with kernel parameters. Ironic 
> will modify these kernel parameters when deploying the image (in a similar 
> way that when it installs bootloader, or generates partitions).
> 

This has been proposed before in ironic-specs 
(https://review.openstack.org/#/c/331564/) and was rejected, as it would 
require Ironic to reach out and modify image contents, which traditionally has 
been considered out of scope for Ironic. I would personally recommend #4, as 
post-boot automation is the safest way to configure node-specific options 
inside an image.

Thanks,
Jay Faulkner
OSIC


> 4. Configure it post-deployment: there can be some puppet element that 
> updates kernel parameters. But it will need a node reboot to be applied, and 
> it's very far from being optimal and acceptable for the end users. Reboots 
> are slow, they can be a problem depending on the number of nodes/hardware, 
> and also the timing of reboot shall be totally controlled (after all puppet 
> has been applied properly).
> 
> 
> In the first three cases, we also hit the problem that TripleO only accepts 
> one single overcloud image for all deployments - there is no way to instruct 
> TripleO to upload and use several images, depending on the node type 
> (although Ironic supports it). Also, we are worried about upgrade paths if we 
> do image customizations. We need a clear way to move forward on it.
> 
> So, we'd like to discuss the possible options there and the action items to 
> take (raise bugs, create some blueprints...). To summarize, our end goal is 
> the following:
> 
> - need to map overcloud-full images to roles
> - need to be done in an automated way, no manual steps enforced, and in a way 
> that can pass properly quality controls
> - reboots are sub-optimal
> 
> What are your thoughts there?
> 
> Best,
> 
> 
> Yolanda Robla
> yrobl...@redhat.com
> Principal Software Engineer - NFV Partner Engineer
> 
> 
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev