Re: [openstack-dev] [Openstack-operators] [ironic] [nova] [tripleo] Deprecation of Nova's integration with Ironic Capabilities and ComputeCapabilitiesFilter

[Dmitry Tantsur]

On 10/2/18 6:17 PM, Mark Goddard wrote:



On Tue, 2 Oct 2018 at 17:10, Jim Rollenhagen > wrote:


On Tue, Oct 2, 2018 at 11:40 AM Eric Fried  wrote:

 > What Eric is proposing (and Julia and I seem to be in favor of), is
 > nearly the same as your proposal. The single difference is that these
 > config templates or deploy templates or whatever could *also* require
 > certain traits, and the scheduler would use that information to pick 
a
 > node. While this does put some scheduling information into the config
 > template, it also means that we can remove some of the flavor 
explosion
 > *and* mostly separate scheduling from configuration.
 >
 > So, you'd have a list of traits on a flavor:
 >
 > required=HW_CPU_X86_VMX,HW_NIC_ACCEL_IPSEC
 >
 > And you would also have a list of traits in the deploy template:
 >
 > {"traits": {"required": ["STORAGE_HARDWARE_RAID"]}, "config": }
 >
 > This allows for making flavors that are reasonably flexible (instead 
of
 > two flavors that do VMX and IPSEC acceleration, one of which does 
RAID).
 > It also allows users to specify a desired configuration without also
 > needing to know how to correctly choose a flavor that can handle that
 > configuration.
 >
 > I think it makes a lot of sense, doesn't impose more work on users, 
and
 > can reduce the number of flavors operators need to manage.
 >
 > Does that make sense?

This is in fact exactly what Jay proposed. And both Julia and I are in
favor of it as an ideal long-term solution. Where Julia and I deviated
from Jay's point of view was in our desire to use "the hack" in the
short term so we can satisfy the majority of use cases right away
without having to wait for that ideal solution to materialize.


Ah, good point, I had missed that initially. Thanks. Let's do that.

So if we all agree Jay's proposal is the right thing to do, is there any
reason to start working on a short-term hack instead of putting those
efforts into the better solution? I don't see why we couldn't get that done
in one cycle, if we're all in agreement on it.


I'm still unclear on the ironic side of this. I can see that config of some sort 
is stored in glance, and referenced upon nova server creation. Somehow this 
would be synced to ironic by the nova virt driver during node provisioning. The 
part that's missing in my mind is how to map from a config in glance to a set of 
actions performed by ironic. Does the config in glance reference a deploy 
template, or a set of ironic deploy steps? Or does ironic (or OpenStack) define 
some config schema that it supports, and use it to generate a set of deploy steps?


I think the most straightforward way is through the same deploy steps mechanism 
we planned. Make the virt driver fetch the config from glance, then pass it to 
the provisioning API. As a bonus, we'll get the same API workflow with 
standalone and nova case.





// jim
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe

http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev




__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Openstack-operators] [ironic] [nova] [tripleo] Deprecation of Nova's integration with Ironic Capabilities and ComputeCapabilitiesFilter

[Eric Fried]

On 10/02/2018 11:09 AM, Jim Rollenhagen wrote:
> On Tue, Oct 2, 2018 at 11:40 AM Eric Fried  wrote:
> 
> > What Eric is proposing (and Julia and I seem to be in favor of), is
> > nearly the same as your proposal. The single difference is that these
> > config templates or deploy templates or whatever could *also* require
> > certain traits, and the scheduler would use that information to pick a
> > node. While this does put some scheduling information into the config
> > template, it also means that we can remove some of the flavor
> explosion
> > *and* mostly separate scheduling from configuration.
> >
> > So, you'd have a list of traits on a flavor:
> >
> > required=HW_CPU_X86_VMX,HW_NIC_ACCEL_IPSEC
> >
> > And you would also have a list of traits in the deploy template:
> >
> > {"traits": {"required": ["STORAGE_HARDWARE_RAID"]}, "config":
> }
> >
> > This allows for making flavors that are reasonably flexible
> (instead of
> > two flavors that do VMX and IPSEC acceleration, one of which does
> RAID).
> > It also allows users to specify a desired configuration without also
> > needing to know how to correctly choose a flavor that can handle that
> > configuration.
> >
> > I think it makes a lot of sense, doesn't impose more work on
> users, and
> > can reduce the number of flavors operators need to manage.
> >
> > Does that make sense?
> 
> This is in fact exactly what Jay proposed. And both Julia and I are in
> favor of it as an ideal long-term solution. Where Julia and I deviated
> from Jay's point of view was in our desire to use "the hack" in the
> short term so we can satisfy the majority of use cases right away
> without having to wait for that ideal solution to materialize.
> 
> 
> Ah, good point, I had missed that initially. Thanks. Let's do that.
> 
> So if we all agree Jay's proposal is the right thing to do, is there any
> reason to start working on a short-term hack instead of putting those
> efforts into the better solution? I don't see why we couldn't get that
> done in one cycle, if we're all in agreement on it.

It takes more than agreement, though. It takes resources. I may have
misunderstood a major theme of the PTG, but I think the Nova team is
pretty overextended already. Even assuming authorship by wicked smaaht
folks such as yourself, the spec and code reviews will require a
nontrivial investment from Nova cores. The result would likely be
de-/re-prioritization of things we just got done agreeing to work on. If
that's The Right Thing, so be it. But we can't just say we're going to
move forward with something of this magnitude without sacrificing
something else.

(Note that the above opinion is based on the assumption that the hacky
way will require *much* less spec/code/review bandwidth to accomplish.
If that's not true, then I totally agree with you that we should spend
our time working on the right solution.)

> 
> // jim
> 
> 
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> 

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Openstack-operators] [ironic] [nova] [tripleo] Deprecation of Nova's integration with Ironic Capabilities and ComputeCapabilitiesFilter

[Mark Goddard]

On Tue, 2 Oct 2018 at 17:10, Jim Rollenhagen  wrote:

> On Tue, Oct 2, 2018 at 11:40 AM Eric Fried  wrote:
>
>> > What Eric is proposing (and Julia and I seem to be in favor of), is
>> > nearly the same as your proposal. The single difference is that these
>> > config templates or deploy templates or whatever could *also* require
>> > certain traits, and the scheduler would use that information to pick a
>> > node. While this does put some scheduling information into the config
>> > template, it also means that we can remove some of the flavor explosion
>> > *and* mostly separate scheduling from configuration.
>> >
>> > So, you'd have a list of traits on a flavor:
>> >
>> > required=HW_CPU_X86_VMX,HW_NIC_ACCEL_IPSEC
>> >
>> > And you would also have a list of traits in the deploy template:
>> >
>> > {"traits": {"required": ["STORAGE_HARDWARE_RAID"]}, "config": > blob>}
>> >
>> > This allows for making flavors that are reasonably flexible (instead of
>> > two flavors that do VMX and IPSEC acceleration, one of which does RAID).
>> > It also allows users to specify a desired configuration without also
>> > needing to know how to correctly choose a flavor that can handle that
>> > configuration.
>> >
>> > I think it makes a lot of sense, doesn't impose more work on users, and
>> > can reduce the number of flavors operators need to manage.
>> >
>> > Does that make sense?
>>
>> This is in fact exactly what Jay proposed. And both Julia and I are in
>> favor of it as an ideal long-term solution. Where Julia and I deviated
>> from Jay's point of view was in our desire to use "the hack" in the
>> short term so we can satisfy the majority of use cases right away
>> without having to wait for that ideal solution to materialize.
>>
>
> Ah, good point, I had missed that initially. Thanks. Let's do that.
>
> So if we all agree Jay's proposal is the right thing to do, is there any
> reason to start working on a short-term hack instead of putting those
> efforts into the better solution? I don't see why we couldn't get that done
> in one cycle, if we're all in agreement on it.
>

I'm still unclear on the ironic side of this. I can see that config of some
sort is stored in glance, and referenced upon nova server creation. Somehow
this would be synced to ironic by the nova virt driver during node
provisioning. The part that's missing in my mind is how to map from a
config in glance to a set of actions performed by ironic. Does the config
in glance reference a deploy template, or a set of ironic deploy steps? Or
does ironic (or OpenStack) define some config schema that it supports, and
use it to generate a set of deploy steps?


> // jim
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Openstack-operators] [ironic] [nova] [tripleo] Deprecation of Nova's integration with Ironic Capabilities and ComputeCapabilitiesFilter

[Jim Rollenhagen]

On Tue, Oct 2, 2018 at 11:40 AM Eric Fried  wrote:

> > What Eric is proposing (and Julia and I seem to be in favor of), is
> > nearly the same as your proposal. The single difference is that these
> > config templates or deploy templates or whatever could *also* require
> > certain traits, and the scheduler would use that information to pick a
> > node. While this does put some scheduling information into the config
> > template, it also means that we can remove some of the flavor explosion
> > *and* mostly separate scheduling from configuration.
> >
> > So, you'd have a list of traits on a flavor:
> >
> > required=HW_CPU_X86_VMX,HW_NIC_ACCEL_IPSEC
> >
> > And you would also have a list of traits in the deploy template:
> >
> > {"traits": {"required": ["STORAGE_HARDWARE_RAID"]}, "config":  blob>}
> >
> > This allows for making flavors that are reasonably flexible (instead of
> > two flavors that do VMX and IPSEC acceleration, one of which does RAID).
> > It also allows users to specify a desired configuration without also
> > needing to know how to correctly choose a flavor that can handle that
> > configuration.
> >
> > I think it makes a lot of sense, doesn't impose more work on users, and
> > can reduce the number of flavors operators need to manage.
> >
> > Does that make sense?
>
> This is in fact exactly what Jay proposed. And both Julia and I are in
> favor of it as an ideal long-term solution. Where Julia and I deviated
> from Jay's point of view was in our desire to use "the hack" in the
> short term so we can satisfy the majority of use cases right away
> without having to wait for that ideal solution to materialize.
>

Ah, good point, I had missed that initially. Thanks. Let's do that.

So if we all agree Jay's proposal is the right thing to do, is there any
reason to start working on a short-term hack instead of putting those
efforts into the better solution? I don't see why we couldn't get that done
in one cycle, if we're all in agreement on it.

// jim
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Openstack-operators] [ironic] [nova] [tripleo] Deprecation of Nova's integration with Ironic Capabilities and ComputeCapabilitiesFilter

[Eric Fried]

> What Eric is proposing (and Julia and I seem to be in favor of), is
> nearly the same as your proposal. The single difference is that these
> config templates or deploy templates or whatever could *also* require
> certain traits, and the scheduler would use that information to pick a
> node. While this does put some scheduling information into the config
> template, it also means that we can remove some of the flavor explosion
> *and* mostly separate scheduling from configuration.
> 
> So, you'd have a list of traits on a flavor:
> 
> required=HW_CPU_X86_VMX,HW_NIC_ACCEL_IPSEC
> 
> And you would also have a list of traits in the deploy template:
> 
> {"traits": {"required": ["STORAGE_HARDWARE_RAID"]}, "config": }
> 
> This allows for making flavors that are reasonably flexible (instead of
> two flavors that do VMX and IPSEC acceleration, one of which does RAID).
> It also allows users to specify a desired configuration without also
> needing to know how to correctly choose a flavor that can handle that
> configuration.
> 
> I think it makes a lot of sense, doesn't impose more work on users, and
> can reduce the number of flavors operators need to manage.
> 
> Does that make sense?

This is in fact exactly what Jay proposed. And both Julia and I are in
favor of it as an ideal long-term solution. Where Julia and I deviated
from Jay's point of view was in our desire to use "the hack" in the
short term so we can satisfy the majority of use cases right away
without having to wait for that ideal solution to materialize.

> 
> // jim
> 
> 
> Best,
> -jay
> 
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe:
> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> 
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> 
> 
> 
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> 

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Openstack-operators] [ironic] [nova] [tripleo] Deprecation of Nova's integration with Ironic Capabilities and ComputeCapabilitiesFilter

[Jim Rollenhagen]

On Mon, Oct 1, 2018 at 6:38 PM Jay Pipes  wrote:

> On 10/01/2018 06:04 PM, Julia Kreger wrote:
> > On Mon, Oct 1, 2018 at 2:41 PM Eric Fried  wrote:
>
> 

>
> > That said, what if it was:
> >
> >   openstack config-profile create --name BOOT_MODE_UEFI --json -
> >   {
> >"type": "boot_mode_scheme",
> >"version": 123,
> >"object": {
> >"boot_mode": "uefi"
> >},
> >"placement": {
> > "traits": {
> >  "required": [
> >   "BOOT_MODE_UEFI"
> >  ]
> > }
> >}
> >   }
> >   ^D
> >
> > And now you could in fact say
> >
> >   openstack server create --flavor foo --config-profile
> BOOT_MODE_UEFI
> >
> > using the profile name, which happens to be the same as the trait
> name
> > because you made it so. Does that satisfy the yen for saying it
> once? (I
> > mean, despite the fact that you first had to say it three times to
> get
> > it set up.)
> >
> 
> >
> > I feel like it might be confusing, but totally +1 to matching required
> > trait name being a thing. That way scheduling is completely decoupled
> > and if everything was correct then the request should already be
> > scheduled properly.
>
> I guess I'll just drop the idea of doing this properly then. It's true
> that the placement traits concept can be hacked up and the virt driver
> can just pass a list of trait strings to the Ironic API and that's the
> most expedient way to get what the 90% of people apparently want. It's
> also true that it will add a bunch of unmaintainable tribal knowledge
> into the interface between Nova and Ironic, but that has been the case
> for multiple years.
>
> The flavor explosion problem will continue to get worse for those of us
> who deal with its pain (Oath in particular feels this) because the
> interface between nova flavors and Ironic instance capabilities will
> continue to be super-tightly-coupled.
>
> For the record, I would have been happier if someone had proposed
> separating the instance configuration data in the flavor extra-specs
> from the notion of required placement constraints (i.e. traits). You
> could call the extra_spec "deploy_template_id" if you wanted and that
> extra spec value could have been passed to Ironic during node
> provisioning instead of the list of placement constraints (traits).
>
> So, you'd have a list of actual placement traits for an instance that
> looked like this:
>
> required=BOOT_MODE_UEFI,STORAGE_HARDWARE_RAID
>
> and you'd have a flavor extra spec called "deploy_template_id" with a
> value of the deploy template configuration data you wanted to
> communicate to Ironic. The Ironic virt driver could then just look for
> the "deploy_template_id" extra spec and pass the value of that to the
> Ironic API instead of passing a list of traits.
>
> That would have at least satisfied my desire to separate configuration
> data from placement constraints.
>
> Anyway, I'm done trying to please my own desires for a clean solution to
> this.
>

Jay, please don't stop - I think we aren't expressing ourselves well, or
you're missing something, or both. I understand this is a frustrating
conversation for everyone. But I think we're making good progress on the
end goal (whether or not we do an intermediate step that hacks on top of
traits). We all want a clean solution to this.

What Eric is proposing (and Julia and I seem to be in favor of), is nearly
the same as your proposal. The single difference is that these config
templates or deploy templates or whatever could *also* require certain
traits, and the scheduler would use that information to pick a node. While
this does put some scheduling information into the config template, it also
means that we can remove some of the flavor explosion *and* mostly separate
scheduling from configuration.

So, you'd have a list of traits on a flavor:

required=HW_CPU_X86_VMX,HW_NIC_ACCEL_IPSEC

And you would also have a list of traits in the deploy template:

{"traits": {"required": ["STORAGE_HARDWARE_RAID"]}, "config": }

This allows for making flavors that are reasonably flexible (instead of two
flavors that do VMX and IPSEC acceleration, one of which does RAID). It
also allows users to specify a desired configuration without also needing
to know how to correctly choose a flavor that can handle that configuration.

I think it makes a lot of sense, doesn't impose more work on users, and can
reduce the number of flavors operators need to manage.

Does that make sense?

// jim


> Best,
> -jay
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
__
OpenStack Development Mailing List (not for usage questions)

Re: [openstack-dev] [Openstack-operators] [ironic] [nova] [tripleo] Deprecation of Nova's integration with Ironic Capabilities and ComputeCapabilitiesFilter

[John Garbutt]

Back to the deprecation for a moment...

My plan was to tell folks to use Traits to influence placement
decisions, rather than capabilities.

We probably can't remove the feature till we have deploy templates,
but it seems wrong not to warn our users to avoid using capabilities,
when 80% of the use cases can be moved to traits today, and give you
better performance, etc.

Thoughts?

On Mon, 1 Oct 2018 at 22:42, Eric Fried  wrote:
> I do want to zoom out a bit and point out that we're talking about
> implementing a new framework of substantial size and impact when the
> original proposal - using the trait for both - would just work out of
> the box today with no changes in either API. Is it really worth it?

Yeah, I think the simpler solution deals with a lot of the cases right now.

Personally, I see using traits as about hiding complexity from the end
user (not the operator). End users are requesting a host with a given
capability (via flavor, image or otherwise), and they don't really
care if the operator has statically configured it, or Ironic
dynamically configures it. Operator still statically configures what
deploy templates are possible on what nodes (last time I read the
spec).

For the common cases, I see us adding standard traits. They would also
be useful to pick between nodes that are statically configured one way
or the other. (Although MarkG keeps telling me (in a British way) that
is probably rubbish, and he might be right...)

I am +1 Jay's idea for the more complicated cases (a bit like what
jroll was saying). For me, the user (gets bad interop and) has no
visibility into what the crazy custom trait means (i.e. the LAYOUT_Y
in efried's example). A validated blob in Glare doesn't seem terrible
for that special case. But generally that seems like quite a different
use case, and its tempting to focus on something well typed that is
disk configuration specific. Although, it is tempting not to block the
simpler solution, while we work out how people use this for real.

Thanks,
John

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Openstack-operators] [ironic] [nova] [tripleo] Deprecation of Nova's integration with Ironic Capabilities and ComputeCapabilitiesFilter

[Dmitry Tantsur]

On 10/2/18 12:36 AM, Jay Pipes wrote:

On 10/01/2018 06:04 PM, Julia Kreger wrote:

On Mon, Oct 1, 2018 at 2:41 PM Eric Fried  wrote:


 > So say the user requests a node that supports UEFI because their
    image
 > needs UEFI. Which workflow would you want here?
 >
 > 1) The operator (or ironic?) has already configured the node to
    boot in
 > UEFI mode. Only pre-configured nodes advertise the "supports
    UEFI" trait.
 >
 > 2) Any node that supports UEFI mode advertises the trait. Ironic
    ensures
 > that UEFI mode is enabled before provisioning the machine.
 >
 > I imagine doing #2 by passing the traits which were specifically
 > requested by the user, from Nova to Ironic, so that Ironic can do the
 > right thing for the user.
 >
 > Your proposal suggests that the user request the "supports UEFI"
    trait,
 > and *also* pass some glance UUID which the user understands will make
 > sure the node actually boots in UEFI mode. Something like:
 >
 > openstack server create --flavor METAL_12CPU_128G --trait
    SUPPORTS_UEFI
 > --config-data $TURN_ON_UEFI_UUID
 >
 > Note that I pass --trait because I hope that will one day be
    supported
 > and we can slow down the flavor explosion.

    IMO --trait would be making things worse (but see below). I think UEFI
    with Jay's model would be more like:

   openstack server create --flavor METAL_12CPU_128G --config-data $UEFI

    where the UEFI profile would be pretty trivial, consisting of
    placement.traits.required = ["BOOT_MODE_UEFI"] and object.boot_mode =
    "uefi".

    I agree that this seems kind of heavy, and that it would be nice to be
    able to say "boot mode is UEFI" just once. OTOH I get Jay's point that
    we need to separate the placement decision from the instance
    configuration.

    That said, what if it was:

  openstack config-profile create --name BOOT_MODE_UEFI --json -
  {
   "type": "boot_mode_scheme",
   "version": 123,
   "object": {
       "boot_mode": "uefi"
   },
   "placement": {
    "traits": {
     "required": [
      "BOOT_MODE_UEFI"
     ]
    }
   }
  }
  ^D

    And now you could in fact say

  openstack server create --flavor foo --config-profile BOOT_MODE_UEFI

    using the profile name, which happens to be the same as the trait name
    because you made it so. Does that satisfy the yen for saying it once? (I
    mean, despite the fact that you first had to say it three times to get
    it set up.)

    

    I do want to zoom out a bit and point out that we're talking about
    implementing a new framework of substantial size and impact when the
    original proposal - using the trait for both - would just work out of
    the box today with no changes in either API. Is it really worth it?


+1000. Reading both of these threads, it feels like we're basically trying to 
make something perfect. I think that is a fine goal, except it is unrealistic 
because the enemy of good is perfection.


    

    By the way, with Jim's --trait suggestion, this:

 > ...dozens of flavors that look like this:
 > - 12CPU_128G_RAID10_DRIVE_LAYOUT_X
 > - 12CPU_128G_RAID5_DRIVE_LAYOUT_X
 > - 12CPU_128G_RAID01_DRIVE_LAYOUT_X
 > - 12CPU_128G_RAID10_DRIVE_LAYOUT_Y
 > - 12CPU_128G_RAID5_DRIVE_LAYOUT_Y
 > - 12CPU_128G_RAID01_DRIVE_LAYOUT_Y

    ...could actually become:

  openstack server create --flavor 12CPU_128G --trait $WHICH_RAID
    --trait
    $WHICH_LAYOUT

    No flavor explosion.


++ I believe this was where this discussion kind of ended up in.. ?Dublin?

The desire and discussion that led us into complex configuration templates and 
profiles being submitted were for highly complex scenarios where users wanted 
to assert detailed raid configurations to disk. Naturally, there are many 
issues there. The ability to provide such detail would be awesome for those 
10% of operators that need such functionality. Of course, if that is the only 
path forward, then we delay the 90% from getting the minimum viable feature 
they need.



    (Maybe if we called it something other than --trait, like maybe
    --config-option, it would let us pretend we're not really overloading a
    trait to do config - it's just a coincidence that the config option has
    the same name as the trait it causes to be required.)


I feel like it might be confusing, but totally +1 to matching required trait 
name being a thing. That way scheduling is completely decoupled and if 
everything was correct then the request should already be scheduled properly.


I guess I'll just drop the idea of doing this properly then. It's true that the 
placement traits concept can be hacked up and the virt driver can just pass a 
list of trait strings to the Ironic API and that's the most expedient way to get 
what the 90% of people apparently want. It's also 

Re: [openstack-dev] [Openstack-operators] [ironic] [nova] [tripleo] Deprecation of Nova's integration with Ironic Capabilities and ComputeCapabilitiesFilter

[Julia Kreger]

On Mon, Oct 1, 2018 at 3:37 PM Jay Pipes  wrote:

> On 10/01/2018 06:04 PM, Julia Kreger wrote:
> > On Mon, Oct 1, 2018 at 2:41 PM Eric Fried  wrote:
> >
> >
> >  > So say the user requests a node that supports UEFI because their
> > image
> >  > needs UEFI. Which workflow would you want here?
> >  >
> >  > 1) The operator (or ironic?) has already configured the node to
> > boot in
> >  > UEFI mode. Only pre-configured nodes advertise the "supports
> > UEFI" trait.
> >  >
> >  > 2) Any node that supports UEFI mode advertises the trait. Ironic
> > ensures
> >  > that UEFI mode is enabled before provisioning the machine.
> >  >
> >  > I imagine doing #2 by passing the traits which were specifically
> >  > requested by the user, from Nova to Ironic, so that Ironic can do
> the
> >  > right thing for the user.
> >  >
> >  > Your proposal suggests that the user request the "supports UEFI"
> > trait,
> >  > and *also* pass some glance UUID which the user understands will
> make
> >  > sure the node actually boots in UEFI mode. Something like:
> >  >
> >  > openstack server create --flavor METAL_12CPU_128G --trait
> > SUPPORTS_UEFI
> >  > --config-data $TURN_ON_UEFI_UUID
> >  >
> >  > Note that I pass --trait because I hope that will one day be
> > supported
> >  > and we can slow down the flavor explosion.
> >
> > IMO --trait would be making things worse (but see below). I think
> UEFI
> > with Jay's model would be more like:
> >
> >openstack server create --flavor METAL_12CPU_128G --config-data
> $UEFI
> >
> > where the UEFI profile would be pretty trivial, consisting of
> > placement.traits.required = ["BOOT_MODE_UEFI"] and object.boot_mode =
> > "uefi".
> >
> > I agree that this seems kind of heavy, and that it would be nice to
> be
> > able to say "boot mode is UEFI" just once. OTOH I get Jay's point
> that
> > we need to separate the placement decision from the instance
> > configuration.
> >
> > That said, what if it was:
> >
> >   openstack config-profile create --name BOOT_MODE_UEFI --json -
> >   {
> >"type": "boot_mode_scheme",
> >"version": 123,
> >"object": {
> >"boot_mode": "uefi"
> >},
> >"placement": {
> > "traits": {
> >  "required": [
> >   "BOOT_MODE_UEFI"
> >  ]
> > }
> >}
> >   }
> >   ^D
> >
> > And now you could in fact say
> >
> >   openstack server create --flavor foo --config-profile
> BOOT_MODE_UEFI
> >
> > using the profile name, which happens to be the same as the trait
> name
> > because you made it so. Does that satisfy the yen for saying it
> once? (I
> > mean, despite the fact that you first had to say it three times to
> get
> > it set up.)
> >
> > 
> >
> > I do want to zoom out a bit and point out that we're talking about
> > implementing a new framework of substantial size and impact when the
> > original proposal - using the trait for both - would just work out of
> > the box today with no changes in either API. Is it really worth it?
> >
> >
> > +1000. Reading both of these threads, it feels like we're basically
> > trying to make something perfect. I think that is a fine goal, except it
> > is unrealistic because the enemy of good is perfection.
> >
> > 
> >
> > By the way, with Jim's --trait suggestion, this:
> >
> >  > ...dozens of flavors that look like this:
> >  > - 12CPU_128G_RAID10_DRIVE_LAYOUT_X
> >  > - 12CPU_128G_RAID5_DRIVE_LAYOUT_X
> >  > - 12CPU_128G_RAID01_DRIVE_LAYOUT_X
> >  > - 12CPU_128G_RAID10_DRIVE_LAYOUT_Y
> >  > - 12CPU_128G_RAID5_DRIVE_LAYOUT_Y
> >  > - 12CPU_128G_RAID01_DRIVE_LAYOUT_Y
> >
> > ...could actually become:
> >
> >   openstack server create --flavor 12CPU_128G --trait $WHICH_RAID
> > --trait
> > $WHICH_LAYOUT
> >
> > No flavor explosion.
> >
> >
> > ++ I believe this was where this discussion kind of ended up in..
> ?Dublin?
> >
> > The desire and discussion that led us into complex configuration
> > templates and profiles being submitted were for highly complex scenarios
> > where users wanted to assert detailed raid configurations to disk.
> > Naturally, there are many issues there. The ability to provide such
> > detail would be awesome for those 10% of operators that need such
> > functionality. Of course, if that is the only path forward, then we
> > delay the 90% from getting the minimum viable feature they need.
> >
> >
> > (Maybe if we called it something other than --trait, like maybe
> > --config-option, it would let us pretend we're not really
> overloading a
> > trait to do config - it's just a coincidence that the config option
> has
> > the same name as the trait it causes to be required.)
> >
> >
> > I feel like it might 

Re: [openstack-dev] [Openstack-operators] [ironic] [nova] [tripleo] Deprecation of Nova's integration with Ironic Capabilities and ComputeCapabilitiesFilter

[Jay Pipes]

On 10/01/2018 06:04 PM, Julia Kreger wrote:

On Mon, Oct 1, 2018 at 2:41 PM Eric Fried  wrote:


 > So say the user requests a node that supports UEFI because their
image
 > needs UEFI. Which workflow would you want here?
 >
 > 1) The operator (or ironic?) has already configured the node to
boot in
 > UEFI mode. Only pre-configured nodes advertise the "supports
UEFI" trait.
 >
 > 2) Any node that supports UEFI mode advertises the trait. Ironic
ensures
 > that UEFI mode is enabled before provisioning the machine.
 >
 > I imagine doing #2 by passing the traits which were specifically
 > requested by the user, from Nova to Ironic, so that Ironic can do the
 > right thing for the user.
 >
 > Your proposal suggests that the user request the "supports UEFI"
trait,
 > and *also* pass some glance UUID which the user understands will make
 > sure the node actually boots in UEFI mode. Something like:
 >
 > openstack server create --flavor METAL_12CPU_128G --trait
SUPPORTS_UEFI
 > --config-data $TURN_ON_UEFI_UUID
 >
 > Note that I pass --trait because I hope that will one day be
supported
 > and we can slow down the flavor explosion.

IMO --trait would be making things worse (but see below). I think UEFI
with Jay's model would be more like:

   openstack server create --flavor METAL_12CPU_128G --config-data $UEFI

where the UEFI profile would be pretty trivial, consisting of
placement.traits.required = ["BOOT_MODE_UEFI"] and object.boot_mode =
"uefi".

I agree that this seems kind of heavy, and that it would be nice to be
able to say "boot mode is UEFI" just once. OTOH I get Jay's point that
we need to separate the placement decision from the instance
configuration.

That said, what if it was:

  openstack config-profile create --name BOOT_MODE_UEFI --json -
  {
   "type": "boot_mode_scheme",
   "version": 123,
   "object": {
       "boot_mode": "uefi"
   },
   "placement": {
    "traits": {
     "required": [
      "BOOT_MODE_UEFI"
     ]
    }
   }
  }
  ^D

And now you could in fact say

  openstack server create --flavor foo --config-profile BOOT_MODE_UEFI

using the profile name, which happens to be the same as the trait name
because you made it so. Does that satisfy the yen for saying it once? (I
mean, despite the fact that you first had to say it three times to get
it set up.)



I do want to zoom out a bit and point out that we're talking about
implementing a new framework of substantial size and impact when the
original proposal - using the trait for both - would just work out of
the box today with no changes in either API. Is it really worth it?


+1000. Reading both of these threads, it feels like we're basically 
trying to make something perfect. I think that is a fine goal, except it 
is unrealistic because the enemy of good is perfection.




By the way, with Jim's --trait suggestion, this:

 > ...dozens of flavors that look like this:
 > - 12CPU_128G_RAID10_DRIVE_LAYOUT_X
 > - 12CPU_128G_RAID5_DRIVE_LAYOUT_X
 > - 12CPU_128G_RAID01_DRIVE_LAYOUT_X
 > - 12CPU_128G_RAID10_DRIVE_LAYOUT_Y
 > - 12CPU_128G_RAID5_DRIVE_LAYOUT_Y
 > - 12CPU_128G_RAID01_DRIVE_LAYOUT_Y

...could actually become:

  openstack server create --flavor 12CPU_128G --trait $WHICH_RAID
--trait
$WHICH_LAYOUT

No flavor explosion.


++ I believe this was where this discussion kind of ended up in.. ?Dublin?

The desire and discussion that led us into complex configuration 
templates and profiles being submitted were for highly complex scenarios 
where users wanted to assert detailed raid configurations to disk. 
Naturally, there are many issues there. The ability to provide such 
detail would be awesome for those 10% of operators that need such 
functionality. Of course, if that is the only path forward, then we 
delay the 90% from getting the minimum viable feature they need.



(Maybe if we called it something other than --trait, like maybe
--config-option, it would let us pretend we're not really overloading a
trait to do config - it's just a coincidence that the config option has
the same name as the trait it causes to be required.)


I feel like it might be confusing, but totally +1 to matching required 
trait name being a thing. That way scheduling is completely decoupled 
and if everything was correct then the request should already be 
scheduled properly.


I guess I'll just drop the idea of doing this properly then. It's true 
that the placement traits concept can be hacked up and the virt driver 
can just pass a list of trait strings to the Ironic API and that's the 
most expedient way to get what the 90% of people apparently want. It's 
also true that it will add a bunch of 

Re: [openstack-dev] [Openstack-operators] [ironic] [nova] [tripleo] Deprecation of Nova's integration with Ironic Capabilities and ComputeCapabilitiesFilter

[Julia Kreger]

On Mon, Oct 1, 2018 at 2:41 PM Eric Fried  wrote:

>
> > So say the user requests a node that supports UEFI because their image
> > needs UEFI. Which workflow would you want here?
> >
> > 1) The operator (or ironic?) has already configured the node to boot in
> > UEFI mode. Only pre-configured nodes advertise the "supports UEFI" trait.
> >
> > 2) Any node that supports UEFI mode advertises the trait. Ironic ensures
> > that UEFI mode is enabled before provisioning the machine.
> >
> > I imagine doing #2 by passing the traits which were specifically
> > requested by the user, from Nova to Ironic, so that Ironic can do the
> > right thing for the user.
> >
> > Your proposal suggests that the user request the "supports UEFI" trait,
> > and *also* pass some glance UUID which the user understands will make
> > sure the node actually boots in UEFI mode. Something like:
> >
> > openstack server create --flavor METAL_12CPU_128G --trait SUPPORTS_UEFI
> > --config-data $TURN_ON_UEFI_UUID
> >
> > Note that I pass --trait because I hope that will one day be supported
> > and we can slow down the flavor explosion.
>
> IMO --trait would be making things worse (but see below). I think UEFI
> with Jay's model would be more like:
>
>   openstack server create --flavor METAL_12CPU_128G --config-data $UEFI
>
> where the UEFI profile would be pretty trivial, consisting of
> placement.traits.required = ["BOOT_MODE_UEFI"] and object.boot_mode =
> "uefi".
>
> I agree that this seems kind of heavy, and that it would be nice to be
> able to say "boot mode is UEFI" just once. OTOH I get Jay's point that
> we need to separate the placement decision from the instance configuration.
>
> That said, what if it was:
>
>  openstack config-profile create --name BOOT_MODE_UEFI --json -
>  {
>   "type": "boot_mode_scheme",
>   "version": 123,
>   "object": {
>   "boot_mode": "uefi"
>   },
>   "placement": {
>"traits": {
> "required": [
>  "BOOT_MODE_UEFI"
> ]
>}
>   }
>  }
>  ^D
>
> And now you could in fact say
>
>  openstack server create --flavor foo --config-profile BOOT_MODE_UEFI
>
> using the profile name, which happens to be the same as the trait name
> because you made it so. Does that satisfy the yen for saying it once? (I
> mean, despite the fact that you first had to say it three times to get
> it set up.)
>
> 
>
> I do want to zoom out a bit and point out that we're talking about
> implementing a new framework of substantial size and impact when the
> original proposal - using the trait for both - would just work out of
> the box today with no changes in either API. Is it really worth it?
>
>
+1000. Reading both of these threads, it feels like we're basically trying
to make something perfect. I think that is a fine goal, except it is
unrealistic because the enemy of good is perfection.


>
> By the way, with Jim's --trait suggestion, this:
>
> > ...dozens of flavors that look like this:
> > - 12CPU_128G_RAID10_DRIVE_LAYOUT_X
> > - 12CPU_128G_RAID5_DRIVE_LAYOUT_X
> > - 12CPU_128G_RAID01_DRIVE_LAYOUT_X
> > - 12CPU_128G_RAID10_DRIVE_LAYOUT_Y
> > - 12CPU_128G_RAID5_DRIVE_LAYOUT_Y
> > - 12CPU_128G_RAID01_DRIVE_LAYOUT_Y
>
> ...could actually become:
>
>  openstack server create --flavor 12CPU_128G --trait $WHICH_RAID --trait
> $WHICH_LAYOUT
>
> No flavor explosion.
>

++ I believe this was where this discussion kind of ended up in.. ?Dublin?

The desire and discussion that led us into complex configuration templates
and profiles being submitted were for highly complex scenarios where users
wanted to assert detailed raid configurations to disk. Naturally, there are
many issues there. The ability to provide such detail would be awesome for
those 10% of operators that need such functionality. Of course, if that is
the only path forward, then we delay the 90% from getting the minimum
viable feature they need.

>
> (Maybe if we called it something other than --trait, like maybe
> --config-option, it would let us pretend we're not really overloading a
> trait to do config - it's just a coincidence that the config option has
> the same name as the trait it causes to be required.)
>

I feel like it might be confusing, but totally +1 to matching required
trait name being a thing. That way scheduling is completely decoupled and
if everything was correct then the request should already be scheduled
properly.


> -efried
> .
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Openstack-operators] [ironic] [nova] [tripleo] Deprecation of Nova's integration with Ironic Capabilities and ComputeCapabilitiesFilter

[Eric Fried]

> So say the user requests a node that supports UEFI because their image
> needs UEFI. Which workflow would you want here?
> 
> 1) The operator (or ironic?) has already configured the node to boot in
> UEFI mode. Only pre-configured nodes advertise the "supports UEFI" trait.
> 
> 2) Any node that supports UEFI mode advertises the trait. Ironic ensures
> that UEFI mode is enabled before provisioning the machine.
> 
> I imagine doing #2 by passing the traits which were specifically
> requested by the user, from Nova to Ironic, so that Ironic can do the
> right thing for the user.
> 
> Your proposal suggests that the user request the "supports UEFI" trait,
> and *also* pass some glance UUID which the user understands will make
> sure the node actually boots in UEFI mode. Something like:
> 
> openstack server create --flavor METAL_12CPU_128G --trait SUPPORTS_UEFI
> --config-data $TURN_ON_UEFI_UUID
> 
> Note that I pass --trait because I hope that will one day be supported
> and we can slow down the flavor explosion.

IMO --trait would be making things worse (but see below). I think UEFI
with Jay's model would be more like:

  openstack server create --flavor METAL_12CPU_128G --config-data $UEFI

where the UEFI profile would be pretty trivial, consisting of
placement.traits.required = ["BOOT_MODE_UEFI"] and object.boot_mode =
"uefi".

I agree that this seems kind of heavy, and that it would be nice to be
able to say "boot mode is UEFI" just once. OTOH I get Jay's point that
we need to separate the placement decision from the instance configuration.

That said, what if it was:

 openstack config-profile create --name BOOT_MODE_UEFI --json -
 {
  "type": "boot_mode_scheme",
  "version": 123,
  "object": {
  "boot_mode": "uefi"
  },
  "placement": {
   "traits": {
"required": [
 "BOOT_MODE_UEFI"
]
   }
  }
 }
 ^D

And now you could in fact say

 openstack server create --flavor foo --config-profile BOOT_MODE_UEFI

using the profile name, which happens to be the same as the trait name
because you made it so. Does that satisfy the yen for saying it once? (I
mean, despite the fact that you first had to say it three times to get
it set up.)



I do want to zoom out a bit and point out that we're talking about
implementing a new framework of substantial size and impact when the
original proposal - using the trait for both - would just work out of
the box today with no changes in either API. Is it really worth it?



By the way, with Jim's --trait suggestion, this:

> ...dozens of flavors that look like this:
> - 12CPU_128G_RAID10_DRIVE_LAYOUT_X
> - 12CPU_128G_RAID5_DRIVE_LAYOUT_X
> - 12CPU_128G_RAID01_DRIVE_LAYOUT_X
> - 12CPU_128G_RAID10_DRIVE_LAYOUT_Y
> - 12CPU_128G_RAID5_DRIVE_LAYOUT_Y
> - 12CPU_128G_RAID01_DRIVE_LAYOUT_Y

...could actually become:

 openstack server create --flavor 12CPU_128G --trait $WHICH_RAID --trait
$WHICH_LAYOUT

No flavor explosion.

(Maybe if we called it something other than --trait, like maybe
--config-option, it would let us pretend we're not really overloading a
trait to do config - it's just a coincidence that the config option has
the same name as the trait it causes to be required.)

-efried
.

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Openstack-operators] [ironic] [nova] [tripleo] Deprecation of Nova's integration with Ironic Capabilities and ComputeCapabilitiesFilter

[Jim Rollenhagen]

On Mon, Oct 1, 2018 at 10:13 AM Jay Pipes  wrote:

> On 10/01/2018 09:01 AM, Jim Rollenhagen wrote:
> > On Mon, Oct 1, 2018 at 8:03 AM Jay Pipes  > > wrote:
> >
> > On 10/01/2018 04:36 AM, John Garbutt wrote:
> >  > On Fri, 28 Sep 2018 at 00:46, Jay Pipes  > 
> >  > >> wrote:
> >  >
> >  > On 09/27/2018 06:23 PM, Matt Riedemann wrote:
> >  >  > On 9/27/2018 3:02 PM, Jay Pipes wrote:
> >  >  >> A great example of this would be the proposed "deploy
> > template"
> >  > from
> >  >  >> [2]. This is nothing more than abusing the placement
> > traits API in
> >  >  >> order to allow passthrough of instance configuration data
> > from the
> >  >  >> nova flavor extra spec directly into the
> nodes.instance_info
> >  > field in
> >  >  >> the Ironic database. It's a hack that is abusing the
> entire
> >  > concept of
> >  >  >> the placement traits concept, IMHO.
> >  >  >>
> >  >  >> We should have a way *in Nova* of allowing instance
> > configuration
> >  >  >> key/value information to be passed through to the virt
> > driver's
> >  >  >> spawn() method, much the same way we provide for
> > user_data that
> >  > gets
> >  >  >> exposed after boot to the guest instance via configdrive
> > or the
> >  >  >> metadata service API. What this deploy template thing is
> > is just a
> >  >  >> hack to get around the fact that nova doesn't have a
> > basic way of
> >  >  >> passing through some collated instance configuration
> > key/value
> >  >  >> information, which is a darn shame and I'm really kind of
> >  > annoyed with
> >  >  >> myself for not noticing this sooner. :(
> >  >  >
> >  >  > We talked about this in Dublin through right? We said a
> good
> >  > thing to do
> >  >  > would be to have some kind of
> template/profile/config/whatever
> >  > stored
> >  >  > off in glare where schema could be registered on that
> > thing, and
> >  > then
> >  >  > you pass a handle (ID reference) to that to nova when
> > creating the
> >  >  > (baremetal) server, nova pulls it down from glare and
> hands it
> >  > off to
> >  >  > the virt driver. It's just that no one is doing that work.
> >  >
> >  > No, nobody is doing that work.
> >  >
> >  > I will if need be if it means not hacking the placement API
> > to serve
> >  > this purpose (for which it wasn't intended).
> >  >
> >  >
> >  > Going back to the point Mark Goddard made, there are two things
> here:
> >  >
> >  > 1) Picking the correct resource provider
> >  > 2) Telling Ironic to transform the picked node in some way
> >  >
> >  > Today we allow the use Capabilities for both.
> >  >
> >  > I am suggesting we move to using Traits only for (1), leaving (2)
> in
> >  > place for now, while we decide what to do (i.e. future of "deploy
> >  > template" concept).
> >  >
> >  > It feels like Ironic's plan to define the "deploy templates" in
> > Ironic
> >  > should replace the dependency on Glare for this use case, largely
> >  > because the definition of the deploy template (in my mind) is very
> >  > heavily related to inspector and driver properties, etc. Mark is
> > looking
> >  > at moving that forward at the moment.
> >
> > That won't do anything about the flavor explosion problem, though,
> > right
> > John?
> >
> >
> > Does nova still plan to allow passing additional desired traits into the
> > server create request?
> > I (we?) was kind of banking on that to solve the Baskin Robbins thing.
>
> That's precisely what I've been looking into.


Right, well aware of that.


> From what I can tell,
> Ironic was planning on using these CUSTOM_DEPLOY_TEMPLATE_XXX traits in
> two ways:
>
> 1) To tell Nova what scheduling constraints the instance needed -- e.g.
> "hey Nova, make sure I land on a node that supports UEFI boot mode
> because my boot image relies on that".
>
> 2) As a convenient (because it would require no changes to Nova) way of
> passing instance pre-spawn configuration data to the Ironic virt driver
> -- e.g. pass the entire set of traits that are in the RequestSpec's
> flavor and image extra specs to Ironic before calling the Ironic node
> provision API.
>
> #1 is fine IMHO, since it (mostly) represents a "capability" that the
> resource provider (the Ironic baremetal node) must support in order for
> the instance to successfully boot.
>

This is the sort of thing I want to initially target. I understand the
deploy templates thing proposes solving both #1 and #2, but I want to back

Re: [openstack-dev] [Openstack-operators] [ironic] [nova] [tripleo] Deprecation of Nova's integration with Ironic Capabilities and ComputeCapabilitiesFilter

[Jay Pipes]

On 10/01/2018 09:01 AM, Jim Rollenhagen wrote:
On Mon, Oct 1, 2018 at 8:03 AM Jay Pipes > wrote:


On 10/01/2018 04:36 AM, John Garbutt wrote:
 > On Fri, 28 Sep 2018 at 00:46, Jay Pipes mailto:jaypi...@gmail.com>
 > >> wrote:
 >
 >     On 09/27/2018 06:23 PM, Matt Riedemann wrote:
 >      > On 9/27/2018 3:02 PM, Jay Pipes wrote:
 >      >> A great example of this would be the proposed "deploy
template"
 >     from
 >      >> [2]. This is nothing more than abusing the placement
traits API in
 >      >> order to allow passthrough of instance configuration data
from the
 >      >> nova flavor extra spec directly into the nodes.instance_info
 >     field in
 >      >> the Ironic database. It's a hack that is abusing the entire
 >     concept of
 >      >> the placement traits concept, IMHO.
 >      >>
 >      >> We should have a way *in Nova* of allowing instance
configuration
 >      >> key/value information to be passed through to the virt
driver's
 >      >> spawn() method, much the same way we provide for
user_data that
 >     gets
 >      >> exposed after boot to the guest instance via configdrive
or the
 >      >> metadata service API. What this deploy template thing is
is just a
 >      >> hack to get around the fact that nova doesn't have a
basic way of
 >      >> passing through some collated instance configuration
key/value
 >      >> information, which is a darn shame and I'm really kind of
 >     annoyed with
 >      >> myself for not noticing this sooner. :(
 >      >
 >      > We talked about this in Dublin through right? We said a good
 >     thing to do
 >      > would be to have some kind of template/profile/config/whatever
 >     stored
 >      > off in glare where schema could be registered on that
thing, and
 >     then
 >      > you pass a handle (ID reference) to that to nova when
creating the
 >      > (baremetal) server, nova pulls it down from glare and hands it
 >     off to
 >      > the virt driver. It's just that no one is doing that work.
 >
 >     No, nobody is doing that work.
 >
 >     I will if need be if it means not hacking the placement API
to serve
 >     this purpose (for which it wasn't intended).
 >
 >
 > Going back to the point Mark Goddard made, there are two things here:
 >
 > 1) Picking the correct resource provider
 > 2) Telling Ironic to transform the picked node in some way
 >
 > Today we allow the use Capabilities for both.
 >
 > I am suggesting we move to using Traits only for (1), leaving (2) in
 > place for now, while we decide what to do (i.e. future of "deploy
 > template" concept).
 >
 > It feels like Ironic's plan to define the "deploy templates" in
Ironic
 > should replace the dependency on Glare for this use case, largely
 > because the definition of the deploy template (in my mind) is very
 > heavily related to inspector and driver properties, etc. Mark is
looking
 > at moving that forward at the moment.

That won't do anything about the flavor explosion problem, though,
right
John?


Does nova still plan to allow passing additional desired traits into the 
server create request?

I (we?) was kind of banking on that to solve the Baskin Robbins thing.


That's precisely what I've been looking into. From what I can tell, 
Ironic was planning on using these CUSTOM_DEPLOY_TEMPLATE_XXX traits in 
two ways:


1) To tell Nova what scheduling constraints the instance needed -- e.g. 
"hey Nova, make sure I land on a node that supports UEFI boot mode 
because my boot image relies on that".


2) As a convenient (because it would require no changes to Nova) way of 
passing instance pre-spawn configuration data to the Ironic virt driver 
-- e.g. pass the entire set of traits that are in the RequestSpec's 
flavor and image extra specs to Ironic before calling the Ironic node 
provision API.


#1 is fine IMHO, since it (mostly) represents a "capability" that the 
resource provider (the Ironic baremetal node) must support in order for 
the instance to successfully boot.


#2 is a problem, though, because it *doesn't* represent a capability. In 
fact, it can represent any and all sorts of key/value, JSON/dict or 
other information and this information is not intended to be passed to 
the placement/scheduler service as a constraint. It is this data, also, 
that tends to create the flavor explosion problem because it means that 
Ironic deployers need to create dozens of flavors that vary only 
slightly based on a user's desired deployment configuration.


So, deployers end up needing to create dozens of flavors varying only 
slightly by things like RAID level or some pre-defined 

Re: [openstack-dev] [Openstack-operators] [ironic] [nova] [tripleo] Deprecation of Nova's integration with Ironic Capabilities and ComputeCapabilitiesFilter

[Jim Rollenhagen]

On Mon, Oct 1, 2018 at 8:03 AM Jay Pipes  wrote:

> On 10/01/2018 04:36 AM, John Garbutt wrote:
> > On Fri, 28 Sep 2018 at 00:46, Jay Pipes  > > wrote:
> >
> > On 09/27/2018 06:23 PM, Matt Riedemann wrote:
> >  > On 9/27/2018 3:02 PM, Jay Pipes wrote:
> >  >> A great example of this would be the proposed "deploy template"
> > from
> >  >> [2]. This is nothing more than abusing the placement traits API
> in
> >  >> order to allow passthrough of instance configuration data from
> the
> >  >> nova flavor extra spec directly into the nodes.instance_info
> > field in
> >  >> the Ironic database. It's a hack that is abusing the entire
> > concept of
> >  >> the placement traits concept, IMHO.
> >  >>
> >  >> We should have a way *in Nova* of allowing instance configuration
> >  >> key/value information to be passed through to the virt driver's
> >  >> spawn() method, much the same way we provide for user_data that
> > gets
> >  >> exposed after boot to the guest instance via configdrive or the
> >  >> metadata service API. What this deploy template thing is is just
> a
> >  >> hack to get around the fact that nova doesn't have a basic way of
> >  >> passing through some collated instance configuration key/value
> >  >> information, which is a darn shame and I'm really kind of
> > annoyed with
> >  >> myself for not noticing this sooner. :(
> >  >
> >  > We talked about this in Dublin through right? We said a good
> > thing to do
> >  > would be to have some kind of template/profile/config/whatever
> > stored
> >  > off in glare where schema could be registered on that thing, and
> > then
> >  > you pass a handle (ID reference) to that to nova when creating the
> >  > (baremetal) server, nova pulls it down from glare and hands it
> > off to
> >  > the virt driver. It's just that no one is doing that work.
> >
> > No, nobody is doing that work.
> >
> > I will if need be if it means not hacking the placement API to serve
> > this purpose (for which it wasn't intended).
> >
> >
> > Going back to the point Mark Goddard made, there are two things here:
> >
> > 1) Picking the correct resource provider
> > 2) Telling Ironic to transform the picked node in some way
> >
> > Today we allow the use Capabilities for both.
> >
> > I am suggesting we move to using Traits only for (1), leaving (2) in
> > place for now, while we decide what to do (i.e. future of "deploy
> > template" concept).
> >
> > It feels like Ironic's plan to define the "deploy templates" in Ironic
> > should replace the dependency on Glare for this use case, largely
> > because the definition of the deploy template (in my mind) is very
> > heavily related to inspector and driver properties, etc. Mark is looking
> > at moving that forward at the moment.
>
> That won't do anything about the flavor explosion problem, though, right
> John?
>

Does nova still plan to allow passing additional desired traits into the
server create request?
I (we?) was kind of banking on that to solve the Baskin Robbins thing.

// jim


>
> -jay
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Openstack-operators] [ironic] [nova] [tripleo] Deprecation of Nova's integration with Ironic Capabilities and ComputeCapabilitiesFilter

[Jay Pipes]

On 10/01/2018 04:36 AM, John Garbutt wrote:
On Fri, 28 Sep 2018 at 00:46, Jay Pipes > wrote:


On 09/27/2018 06:23 PM, Matt Riedemann wrote:
 > On 9/27/2018 3:02 PM, Jay Pipes wrote:
 >> A great example of this would be the proposed "deploy template"
from
 >> [2]. This is nothing more than abusing the placement traits API in
 >> order to allow passthrough of instance configuration data from the
 >> nova flavor extra spec directly into the nodes.instance_info
field in
 >> the Ironic database. It's a hack that is abusing the entire
concept of
 >> the placement traits concept, IMHO.
 >>
 >> We should have a way *in Nova* of allowing instance configuration
 >> key/value information to be passed through to the virt driver's
 >> spawn() method, much the same way we provide for user_data that
gets
 >> exposed after boot to the guest instance via configdrive or the
 >> metadata service API. What this deploy template thing is is just a
 >> hack to get around the fact that nova doesn't have a basic way of
 >> passing through some collated instance configuration key/value
 >> information, which is a darn shame and I'm really kind of
annoyed with
 >> myself for not noticing this sooner. :(
 >
 > We talked about this in Dublin through right? We said a good
thing to do
 > would be to have some kind of template/profile/config/whatever
stored
 > off in glare where schema could be registered on that thing, and
then
 > you pass a handle (ID reference) to that to nova when creating the
 > (baremetal) server, nova pulls it down from glare and hands it
off to
 > the virt driver. It's just that no one is doing that work.

No, nobody is doing that work.

I will if need be if it means not hacking the placement API to serve
this purpose (for which it wasn't intended).


Going back to the point Mark Goddard made, there are two things here:

1) Picking the correct resource provider
2) Telling Ironic to transform the picked node in some way

Today we allow the use Capabilities for both.

I am suggesting we move to using Traits only for (1), leaving (2) in 
place for now, while we decide what to do (i.e. future of "deploy 
template" concept).


It feels like Ironic's plan to define the "deploy templates" in Ironic 
should replace the dependency on Glare for this use case, largely 
because the definition of the deploy template (in my mind) is very 
heavily related to inspector and driver properties, etc. Mark is looking 
at moving that forward at the moment.


That won't do anything about the flavor explosion problem, though, right 
John?


-jay

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Openstack-operators] [ironic] [nova] [tripleo] Deprecation of Nova's integration with Ironic Capabilities and ComputeCapabilitiesFilter

[John Garbutt]

On Fri, 28 Sep 2018 at 00:46, Jay Pipes  wrote:

> On 09/27/2018 06:23 PM, Matt Riedemann wrote:
> > On 9/27/2018 3:02 PM, Jay Pipes wrote:
> >> A great example of this would be the proposed "deploy template" from
> >> [2]. This is nothing more than abusing the placement traits API in
> >> order to allow passthrough of instance configuration data from the
> >> nova flavor extra spec directly into the nodes.instance_info field in
> >> the Ironic database. It's a hack that is abusing the entire concept of
> >> the placement traits concept, IMHO.
> >>
> >> We should have a way *in Nova* of allowing instance configuration
> >> key/value information to be passed through to the virt driver's
> >> spawn() method, much the same way we provide for user_data that gets
> >> exposed after boot to the guest instance via configdrive or the
> >> metadata service API. What this deploy template thing is is just a
> >> hack to get around the fact that nova doesn't have a basic way of
> >> passing through some collated instance configuration key/value
> >> information, which is a darn shame and I'm really kind of annoyed with
> >> myself for not noticing this sooner. :(
> >
> > We talked about this in Dublin through right? We said a good thing to do
> > would be to have some kind of template/profile/config/whatever stored
> > off in glare where schema could be registered on that thing, and then
> > you pass a handle (ID reference) to that to nova when creating the
> > (baremetal) server, nova pulls it down from glare and hands it off to
> > the virt driver. It's just that no one is doing that work.
>
> No, nobody is doing that work.
>
> I will if need be if it means not hacking the placement API to serve
> this purpose (for which it wasn't intended).
>

Going back to the point Mark Goddard made, there are two things here:

1) Picking the correct resource provider
2) Telling Ironic to transform the picked node in some way

Today we allow the use Capabilities for both.

I am suggesting we move to using Traits only for (1), leaving (2) in place
for now, while we decide what to do (i.e. future of "deploy template"
concept).

It feels like Ironic's plan to define the "deploy templates" in Ironic
should replace the dependency on Glare for this use case, largely because
the definition of the deploy template (in my mind) is very heavily related
to inspector and driver properties, etc. Mark is looking at moving that
forward at the moment.

Thanks,
John
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Openstack-operators] [ironic] [nova] [tripleo] Deprecation of Nova's integration with Ironic Capabilities and ComputeCapabilitiesFilter

[Sylvain Bauza]

On Fri, Sep 28, 2018 at 12:50 AM melanie witt  wrote:

> On Thu, 27 Sep 2018 17:23:26 -0500, Matt Riedemann wrote:
> > On 9/27/2018 3:02 PM, Jay Pipes wrote:
> >> A great example of this would be the proposed "deploy template" from
> >> [2]. This is nothing more than abusing the placement traits API in order
> >> to allow passthrough of instance configuration data from the nova flavor
> >> extra spec directly into the nodes.instance_info field in the Ironic
> >> database. It's a hack that is abusing the entire concept of the
> >> placement traits concept, IMHO.
> >>
> >> We should have a way *in Nova* of allowing instance configuration
> >> key/value information to be passed through to the virt driver's spawn()
> >> method, much the same way we provide for user_data that gets exposed
> >> after boot to the guest instance via configdrive or the metadata service
> >> API. What this deploy template thing is is just a hack to get around the
> >> fact that nova doesn't have a basic way of passing through some collated
> >> instance configuration key/value information, which is a darn shame and
> >> I'm really kind of annoyed with myself for not noticing this sooner. :(
> >
> > We talked about this in Dublin through right? We said a good thing to do
> > would be to have some kind of template/profile/config/whatever stored
> > off in glare where schema could be registered on that thing, and then
> > you pass a handle (ID reference) to that to nova when creating the
> > (baremetal) server, nova pulls it down from glare and hands it off to
> > the virt driver. It's just that no one is doing that work.
>
> If I understood correctly, that discussion was around adding a way to
> pass a desired hardware configuration to nova when booting an ironic
> instance. And that it's something that isn't yet possible to do using
> the existing ComputeCapabilitiesFilter. Someone please correct me if I'm
> wrong there.
>
> That said, I still don't understand why we are talking about deprecating
> the ComputeCapabilitiesFilter if there's no supported way to replace it
> yet. If boolean traits are not enough to replace it, then we need to
> hold off on deprecating it, right? Would the
> template/profile/config/whatever in glare approach replace what the
> ComputeCapabilitiesFilter is doing or no? Sorry, I'm just not clearly
> understanding this yet.
>
>
I just feel some new traits have to be defined, like Jay said, and some
work has to be done on the Ironic side to make sure they expose them as
traits and not by the old way.
That leaves tho a question : does Ironic support custom capabilities ? If
so, that leads to Jay's point about the key/pair information that's not
intented for traits. If we all agree on the fact that traits shouldn't be
allowed for key/value pairs, could we somehow imagine Ironic to change the
customization mechanism to be boolean only ?

Also, I'm a bit confused whether operators make use of Ironic capabilities
for fancy operational queries, like the ones we have in
https://github.com/openstack/nova/blob/3716752/nova/scheduler/filters/extra_specs_ops.py#L24-L35
and if Ironic correctly documents how to put such things into traits ? (eg.
say CUSTOM_I_HAVE_MORE_THAN_2_GPUS)

All of the above makes me a bit worried by a possible
ComputeCapabilitiesFilter deprecation, if we aren't yet able to provide a
clear upgrade path for our users.

-Sylvain

-melanie
>
>
>
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Openstack-operators] [ironic] [nova] [tripleo] Deprecation of Nova's integration with Ironic Capabilities and ComputeCapabilitiesFilter

[Jay Pipes]

On 09/27/2018 06:23 PM, Matt Riedemann wrote:

On 9/27/2018 3:02 PM, Jay Pipes wrote:
A great example of this would be the proposed "deploy template" from 
[2]. This is nothing more than abusing the placement traits API in 
order to allow passthrough of instance configuration data from the 
nova flavor extra spec directly into the nodes.instance_info field in 
the Ironic database. It's a hack that is abusing the entire concept of 
the placement traits concept, IMHO.


We should have a way *in Nova* of allowing instance configuration 
key/value information to be passed through to the virt driver's 
spawn() method, much the same way we provide for user_data that gets 
exposed after boot to the guest instance via configdrive or the 
metadata service API. What this deploy template thing is is just a 
hack to get around the fact that nova doesn't have a basic way of 
passing through some collated instance configuration key/value 
information, which is a darn shame and I'm really kind of annoyed with 
myself for not noticing this sooner. :(


We talked about this in Dublin through right? We said a good thing to do 
would be to have some kind of template/profile/config/whatever stored 
off in glare where schema could be registered on that thing, and then 
you pass a handle (ID reference) to that to nova when creating the 
(baremetal) server, nova pulls it down from glare and hands it off to 
the virt driver. It's just that no one is doing that work.


No, nobody is doing that work.

I will if need be if it means not hacking the placement API to serve 
this purpose (for which it wasn't intended).


-jay

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Openstack-operators] [ironic] [nova] [tripleo] Deprecation of Nova's integration with Ironic Capabilities and ComputeCapabilitiesFilter

[melanie witt]

On Thu, 27 Sep 2018 17:23:26 -0500, Matt Riedemann wrote:

On 9/27/2018 3:02 PM, Jay Pipes wrote:

A great example of this would be the proposed "deploy template" from
[2]. This is nothing more than abusing the placement traits API in order
to allow passthrough of instance configuration data from the nova flavor
extra spec directly into the nodes.instance_info field in the Ironic
database. It's a hack that is abusing the entire concept of the
placement traits concept, IMHO.

We should have a way *in Nova* of allowing instance configuration
key/value information to be passed through to the virt driver's spawn()
method, much the same way we provide for user_data that gets exposed
after boot to the guest instance via configdrive or the metadata service
API. What this deploy template thing is is just a hack to get around the
fact that nova doesn't have a basic way of passing through some collated
instance configuration key/value information, which is a darn shame and
I'm really kind of annoyed with myself for not noticing this sooner. :(


We talked about this in Dublin through right? We said a good thing to do
would be to have some kind of template/profile/config/whatever stored
off in glare where schema could be registered on that thing, and then
you pass a handle (ID reference) to that to nova when creating the
(baremetal) server, nova pulls it down from glare and hands it off to
the virt driver. It's just that no one is doing that work.


If I understood correctly, that discussion was around adding a way to 
pass a desired hardware configuration to nova when booting an ironic 
instance. And that it's something that isn't yet possible to do using 
the existing ComputeCapabilitiesFilter. Someone please correct me if I'm 
wrong there.


That said, I still don't understand why we are talking about deprecating 
the ComputeCapabilitiesFilter if there's no supported way to replace it 
yet. If boolean traits are not enough to replace it, then we need to 
hold off on deprecating it, right? Would the 
template/profile/config/whatever in glare approach replace what the 
ComputeCapabilitiesFilter is doing or no? Sorry, I'm just not clearly 
understanding this yet.


-melanie




__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Openstack-operators] [ironic] [nova] [tripleo] Deprecation of Nova's integration with Ironic Capabilities and ComputeCapabilitiesFilter

[Matt Riedemann]

On 9/27/2018 3:02 PM, Jay Pipes wrote:
A great example of this would be the proposed "deploy template" from 
[2]. This is nothing more than abusing the placement traits API in order 
to allow passthrough of instance configuration data from the nova flavor 
extra spec directly into the nodes.instance_info field in the Ironic 
database. It's a hack that is abusing the entire concept of the 
placement traits concept, IMHO.


We should have a way *in Nova* of allowing instance configuration 
key/value information to be passed through to the virt driver's spawn() 
method, much the same way we provide for user_data that gets exposed 
after boot to the guest instance via configdrive or the metadata service 
API. What this deploy template thing is is just a hack to get around the 
fact that nova doesn't have a basic way of passing through some collated 
instance configuration key/value information, which is a darn shame and 
I'm really kind of annoyed with myself for not noticing this sooner. :(


We talked about this in Dublin through right? We said a good thing to do 
would be to have some kind of template/profile/config/whatever stored 
off in glare where schema could be registered on that thing, and then 
you pass a handle (ID reference) to that to nova when creating the 
(baremetal) server, nova pulls it down from glare and hands it off to 
the virt driver. It's just that no one is doing that work.


--

Thanks,

Matt

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev