Aaaand sorry again, but due to sudden errands I won't be able to attend. Please feel free to use my bluejeans room anyway. I think my position on traits is more or less clear from previous discussions with John, Sam and Eric.
2017-10-24 18:07 GMT+02:00 Dmitry Tantsur <dtant...@redhat.com>: > Sigh, sorry. I forgot that we're moving back to winter time this weekend. > I *think* the time is 3pm UTC then. It seems to be 11am eastern US: > https://www.timeanddate.com/worldclock/converter.html?iso=20 > 171030T150000&p1=37&p2=tz_et. > > > On 10/24/2017 06:00 PM, Dmitry Tantsur wrote: > >> And the winner is Mon, 30 Oct, 2pm UTC! >> >> The bluejeans ID is https://bluejeans.com/757528759 >> (works without plugins in recent FF and Chrome; if it asks to install an >> app, ignore it and look for a link saying "join with browser") >> >> On 10/23/2017 05:02 PM, Dmitry Tantsur wrote: >> >>> Hi all! >>> >>> I'd like to invite you to the discussion of the way to implement traits >>> in >>> ironic and the ironic virt driver. Please vote for the time at >>> https://doodle.com/poll/ts43k98kkvniv8uz. Please vote by EOD tomorrow. >>> >>> Note that it's going to be a technical discussion - please make sure you >>> understand what traits are and why ironic cares about them. See below >>> for more >>> context. >>> >>> We'll probably use my bluejeans account, as it works without plugins in >>> modern >>> browsers. I'll post a meeting ID when we pick the date. >>> >>> >>> On 10/23/2017 04:09 PM, Eric Fried wrote: >>> >>>> We discussed this a little bit further in IRC [1]. We're all in >>>> agreement, but it's worth being precise on a couple of points: >>>> >>>> * We're distinguishing between a "feature" and the "trait" that >>>> represents it in placement. For the sake of this discussion, a >>>> "feature" can (maybe) be switched on or off, but a "trait" can either be >>>> present or absent on a RP. >>>> * It matters *who* can turn a feature on/off. >>>> * If it can be done by virt at spawn time, then it makes sense to >>>> have >>>> the trait on the RP, and you can switch the feature on/off via a >>>> separate extra_spec. >>>> * But if it's e.g. an admin action, and spawn has no control, then >>>> the >>>> trait needs to be *added* whenever the feature is *on*, and *removed* >>>> whenever the feature is *off*. >>>> >>>> [1] >>>> http://eavesdrop.openstack.org/irclogs/%23openstack-nova/% >>>> 23openstack-nova.2017-10-23.log.html#t2017-10-23T13:12:13 >>>> >>>> On 10/23/2017 08:15 AM, Sylvain Bauza wrote: >>>> >>>>> >>>>> >>>>> On Mon, Oct 23, 2017 at 2:54 PM, Eric Fried <openst...@fried.cc >>>>> <mailto:openst...@fried.cc>> wrote: >>>>> >>>>> I agree with Sean. In general terms: >>>>> >>>>> * A resource provider should be marked with a trait if that >>>>> feature >>>>> * Can be turned on or off (whether it's currently on or not); >>>>> or >>>>> * Is always on and can't ever be turned off. >>>>> >>>>> >>>>> No, traits are not boolean. If a resource provider stops providing a >>>>> capability, then the existing related trait should just be removed, >>>>> that's it. >>>>> If you see a trait, that's just means that the related capability for >>>>> the Resource Provider is supported, that's it too. >>>>> >>>>> MHO. >>>>> >>>>> -Sylvain >>>>> >>>>> >>>>> >>>>> * A consumer wanting that feature present (doesn't matter >>>>> whether it's >>>>> on or off) should specify it as a required *trait*. >>>>> * A consumer wanting that feature present and turned on should >>>>> * Specify it as a required trait; AND >>>>> * Indicate that it be turned on via some other mechanism (e.g. >>>>> a >>>>> separate extra_spec). >>>>> >>>>> I believe this satisfies Dmitry's (Ironic's) needs, but also >>>>> Jay's drive >>>>> for placement purity. >>>>> >>>>> Please invite me to the hangout or whatever. >>>>> >>>>> Thanks, >>>>> Eric >>>>> >>>>> On 10/23/2017 07:22 AM, Mooney, Sean K wrote: >>>>> > >>>>> > >>>>> > >>>>> > >>>>> > *From:*Jay Pipes [mailto:jaypi...@gmail.com >>>>> <mailto:jaypi...@gmail.com>] >>>>> > *Sent:* Monday, October 23, 2017 12:20 PM >>>>> > *To:* OpenStack Development Mailing List >>>>> <openstack-dev@lists.openstack.org >>>>> <mailto:openstack-dev@lists.openstack.org>> >>>>> > *Subject:* Re: [openstack-dev] [ironic] ironic and traits >>>>> > >>>>> > >>>>> > >>>>> > Writing from my phone... May I ask that before you proceed >>>>> with any plan >>>>> > that uses traits for state information that we have a hangout >>>>> or >>>>> > videoconference to discuss this? Unfortunately today and >>>>> tomorrow I'm >>>>> > not able to do a hangout but I can do one on Wednesday any >>>>> time of the day. >>>>> > >>>>> > >>>>> > >>>>> > */[Mooney, Sean K] on the uefi boot topic I did bring up at the >>>>> ptg that >>>>> > we wanted to standardizes tratis for “verified boot” /* >>>>> > >>>>> > */that included a trait for uefi secure boot enabled and to >>>>> indicated a >>>>> > hardware root of trust, e.g. intel boot guard or similar/* >>>>> > >>>>> > */we distinctly wanted to be able to tag nova compute hosts >>>>> with those >>>>> > new traits so we could require that vms that request/* >>>>> > >>>>> > */a host with uefi secure boot enabled and a hardware root of >>>>> trust are >>>>> > scheduled only to those nodes. /* >>>>> > >>>>> > */ /* >>>>> > >>>>> > */There are many other examples that effect both vms and bare >>>>> metal such >>>>> > as, ecc/interleaved memory, cluster on die, /* >>>>> > >>>>> > */l3 cache code and data prioritization, vt-d/vt-c, HPET, Hyper >>>>> > threading, power states … all of these feature may be present >>>>> on the >>>>> > platform/* >>>>> > >>>>> > */but I also need to know if they are turned on. Ruling out >>>>> state in >>>>> > traits means all of this logic will eventually get pushed to >>>>> scheduler >>>>> > filters/* >>>>> > >>>>> > */which will be suboptimal long term as more state is tracked. >>>>> Software >>>>> > defined infrastructure may be the future but hardware defined >>>>> software/* >>>>> > >>>>> > */is sadly the present…/* >>>>> > >>>>> > */ /* >>>>> > >>>>> > */I do however think there should be a sperateion between >>>>> asking for a >>>>> > host that provides x with a trait and asking for x to be >>>>> configure via/* >>>>> > >>>>> > */A trait. The trait secure_boot_enabled should never result >>>>> in the >>>>> > feature being enabled It should just find a host with it on. If >>>>> you want/* >>>>> > >>>>> > */To request it to be turned on you would request a host with >>>>> > secure_boot_capable as a trait and have a flavor extra spec or >>>>> image >>>>> > property to request/* >>>>> > >>>>> > */Ironic to enabled it. these are two very different request >>>>> and >>>>> should >>>>> > not be treated the same. /* >>>>> > >>>>> > >>>>> > >>>>> > >>>>> > >>>>> > Lemme know! >>>>> > >>>>> > -jay >>>>> > >>>>> > >>>>> > >>>>> > On Oct 23, 2017 5:01 AM, "Dmitry Tantsur" <dtant...@redhat.com >>>>> <mailto:dtant...@redhat.com> >>>>> > <mailto:dtant...@redhat.com <mailto:dtant...@redhat.com>>> >>>>> wrote: >>>>> > >>>>> > Hi Jay! >>>>> > >>>>> > I appreciate your comments, but I think you're approaching >>>>> the >>>>> > problem from purely VM point of view. Things simply don't >>>>> work the >>>>> > same way in bare metal, at least not if we want to provide >>>>> the same >>>>> > user experience. >>>>> > >>>>> > >>>>> > >>>>> > On Sun, Oct 22, 2017 at 2:25 PM, Jay Pipes < >>>>> jaypi...@gmail.com <mailto:jaypi...@gmail.com> >>>>> > <mailto:jaypi...@gmail.com <mailto:jaypi...@gmail.com>>> >>>>> wrote: >>>>> > >>>>> > Sorry for delay, took a week off before starting a new >>>>> job. >>>>> > Comments inline. >>>>> > >>>>> > On 10/16/2017 12:24 PM, Dmitry Tantsur wrote: >>>>> > >>>>> > Hi all, >>>>> > >>>>> > I promised John to dump my thoughts on traits to >>>>> the >>>>> ML, so >>>>> > here we go :) >>>>> > >>>>> > I see two roles of traits (or kinds of traits) for >>>>> bare metal: >>>>> > 1. traits that say what the node can do already >>>>> (e.g. "the >>>>> > node is >>>>> > doing UEFI boot") >>>>> > 2. traits that say what the node can be >>>>> *configured* to do >>>>> > (e.g. "the node can >>>>> > boot in UEFI mode") >>>>> > >>>>> > >>>>> > There's only one role for traits. #2 above. #1 is state >>>>> > information. Traits are not for state information. >>>>> Traits are >>>>> > only for communicating capabilities of a resource >>>>> provider >>>>> > (baremetal node). >>>>> > >>>>> > >>>>> > >>>>> > These are not different, that's what I'm talking about >>>>> here. No >>>>> > users care about the difference between "this node was put >>>>> in UEFI >>>>> > mode by an operator in advance", "this node was put in UEFI >>>>> mode by >>>>> > an ironic driver on demand" and "this node is always in >>>>> UEFI mode, >>>>> > because it's AARCH64 and it does not have BIOS". These >>>>> situation >>>>> > produce the same result (the node is booted in UEFI mode), >>>>> and >>>>> thus >>>>> > it's up to ironic to hide this difference. >>>>> > >>>>> > >>>>> > >>>>> > My suggestion with traits is one way to do it, I'm not sure >>>>> what you >>>>> > suggest though. >>>>> > >>>>> > >>>>> > >>>>> > >>>>> > For example, let's say we add the following to the >>>>> os-traits >>>>> > library [1] >>>>> > >>>>> > * STORAGE_RAID_0 >>>>> > * STORAGE_RAID_1 >>>>> > * STORAGE_RAID_5 >>>>> > * STORAGE_RAID_6 >>>>> > * STORAGE_RAID_10 >>>>> > >>>>> > The Ironic administrator would add all RAID-related >>>>> traits to >>>>> > the baremetal nodes that had the *capability* of >>>>> supporting that >>>>> > particular RAID setup [2] >>>>> > >>>>> > When provisioned, the baremetal node would either have >>>>> RAID >>>>> > configured in a certain level or not configured at all. >>>>> > >>>>> > >>>>> > A very important note: the Placement API and Nova >>>>> scheduler (or >>>>> > future Ironic scheduler) doesn't care about this. At >>>>> all. >>>>> I know >>>>> > it sounds like I'm being callous, but I'm not. >>>>> Placement and >>>>> > scheduling doesn't care about the state of things. It >>>>> only >>>>> cares >>>>> > about the capabilities of target destinations. That's >>>>> it. >>>>> > >>>>> > >>>>> > >>>>> > Yes, because VMs always start with a clean state, and >>>>> hypervisor is >>>>> > there to ensure that. We don't have this luxury in ironic >>>>> :) E.g. >>>>> > our SNMP driver is not even aware of boot modes (or RAID, >>>>> or BIOS >>>>> > configuration), which does not mean that a node using it >>>>> cannot be >>>>> > in UEFI mode (have a RAID or BIOS pre-configured, etc, >>>>> etc). >>>>> > >>>>> > >>>>> > >>>>> > >>>>> > >>>>> > This seems confusing, but it's actually very >>>>> useful. >>>>> Say, I >>>>> > have a flavor that >>>>> > requests UEFI boot via a trait. It will match both >>>>> the >>>>> nodes >>>>> > that are already in >>>>> > UEFI mode, as well as nodes that can be put in >>>>> UEFI mode. >>>>> > >>>>> > >>>>> > No :) It will only match nodes that have the UEFI >>>>> capability. >>>>> > The set of providers that have the ability to be booted >>>>> via UEFI >>>>> > is *always* a superset of the set of providers that >>>>> *have been >>>>> > booted via UEFI*. Placement and scheduling decisions >>>>> only care >>>>> > about that superset -- the providers with a particular >>>>> capability. >>>>> > >>>>> > >>>>> > >>>>> > Well, no, it will. Again, you're purely basing on the VM >>>>> idea, >>>>> where >>>>> > a VM is always *put* in UEFI mode, no matter how the >>>>> hypervisor >>>>> > looks like. It is simply not the case for us. You have to >>>>> care >>>>> what >>>>> > state the node is, because many drivers cannot change this >>>>> state. >>>>> > >>>>> > >>>>> > >>>>> > >>>>> > >>>>> > This idea goes further with deploy templates (new >>>>> concept >>>>> > we've been thinking >>>>> > about). A flavor can request something like >>>>> CUSTOM_RAID_5, >>>>> > and it will match the >>>>> > nodes that already have RAID 5, or, more >>>>> interestingly, the >>>>> > nodes on which we >>>>> > can build RAID 5 before deployment. The UEFI >>>>> example above >>>>> > can be treated in a >>>>> > similar way. >>>>> > >>>>> > This ends up with two sources of knowledge about >>>>> traits in >>>>> > ironic: >>>>> > 1. Operators setting something they know about >>>>> hardware >>>>> > ("this node is in UEFI >>>>> > mode"), >>>>> > 2. Ironic drivers reporting something they >>>>> > 2.1. know about hardware ("this node is in UEFI >>>>> mode" - >>>>> > again) >>>>> > 2.2. can do about hardware ("I can put this >>>>> node in >>>>> UEFI >>>>> > mode") >>>>> > >>>>> > >>>>> > You're correct that both pieces of information are >>>>> important. >>>>> > However, only the "can do about hardware" part is >>>>> relevant to >>>>> > Placement and Nova. >>>>> > >>>>> > For case #1 we are planning on a new CRUD API to >>>>> set/unset >>>>> > traits for a node. >>>>> > >>>>> > >>>>> > I would *strongly* advise against this. Traits are not >>>>> for >>>>> state >>>>> > information. >>>>> > >>>>> > Instead, consider having a DB (or JSON) schema that >>>>> lists >>>>> state >>>>> > information in fields that are explicitly for that >>>>> state >>>>> > information. >>>>> > >>>>> > For example, a schema that looks like this: >>>>> > >>>>> > { >>>>> > "boot": { >>>>> > "mode": <one of 'bios' or 'uefi'>, >>>>> > "params": <dict> >>>>> > }, >>>>> > "disk": { >>>>> > "raid": { >>>>> > "level": <int>, >>>>> > "controller": <one of 'sw' or 'hw'>, >>>>> > "driver": <string>, >>>>> > "params": <dict> >>>>> > }, ... >>>>> > }, >>>>> > "network": { >>>>> > ... >>>>> > } >>>>> > } >>>>> > >>>>> > etc, etc. >>>>> > >>>>> > Don't use trait strings to represent state information. >>>>> > >>>>> > >>>>> > >>>>> > I don't see an alternative proposal that will satisfy what >>>>> we have >>>>> > to solve. >>>>> > >>>>> > >>>>> > >>>>> > >>>>> > Best, >>>>> > -jay >>>>> > >>>>> > Case #2 is more interesting. We have two options, >>>>> I think: >>>>> > >>>>> > a) Operators still set traits on nodes, drivers >>>>> are simply >>>>> > validating them. E.g. >>>>> > an operators sets CUSTOM_RAID_5, and the node's >>>>> RAID >>>>> > interface checks if it is >>>>> > possible to do. The downside is obvious - with a >>>>> lot of >>>>> > deploy templates >>>>> > available it can be a lot of manual work. >>>>> > >>>>> > b) Drivers report the traits, and they get somehow >>>>> added to >>>>> > the traits provided >>>>> > by an operator. Technically, there are sub-cases >>>>> again: >>>>> > b.1) The new traits API returns a union of >>>>> > operator-provided and >>>>> > driver-provided traits >>>>> > b.2) The new traits API returns only >>>>> operator-provided >>>>> > traits; driver-provided >>>>> > traits are returned e.g. via a new field >>>>> > (node.driver_traits). Then nova will >>>>> > have to merge the lists itself. >>>>> > >>>>> > My personal favorite is the last option: I'd like >>>>> a clear >>>>> > distinction between >>>>> > different "sources" of traits, but I'd also like >>>>> to reduce >>>>> > manual work for >>>>> > operators. >>>>> > >>>>> > A valid counter-argument is: what if an operator >>>>> wants to >>>>> > override a >>>>> > driver-provided trait? E.g. a node can do RAID 5, >>>>> but I >>>>> > don't want this >>>>> > particular node to do it for any reason. I'm not >>>>> sure if >>>>> > it's a valid case, and >>>>> > what to do about it. >>>>> > >>>>> > Let me know what you think. >>>>> > >>>>> > Dmitry >>>>> > >>>>> > >>>>> > [1] >>>>> http://git.openstack.org/cgit/openstack/os-traits/tree/ >>>>> <http://git.openstack.org/cgit/openstack/os-traits/tree/> >>>>> > [2] Based on how many attached disks the node had, the >>>>> presence >>>>> > and abilities of a hardware RAID controller, etc >>>>> > >>>>> > >>>>> > >>>>> > >>>>> ___________________________________________________________ >>>>> _______________ >>>>> > OpenStack Development Mailing List (not for usage >>>>> questions) >>>>> > Unsubscribe: >>>>> > >>>>> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe >>>>> <http://openstack-dev-requ...@lists.openstack.org?subject:un >>>>> subscribe> >>>>> > >>>>> <http://openstack-dev-requ...@lists.openstack.org?subject:un >>>>> subscribe >>>>> <http://openstack-dev-requ...@lists.openstack.org?subject:un >>>>> subscribe>> >>>>> > http://lists.openstack.org/cg >>>>> i-bin/mailman/listinfo/openstack-dev >>>>> <http://lists.openstack.org/cgi-bin/mailman/listinfo/opensta >>>>> ck-dev> >>>>> > >>>>> > >>>>> > >>>>> > >>>>> > _____________________________ >>>>> _____________________________________________ >>>>> > OpenStack Development Mailing List (not for usage >>>>> questions) >>>>> > Unsubscribe: >>>>> > openstack-dev-requ...@lists.op >>>>> enstack.org?subject:unsubscribe >>>>> <http://openstack-dev-requ...@lists.openstack.org?subject:un >>>>> subscribe> >>>>> > >>>>> <http://openstack-dev-requ...@lists.openstack.org?subject:un >>>>> subscribe >>>>> <http://openstack-dev-requ...@lists.openstack.org?subject:un >>>>> subscribe>> >>>>> > >>>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstac >>>>> k-dev >>>>> <http://lists.openstack.org/cgi-bin/mailman/listinfo/opensta >>>>> ck-dev> >>>>> > >>>>> > >>>>> > >>>>> > >>>>> ____________________________________________________________ >>>>> ______________ >>>>> > OpenStack Development Mailing List (not for usage questions) >>>>> > Unsubscribe: >>>>> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe >>>>> <http://openstack-dev-requ...@lists.openstack.org?subject:un >>>>> subscribe> >>>>> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstac >>>>> k-dev >>>>> <http://lists.openstack.org/cgi-bin/mailman/listinfo/opensta >>>>> ck-dev> >>>>> > >>>>> >>>>> ____________________________________________________________ >>>>> ______________ >>>>> OpenStack Development Mailing List (not for usage questions) >>>>> Unsubscribe: >>>>> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe >>>>> <http://openstack-dev-requ...@lists.openstack.org?subject:un >>>>> subscribe> >>>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstac >>>>> k-dev >>>>> <http://lists.openstack.org/cgi-bin/mailman/listinfo/opensta >>>>> ck-dev> >>>>> >>>>> >>>>> >>>>> >>>>> ____________________________________________________________ >>>>> ______________ >>>>> OpenStack Development Mailing List (not for usage questions) >>>>> Unsubscribe: openstack-dev-requ...@lists.op >>>>> enstack.org?subject:unsubscribe >>>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >>>>> >>>>> >>>> ____________________________________________________________ >>>> ______________ >>>> OpenStack Development Mailing List (not for usage questions) >>>> Unsubscribe: openstack-dev-requ...@lists.op >>>> enstack.org?subject:unsubscribe >>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >>>> >>>> >> > > __________________________________________________________________________ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > -- -- -- Dmitry Tantsur --
__________________________________________________________________________ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev