Re: [openstack-dev] [placement] The "intended purpose" of traits
On 09/28/2018 07:23 PM, Mohammed Naser wrote: > On Fri, Sep 28, 2018 at 7:17 PM Chris Dent wrote: >> >> On Fri, 28 Sep 2018, melanie witt wrote: >> >>> I'm concerned about a lot of repetition here and maintenance headache for >>> operators. That's where the thoughts about whether we should provide >>> something like a key-value construct to API callers where they can instead >>> say: >>> >>> * OWNER=CINDER >>> * RAID=10 >>> * NUMA_CELL=0 >>> >>> for each resource provider. >>> >>> If I'm off base with my example, please let me know. I'm not a placement >>> expert. >>> >>> Anyway, I hope that gives an idea of what I'm thinking about in this >>> discussion. I agree we need to pick a direction and go with it. I'm just >>> trying to look out for the experience operators are going to be using this >>> and maintaining it in their deployments. >> >> Despite saying "let's never do this" with regard to having formal >> support for key/values in placement, if we did choose to do it (if >> that's what we chose, I'd live with it), when would we do it? We >> have a very long backlog of features that are not yet done. I >> believe (I hope obviously) that we will be able to accelerate >> placement's velocity with it being extracted, but that won't be >> enough to suddenly be able to do quickly do all the things we have >> on the plate. >> >> Are we going to make people wait for some unknown amount of time, >> in the meantime? While there is a grammar that could do some of >> these things? >> >> Unless additional resources come on the scene I don't think is >> either feasible or reasonable for us to considering doing any model >> extending at this time (irrespective of the merit of the idea). >> >> In some kind of weird belief way I'd really prefer we keep the >> grammar placement exposes simple, because my experience with HTTP >> APIs strongly suggests that's very important, and that experience is >> effectively why I am here, but I have no interest in being a >> fundamentalist about it. We should argue about it strongly to make >> sure we get the right result, but it's not a huge deal either way. > > Is there a spec up for this should anyone want to implement it? By "this" are you referring to a placement key/value primitive? There is not a spec or blueprint that I'm aware of. And I think the reason is the strong and immediate resistance to the very idea any time it is mentioned. Who would want to write a spec that's almost certain to be vetoed? > >> -- >> Chris Dent ٩◔̯◔۶ https://anticdent.org/ >> freenode: cdent tw: >> @anticdent__ >> OpenStack Development Mailing List (not for usage questions) >> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > > > __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [placement] The "intended purpose" of traits
Hi All, I'm quite late to the discussion, because I'm on vacation and I missed the beginning of this thread, but let me share a few thoughts. On Fri, Sep 28, 2018 at 6:13 PM Jay Pipes wrote: > * Does the provider belong to physical network "corpnet" and also > support creation of virtual NICs of type either "DIRECT" or "NORMAL"? I'd like to split this question into two, because I think modeling vnic_types and physnets as traits are different. I'll start with the simpler: vnic_types. I may have missed some of the arguments in this very long thread, but I honestly do not see what is the problem with vnic_type traits. These are true capabilities of the backend - not binary though. When it comes to DIRECT and NORMAL the difference is basically if the backend can do SR-IOV or not. On the other hand I have my reservations about physnet traits. I have an item on my todo list to look into Placement aggregates and explore if those are better representing a physnet. Before committing to using aggregates for physnets I know I should fully discover the aggregates API though. And mention one concern which could lead to a usability problem today: aggregates seem to have no names. I think they should. The operator is helpless without them. On Fri, Sep 28, 2018 at 11:51 PM Jay Pipes wrote: > That's correct, because you're encoding >1 piece of information into the > single string (the fact that it's a temperature *and* the value of that > temperature are the two pieces of information encoded into the single > string). > > Now that there's multiple pieces of information encoded in the string > the reader of the trait string needs to know how to decode those bits of > information, which is exactly what we're trying to avoid doing [...]. Technically Placement traits today can be used as a covert communication channel. And doing that is tempting. One component encodes information into a trait name. Another reads it (ie. the trait on the allocated RP) and decodes it. Maybe that trait wasn't influencing placement at all. This is the metadata use case. (If it is a use case at all.) I think the most problematic is when we unknowingly mix placement-influencing info and effectless-metadata into a single blob (as a trait name). One good way to avoid this is to fully and actively discourage the use of traits as a covert communication channel. I can totally support that. I want to mention that in the work-in-progress implementation of the minimum guaranteed bandwidth we considered and then conciously avoided using this covert communication channel. Neutron agents and servers use their usual communication channels to share resource information between them. None of them ever decodes a trait name. All we ever ask of them after allocation is this: Are you responsible for this RP UUID? (For example see https://review.openstack.org/574783.) Cheers, Bence __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [placement] The "intended purpose" of traits
>> I still want to use something like "Is capable of RAID5" and/or "Has >> RAID5 already configured" as part of a scheduling and placement >> decision. Being able to have the GET /a_c response filtered down to >> providers with those, ahem, traits is the exact purpose of that operation. > > And yep, I have zero problem with this either, as I've noted. This is > precisely what placement and traits were designed for. Same. >> While we're in the neighborhood, we agreed in Denver to use a trait to >> indicate which service "owns" a provider [1], so we can eventually >> coordinate a smooth handoff of e.g. a device provider from nova to >> cyborg. This is certainly not a capability (but it is a trait), and it >> can certainly be construed as a key/value (owning_service=cyborg). Are >> we rescinding that decision? > > Unfortunately I have zero recollection of a conversation about using > traits for indicating who "owns" a provider. :( I definitely do. > I don't think I would support such a thing -- rather, I would support > adding an attribute to the provider model itself for an owning service > or such thing. > > That's a great example of where the attribute has specific conceptual > meaning to placement (the concept of ownership) and should definitely > not be tucked away, encoded into a trait string. No, as I recall it means nothing to placement - it means something to the consumers. A gentleperson's agreement for identifying who owns what if we're going to, say, remove things that might be stale from placement when updating the provider tree. --Dan __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [placement] The "intended purpose" of traits
On 10/01/2018 01:20 PM, Eric Fried wrote:
I agree that we should not overload placement as a mechanism to pass
configuration information ("set up RAID5 on my storage, please") to the
driver. So let's put that aside. (Looking forward to that spec.)
ack.
I still want to use something like "Is capable of RAID5" and/or "Has
RAID5 already configured" as part of a scheduling and placement
decision. Being able to have the GET /a_c response filtered down to
providers with those, ahem, traits is the exact purpose of that operation.
And yep, I have zero problem with this either, as I've noted. This is
precisely what placement and traits were designed for.
While we're in the neighborhood, we agreed in Denver to use a trait to
indicate which service "owns" a provider [1], so we can eventually
coordinate a smooth handoff of e.g. a device provider from nova to
cyborg. This is certainly not a capability (but it is a trait), and it
can certainly be construed as a key/value (owning_service=cyborg). Are
we rescinding that decision?
Unfortunately I have zero recollection of a conversation about using
traits for indicating who "owns" a provider. :(
I don't think I would support such a thing -- rather, I would support
adding an attribute to the provider model itself for an owning service
or such thing.
That's a great example of where the attribute has specific conceptual
meaning to placement (the concept of ownership) and should definitely
not be tucked away, encoded into a trait string.
OK, I'll get back to that spec now... :)
Best,
-jay
[1] https://review.openstack.org/#/c/602160/
I'm working on a spec that will describe a way for the user to instruct
Nova to pass configuration data to the virt driver (or device manager)
before instance spawn. This will have nothing to do with placement or
traits, since this configuration data is not modeling scheduling and
placement decisions.
I hope to have that spec done by Monday so we can discuss on the spec.
Best,
-jay
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [placement] The "intended purpose" of traits
On 09/29/2018 10:40 AM, Jay Pipes wrote:
> On 09/28/2018 04:36 PM, Eric Fried wrote:
>> So here it is. Two of the top influencers in placement, one saying we
>> shouldn't overload traits, the other saying we shouldn't add a primitive
>> that would obviate the need for that. Historically, this kind of
>> disagreement seems to result in an impasse: neither thing happens and
>> those who would benefit are forced to find a workaround or punt.
>> Frankly, I don't particularly care which way we go; I just want to be
>> able to do the things.
>
> I don't think that's a fair statement. You absolutely *do* care which
> way we go. You want to encode multiple bits of information into a trait
> string -- such as "PCI_ADDRESS_01_AB_23_CD" -- and leave it up to the
> caller to have to understand that this trait string has multiple bits of
> information encoded in it (the fact that it's a PCI device and that the
> PCI device is at 01_AB_23_CD).
It was an oversimplification to say I don't care. I would like, ideally,
long-term, to see a true key/value primitive, because I think it's much
more powerful and less hacky. But am sympathetic to what Chris brought
up about full plate and timeline. So while we're waiting for that to fit
into the schedule, I wouldn't mind the ability to use encoded traits to
some extent to satisfy the use cases.
> You don't see a problem encoding these variants inside a string. Chris
> doesn't either.
Yeah, I see the problem, and I don't love the idea - as I say, I would
prefer a true key/value primitive. But I would rather use encoded traits
as a temporary measure to get stuff done than a) work around things with
a mess of extra specs and/or b) wait, potentially until the heat death
of the universe if we remain deadlocked on whether a key/value primitive
should happen.
>
> I *do* see a problem with it, based on my experience in Nova where this
> kind of thing leads to ugly, unmaintainable, and incomprehensible code
> as I have pointed to in previous responses.
>
> Furthermore, your point isn't that "you just want to be able to do the
> things". Your point (and the point of others, from Cyborg and Ironic) is
> that you want to be able to use placement to pass various bits of
> information to an instance, and placement wasn't designed for that
> purpose. Nova was.
>
> So, instead of working out a solution with the Nova team for passing
> configuration data about an instance, the proposed solution is instead
> to hack/encode multiple bits of information into a trait string. This
> proposed solution is seen as a way around having to work out a more
> appropriate solution that has Nova pass that configuration data (as is
> appropriate, since nova is the project that manages instances) to the
> virt driver or generic device manager (i.e. Cyborg) before the instance
> spawns.
I agree that we should not overload placement as a mechanism to pass
configuration information ("set up RAID5 on my storage, please") to the
driver. So let's put that aside. (Looking forward to that spec.)
I still want to use something like "Is capable of RAID5" and/or "Has
RAID5 already configured" as part of a scheduling and placement
decision. Being able to have the GET /a_c response filtered down to
providers with those, ahem, traits is the exact purpose of that operation.
While we're in the neighborhood, we agreed in Denver to use a trait to
indicate which service "owns" a provider [1], so we can eventually
coordinate a smooth handoff of e.g. a device provider from nova to
cyborg. This is certainly not a capability (but it is a trait), and it
can certainly be construed as a key/value (owning_service=cyborg). Are
we rescinding that decision?
[1] https://review.openstack.org/#/c/602160/
> I'm working on a spec that will describe a way for the user to instruct
> Nova to pass configuration data to the virt driver (or device manager)
> before instance spawn. This will have nothing to do with placement or
> traits, since this configuration data is not modeling scheduling and
> placement decisions.
>
> I hope to have that spec done by Monday so we can discuss on the spec.
>
> Best,
> -jay
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [placement] The "intended purpose" of traits
> It sounds like you might be saying, "I would rather not see encoded > trait names OR a new key/value primitive; but if the alternative is > ending up with 'a much larger mess', I would accept..." ...which? > > Or is it, "We should not implement a key/value primitive, nor should we > implement restrictions on trait names; but we should continue to > discourage (ab)use of trait names by steering placement consumers to..." > ...do what? The second one. > The restriction is real, not perceived. Without key/value (either > encoded or explicit), how should we steer placement consumers to satisfy > e.g., "Give me disk from a provider with RAID5"? Sure, I'm not doubting the need to find providers with certain abilities. What I'm saying (and I assume Jay is as well), is that finding things with more domain-specific attributes is the job of the domain controller (i.e. nova). Placement's strength, IMHO, is the unified and extremely simple data model and consistency guarantees that it provides. It takes a lot of the work of searching and atomic accounting of enumerable and qualitative things out of the scheduler of the consumer. IMHO, it doesn't (i.e. won't ever) and shouldn't replace all the things that nova's scheduler needs to do. I think it's useful to draw the line in front of a full-blown key=value store and DSL grammar for querying everything with all the operations anyone could ever need. Unifying the simpler and more common bits into placement and keeping the domain-specific consideration and advanced filtering of the results in nova/ironic/etc is the right separation of responsibilities, IMHO. RAID level is, of course, an overly simplistic example to use, which makes the problem seem small, but we know more complicated examples exist. --Dan __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [placement] The "intended purpose" of traits
Dan- On 10/01/2018 10:06 AM, Dan Smith wrote: > I was out when much of this conversation happened, so I'm going to > summarize my opinion here. > >> So from a code perspective _placement_ is completely agnostic to >> whether a trait is "PCI_ADDRESS_01_AB_23_CD", "STORAGE_DISK_SSD", or >> "JAY_LIKES_CRUNCHIE_BARS". >> >> However, things which are using traits (e.g., nova, ironic) need to >> make their own decisions about how the value of traits are >> interpreted. I don't have a strong position on that except to say >> that _if_ we end up in a position of there being lots of traits >> willy nilly, people who have chosen to do that need to know that the >> contract presented by traits right now (present or not present, no >> value comprehension) is fixed. > > I agree with what Chris holds sacred here, which is that placement > shouldn't ever care about what the trait names are or what they mean to > someone else. That also extends to me hoping we never implement a > generic key=value store on resource providers in placement. > >>> I *do* see a problem with it, based on my experience in Nova where >>> this kind of thing leads to ugly, unmaintainable, and >>> incomprehensible code as I have pointed to in previous responses. > > I definitely agree with what Jay holds sacred here, which is that > abusing the data model to encode key=value information into single trait > strings is bad (which is what you're doing with something like > PCI_ADDRESS_01_AB_23_CD). > > I don't want placement (the code) to try to put any technical > restrictions on the meaning of trait names, in an attempt to try to > prevent the above abuse. I agree that means people _can_ abuse it if > they wish, which I think is Chris' point. However, I think it _is_ > important for the placement team (the people) to care about how > consumers (nova, etc) use traits, and thus provide guidance on that is > necessary. Not everyone will follow that guidance, but we should provide > it. Projects with history-revering developers on both sides of the fence > can help this effort if they lead by example. > > If everyone goes off and implements their way around the perceived > restriction of not being able to ask placement for RAID_LEVEL>=5, we're > going to have a much larger mess than the steaming pile of extra specs > in nova that we're trying to avoid. Sorry, I'm having a hard time understanding where you're landing here. It sounds like you might be saying, "I would rather not see encoded trait names OR a new key/value primitive; but if the alternative is ending up with 'a much larger mess', I would accept..." ...which? Or is it, "We should not implement a key/value primitive, nor should we implement restrictions on trait names; but we should continue to discourage (ab)use of trait names by steering placement consumers to..." ...do what? The restriction is real, not perceived. Without key/value (either encoded or explicit), how should we steer placement consumers to satisfy e.g., "Give me disk from a provider with RAID5"? (Put aside the ability to do comparisons other than straight equality - so limiting the discussion to RAID_LEVEL=5, ignoring RAID_LEVEL>=5. Also limiting the discussion to what we want out of GET /a_c - so this excludes, "And then go configure RAID5 on my storage.") > > --Dan > > __ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [placement] The "intended purpose" of traits
I was out when much of this conversation happened, so I'm going to summarize my opinion here. > So from a code perspective _placement_ is completely agnostic to > whether a trait is "PCI_ADDRESS_01_AB_23_CD", "STORAGE_DISK_SSD", or > "JAY_LIKES_CRUNCHIE_BARS". > > However, things which are using traits (e.g., nova, ironic) need to > make their own decisions about how the value of traits are > interpreted. I don't have a strong position on that except to say > that _if_ we end up in a position of there being lots of traits > willy nilly, people who have chosen to do that need to know that the > contract presented by traits right now (present or not present, no > value comprehension) is fixed. I agree with what Chris holds sacred here, which is that placement shouldn't ever care about what the trait names are or what they mean to someone else. That also extends to me hoping we never implement a generic key=value store on resource providers in placement. >> I *do* see a problem with it, based on my experience in Nova where >> this kind of thing leads to ugly, unmaintainable, and >> incomprehensible code as I have pointed to in previous responses. I definitely agree with what Jay holds sacred here, which is that abusing the data model to encode key=value information into single trait strings is bad (which is what you're doing with something like PCI_ADDRESS_01_AB_23_CD). I don't want placement (the code) to try to put any technical restrictions on the meaning of trait names, in an attempt to try to prevent the above abuse. I agree that means people _can_ abuse it if they wish, which I think is Chris' point. However, I think it _is_ important for the placement team (the people) to care about how consumers (nova, etc) use traits, and thus provide guidance on that is necessary. Not everyone will follow that guidance, but we should provide it. Projects with history-revering developers on both sides of the fence can help this effort if they lead by example. If everyone goes off and implements their way around the perceived restriction of not being able to ask placement for RAID_LEVEL>=5, we're going to have a much larger mess than the steaming pile of extra specs in nova that we're trying to avoid. --Dan __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [placement] The "intended purpose" of traits
> So from a code perspective _placement_ is completely agnostic to > whether a trait is "PCI_ADDRESS_01_AB_23_CD", "STORAGE_DISK_SSD", or > "JAY_LIKES_CRUNCHIE_BARS". Right, but words have meanings, and everyone is better off if that meaning is common amongst everyone doing the talking. So if placement understands traits as "a unitary piece of information that is either true or false" (ex: HAS_SSD), but nova understands it as "multiple pieces of information, all of which are either true or false" (ex: HAS_PCI_DE_AD_BE_EF), then that's asking for trouble. Can it work out? Probably, but it'll be more by accident that by design, sort of like French and Spanish sharing certain words, but then having some similar sounding words mean something completely different. > However, things which are using traits (e.g., nova, ironic) need to > make their own decisions about how the value of traits are > interpreted. Well... if placement is saying "here's the primitives I can work with and can expose to my users", but nova is saying "well, we like this one primitive, but what we really need is this other primitive, and you don't have it, but we can totally hack this first primitive that you do have to do what we want"... That's ugly. From what I understand, Nova needs *resources* (not resources providers) to have *quantities* of things, and this is not something that placement can currently work with, which is why we're having this flamewar ;) > I don't have a strong position on that except to say > that _if_ we end up in a position of there being lots of traits > willy nilly, people who have chosen to do that need to know that the > contract presented by traits right now (present or not present, no > value comprehension) is fixed. > > > I *do* see a problem with it, based on my experience in Nova where this kind > > of thing leads to ugly, unmaintainable, and incomprehensible code as I have > > pointed to in previous responses. > > I think there are many factors that have led to nova being > incomprehensible and indeed bad representations is one of them, but > I think reasonable people can disagree on which factors are the most > important and with sufficient discussion come to some reasonable > compromises. I personally feel that while the bad representations > (encoding stuff in strings or json blobs) thing is a big deal, > another major factor is a predilection to make new apis, new > abstractions, and new representations rather than working with and > adhering to the constraints of the existing ones. This leads to a > lot of code that encodes business logic in itself (e.g., several > different ways and layers of indirection to think about allocation > ratios) rather than working within strong and constraining > contracts. > > From my standpoint there isn't much to talk about here from a > placement code standpoint. We should clearly document the functional > contract (and stick to it) and we should come up with exemplars > for how to make the best use of traits. > > I think this conversation could allow us to find those examples. > > I don't, however, want placement to be a traffic officer for how > people do things. In the context of the orchestration between nova > and ironic and how that interaction happens, nova has every right to > set some guidelines if it needs to. > > -- > Chris Dent ٩◔̯◔۶ https://anticdent.org/ > freenode: cdent tw: > @anticdent__ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev -- -- Artom Lifshitz Software Engineer, OpenStack Compute DFG __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [placement] The "intended purpose" of traits
On 28/09/18 1:19 PM, Chris Dent wrote: They aren't arbitrary. They are there for a reason: a trait is a boolean capability. It describes something that either a provider is capable of supporting or it isn't. This is somewhat (maybe even only slightly) different from what I think the definition of a trait is, and that nuance may be relevant. I describe a trait as a "quality that a resource provider has" (the car is blue). This contrasts with a resource class which is a "quantity that a resource provider has" (the car has 4 doors). I'm not sure that quality vs. quantity is actually the right distinction here... someone could equally argue that having 4 doors is itself a quality[1] of a car, and they could certainly come up with a formulation that obscures the role of quantity at all (the car is a sedan). I think the actual distinction you're describing is between discrete (or perhaps just enumerable) and continuous (or at least innumerable) values. What that misses is that if the car is blue, it cannot also be green. Since placement absolutely should not know anything at all about the meaning of traits, this means that clients will be required to implement a bunch of business logic to maintain consistency. Furthermore, should the colour of the car change from blue to green at some point in the future[2], I am assuming that placement will not offer an API that allows both traits to be updated atomically. Those are problems that key-value solves. It could be the case that those problems are not considered important in this context; if so I'd expect to see the reasons explained as part of this discussion. cheers, Zane. [1] Resisting the urge to quote Stalin here. [2] https://en.wikipedia.org/wiki/New_riddle_of_induction __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [placement] The "intended purpose" of traits
On Sat, 29 Sep 2018, Jay Pipes wrote: I don't think that's a fair statement. You absolutely *do* care which way we go. You want to encode multiple bits of information into a trait string -- such as "PCI_ADDRESS_01_AB_23_CD" -- and leave it up to the caller to have to understand that this trait string has multiple bits of information encoded in it (the fact that it's a PCI device and that the PCI device is at 01_AB_23_CD). You don't see a problem encoding these variants inside a string. Chris doesn't either. Lest I be misconstrued, I'd like to clarify: What I was trying to say elsewhere in the thread was that placement should never be aware of _anything_ that is in the trait string (except CUSTOM_* when validating ones that are added, and MISC_SHARES[...] for sharing providers). On the placement server side, input is compared solely for equality with stored data and nothing else, and we should never allow value comparisons, string fragments, regex, etc. So from a code perspective _placement_ is completely agnostic to whether a trait is "PCI_ADDRESS_01_AB_23_CD", "STORAGE_DISK_SSD", or "JAY_LIKES_CRUNCHIE_BARS". However, things which are using traits (e.g., nova, ironic) need to make their own decisions about how the value of traits are interpreted. I don't have a strong position on that except to say that _if_ we end up in a position of there being lots of traits willy nilly, people who have chosen to do that need to know that the contract presented by traits right now (present or not present, no value comprehension) is fixed. I *do* see a problem with it, based on my experience in Nova where this kind of thing leads to ugly, unmaintainable, and incomprehensible code as I have pointed to in previous responses. I think there are many factors that have led to nova being incomprehensible and indeed bad representations is one of them, but I think reasonable people can disagree on which factors are the most important and with sufficient discussion come to some reasonable compromises. I personally feel that while the bad representations (encoding stuff in strings or json blobs) thing is a big deal, another major factor is a predilection to make new apis, new abstractions, and new representations rather than working with and adhering to the constraints of the existing ones. This leads to a lot of code that encodes business logic in itself (e.g., several different ways and layers of indirection to think about allocation ratios) rather than working within strong and constraining contracts. From my standpoint there isn't much to talk about here from a placement code standpoint. We should clearly document the functional contract (and stick to it) and we should come up with exemplars for how to make the best use of traits. I think this conversation could allow us to find those examples. I don't, however, want placement to be a traffic officer for how people do things. In the context of the orchestration between nova and ironic and how that interaction happens, nova has every right to set some guidelines if it needs to. -- Chris Dent ٩◔̯◔۶ https://anticdent.org/ freenode: cdent tw: @anticdent__ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [placement] The "intended purpose" of traits
On Sat, Sep 29, 2018 at 12:02 PM Ed Leafe wrote:
> On Sep 28, 2018, at 12:19 PM, Chris Dent wrote:
> > I don't think placement should be concerned about temporal aspects
> > of traits. If we can't write a web service that can handle setting
> > lots of traits every second of every day, we should go home. If
> > clients of placement want to set weird traits, more power to them.
> >
> > However, if clients of placement (such as nova) which are being the
> > orchestrator of resource providers manipulated by multiple systems
> > (neutron, cinder, ironic, cyborg, etc) wish to set some constraints
> > on how and what traits can do and mean, then that is up to them.
> It is up to the clients to determine how to use Placement. But it is up to
> Placement to give guidance as to how to best use it. If a client wants to
> hack trait names, then they certainly can, and it might work out just fine.
>
It is only up to the clients to determine how to use Placement, if the
Placement team specifies this as an intent. It's very important for users
of the system to be in alignment with the providers of the system to avoid
surprise and complications. Yes, users can hang themselves with enough rope
- but this is a reckless position to hold. Let us say that Placement and
Nova evolve traits according to a roadmap that ends up causing the users to
get stuck on an old release because they have no migration path because
they used the API in a way that was never intended, and never acknowledged.
I'm reading along with interest, and the main issue I see here is what Jay
referred to, which is that there is a process to get well formulated ideas
agreed upon and incorporated, and it does seem like this is a threat to do
something that works without following this process, not because it is
smart, but because it is possible.
I have a similar dilemma on a different system which has to do with the
proper use of labels in a system like Jira. Jira issue labels are intended
to be more flexible and defined according to user requirements to meet
needs not envisioned by the designers of the issue field schemes. They work
well well for certain ad-hoc uses, but they had side effects and
limitations which make them very poor for many uses. A key one is related
to this discussion, in that you can't easily evaluate only part of a label,
so if you did non-advisable things like to use labels instead of priority,
and you had multiple label values to indicate each priority, you end up
with rather silly evaluations like "label in ('priority_high',
'priority_medium', 'priority_low')". This gets more complex over time and
ends up being a huge burden for whoever inherits such a system. Yes, it was
to solve an original problem - but the smart person would have addressed
this with the issue field scheme and would have prevented this technical
debt burden from existing in the first place.
I think you should push Jay to propose what he thinks a good solution would
be, and start discussing these options. I'm interested in reading more
about these options.
I really don't like the idea of embedding data bits into a single string
that effectively includes both a "key" and a "value" as my own experience
suggests this is a very bad idea.
Myself, I think "traits" are not necessarily boolean. I see "brown hair" as
a human trait. However, if they are not boolean than this invites the need
to evaluate with comparison operators, and this would morph into a much
more complex system.
The idea of passing configuration data in might solve a subset of the cases
- and it might solve your subset. But, I do think there is a more general
solution possible if the requirements warranted the development.
I also want to add that I think traits should not be dynamic things that
change from second to second. The idea of passing temperature information
via traits sounded somewhat ridiculous to me. I think that might have been
the intent of the original poster to present a ridiculous example and hope
people understood. I hope nobody was taking it seriously. :-)
--
Mark Mielke
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [placement] The "intended purpose" of traits
On Sep 28, 2018, at 12:19 PM, Chris Dent wrote: > > I don't think placement should be concerned about temporal aspects > of traits. If we can't write a web service that can handle setting > lots of traits every second of every day, we should go home. If > clients of placement want to set weird traits, more power to them. > > However, if clients of placement (such as nova) which are being the > orchestrator of resource providers manipulated by multiple systems > (neutron, cinder, ironic, cyborg, etc) wish to set some constraints > on how and what traits can do and mean, then that is up to them. This. It is up to the clients to determine how to use Placement. But it is up to Placement to give guidance as to how to best use it. If a client wants to hack trait names, then they certainly can, and it might work out just fine. > On Sep 28, 2018, at 8:25 AM, Eric Fried wrote: > > Bubble wrap was originally intended as a textured wallpaper and a > greenhouse insulator. Can we accept the fact that traits have (many, > many) uses beyond marking capabilities, and quit with the arbitrary > restrictions? The crux here is what one considers "arbitrary". If we as a project state "sure, go ahead and use this however you like", we are going to be complicit in the technical debt they accumulate, as Jay has referenced with regards to Nova's ComputeCapabilities filter. We want consumers of Placement to write good, solid code that doesn't encourage technical debt accumulation. We have no way to prevent bad decisions by others, but we should never document that such usages are fine. -- Ed Leafe signature.asc Description: Message signed with OpenPGP __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [placement] The "intended purpose" of traits
On Sep 29, 2018, at 10:40 AM, Jay Pipes wrote: > >> So here it is. Two of the top influencers in placement, one saying we >> shouldn't overload traits, the other saying we shouldn't add a primitive >> that would obviate the need for that. Historically, this kind of >> disagreement seems to result in an impasse: neither thing happens and >> those who would benefit are forced to find a workaround or punt. >> Frankly, I don't particularly care which way we go; I just want to be >> able to do the things. > > I don't think that's a fair statement. You absolutely *do* care which way we > go. You want to encode multiple bits of information into a trait string -- > such as "PCI_ADDRESS_01_AB_23_CD" -- and leave it up to the caller to have to > understand that this trait string has multiple bits of information encoded in > it (the fact that it's a PCI device and that the PCI device is at > 01_AB_23_CD). > > You don't see a problem encoding these variants inside a string. Chris > doesn't either. > > I *do* see a problem with it, based on my experience in Nova where this kind > of thing leads to ugly, unmaintainable, and incomprehensible code as I have > pointed to in previous responses. I think that there is huge difference between the Placement service stating "this is how you use this service" and actively preventing others from doing dumb things with Placement. If we, as a team, tell others that it is OK to manage state, or use a trait name to encode multiple bits of information, then others will be more likely to do just that, and end up with an unreadable mess around their part of the code that works with placement. The result will be a perception among others along the lines of "placement sucks". If we state clearly that this is not a good way to work with Placement, and they do so anyway, well, that's on them. So we shouldn't enforce anything about trait names except the custom namespace and the length. If other service want to be overly clever and try to overload a trait name, it's up to them to deal with the resulting mess. But in no way should we *ever* encourage, even tacitly, this approach. -- Ed Leafe signature.asc Description: Message signed with OpenPGP __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [placement] The "intended purpose" of traits
On 09/28/2018 04:36 PM, Eric Fried wrote: So here it is. Two of the top influencers in placement, one saying we shouldn't overload traits, the other saying we shouldn't add a primitive that would obviate the need for that. Historically, this kind of disagreement seems to result in an impasse: neither thing happens and those who would benefit are forced to find a workaround or punt. Frankly, I don't particularly care which way we go; I just want to be able to do the things. I don't think that's a fair statement. You absolutely *do* care which way we go. You want to encode multiple bits of information into a trait string -- such as "PCI_ADDRESS_01_AB_23_CD" -- and leave it up to the caller to have to understand that this trait string has multiple bits of information encoded in it (the fact that it's a PCI device and that the PCI device is at 01_AB_23_CD). You don't see a problem encoding these variants inside a string. Chris doesn't either. I *do* see a problem with it, based on my experience in Nova where this kind of thing leads to ugly, unmaintainable, and incomprehensible code as I have pointed to in previous responses. Furthermore, your point isn't that "you just want to be able to do the things". Your point (and the point of others, from Cyborg and Ironic) is that you want to be able to use placement to pass various bits of information to an instance, and placement wasn't designed for that purpose. Nova was. So, instead of working out a solution with the Nova team for passing configuration data about an instance, the proposed solution is instead to hack/encode multiple bits of information into a trait string. This proposed solution is seen as a way around having to work out a more appropriate solution that has Nova pass that configuration data (as is appropriate, since nova is the project that manages instances) to the virt driver or generic device manager (i.e. Cyborg) before the instance spawns. I'm working on a spec that will describe a way for the user to instruct Nova to pass configuration data to the virt driver (or device manager) before instance spawn. This will have nothing to do with placement or traits, since this configuration data is not modeling scheduling and placement decisions. I hope to have that spec done by Monday so we can discuss on the spec. Best, -jay __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [placement] The "intended purpose" of traits
On Fri, 28 Sep 2018 at 22:07, melanie witt wrote: > On Fri, 28 Sep 2018 15:42:23 -0500, Eric Fried wrote: > > On 09/28/2018 09:41 AM, Balázs Gibizer wrote: > >> > >> > >> On Fri, Sep 28, 2018 at 3:25 PM, Eric Fried wrote: > >>> It's time somebody said this. > >>> > >>> Every time we turn a corner or look under a rug, we find another use > >>> case for provider traits in placement. But every time we have to have > >>> the argument about whether that use case satisfies the original > >>> "intended purpose" of traits. > >>> > >>> That's only reason I've ever been able to glean: that it (whatever "it" > >>> is) wasn't what the architects had in mind when they came up with the > >>> idea of traits. We're not even talking about anything that would > require > >>> changes to the placement API. Just, "Oh, that's not a *capability* - > >>> shut it down." > >>> > >>> Bubble wrap was originally intended as a textured wallpaper and a > >>> greenhouse insulator. Can we accept the fact that traits have (many, > >>> many) uses beyond marking capabilities, and quit with the arbitrary > >>> restrictions? > >> > >> How far are we willing to go? Does an arbitrary (key: value) pair > >> encoded in a trait name like key_`str(value)` (e.g. CURRENT_TEMPERATURE: > >> 85 encoded as CUSTOM_TEMPERATURE_85) something we would be OK to see in > >> placement? > > > > Great question. Perhaps TEMPERATURE_DANGEROUSLY_HIGH is okay, but > > TEMPERATURE_ is not. This thread isn't about setting > > these parameters; it's about getting us to a point where we can discuss > > a question just like this one without running up against: > > > > "That's a hard no, because you shouldn't encode key/value pairs in > traits." > > > > "Oh, why's that?" > > > > "Because that's not what we intended when we created traits." > > > > "But it would work, and the alternatives are way harder." > > > > "-1" > > > > "But..." > > > > "-1" > I think it's not so much about the intention when traits were created > and more about what UX callers of the API are left with, if we were to > recommend representing everything with traits and not providing another > API for key-value use cases. We need to think about what the maintenance > of their deployments will look like if traits are the only tool we provide. > > I get that we don't want to put artificial restrictions on how API > callers can and can't use the traits API, but will they be left with a > manageable experience if that's all that's available? > > I don't have time right now to come up with a really great example, but > I'm thinking along the lines of, can this get out of hand (a la "flavor > explosion") for an operator using traits to model what their compute > hosts can do? > > Please forgive the oversimplified example I'm going to try to use to > illustrate my concern: > > We all agree we can have traits for resource providers like: > > * HAS_SSD > * HAS_GPU > * HAS_WINDOWS > > But things get less straightforward when we think of traits like: > > * HAS_OWNER_CINDER > * HAS_OWNER_NEUTRON > * HAS_OWNER_CYBORG > * HAS_RAID_0 > * HAS_RAID_1 > * HAS_RAID_5 > * HAS_RAID_6 > * HAS_RAID_10 > I think the numbers are a red herring here. RAID levels include a limited set of combinations, of which only a handful are frequently used. It's not the same as the temperature example, which is a continuous range of numbers. That said, a key:value encoding could work well for RAID. * HAS_NUMA_CELL_0 > * HAS_NUMA_CELL_1 > * HAS_NUMA_CELL_2 > * HAS_NUMA_CELL_3 > > I'm concerned about a lot of repetition here and maintenance headache > for operators. That's where the thoughts about whether we should provide > something like a key-value construct to API callers where they can > instead say: > > * OWNER=CINDER > * RAID=10 > * NUMA_CELL=0 > > for each resource provider. > > If I'm off base with my example, please let me know. I'm not a placement > expert. > > Anyway, I hope that gives an idea of what I'm thinking about in this > discussion. I agree we need to pick a direction and go with it. I'm just > trying to look out for the experience operators are going to be using > this and maintaining it in their deployments. > > Cheers, > -melanie > > > > > > > > > > > > > > __ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [placement] The "intended purpose" of traits
To add some context around what I suspect is the reason for the most recent incarnation of this debate, many Ironic users have a requirement to be able to influence the configuration of a server at deploy time, beyond the existing supported mechanisms. The classic example is hardware RAID - the ability to support workloads with different requirements is important, since if you're paying for bare metal cloud resources you'll want to make sure you're getting the most out of them. Another example that comes up is hyperthreading - often this is disabled for HPC workloads but enabled for HTC. We've had a plan to support deploy-time configuration that has existed for a few cycles. It began with adding support for traits [1] in Queens, and continued with the deploy steps framework [2] in Rocky. At the Stein PTG we had a lot of support [3] for finishing the job by implementing the deploy templates [4] spec that is currently in review. At a very high level, deploy templates allow us to map a reuired trait specified on a flavor to a set of deploy steps in ironic. These deploy steps are based on the existing cleaning steps framework that has existed in ironic for many releases, and should feel familiar to users of ironic. This scheme is conceptually quite simple, which I like. After a negative review on the spec from Jay on Thursday, I added a design to the alternatives section of the spec that I thought might align better with his view of the world. Essentially, decouple the scheduling and configuration - flavors may specify required traits as they can today, but also a more explicit list of names or UUIDs of ironic deploy templates. I'm still not sure how I feel about this. Architecturally it's cleaner, and is more flexible but from a usability perspective feels a little clunky. There was a discussion [5] in ironic's IRC yesterday that I missed, in which Jay offered to write up an alternative spec that uses glance metadata [6]. There were some concerns about adding a hard requirement on glance for the standalone use case, but we may be able to provide an alternative solution analogous to manual cleaning that fills that gap. I'm certainly interested to see what Jay comes up with. If there is a better way of doing this, I'm all ears. That said, this is something the ironic community has been wanting for a long time now, and I can't see us waiting for a multi-cycle feature to land in nova, given that deploy templates currently requires no changes in nova. [1] http://specs.openstack.org/openstack/ironic-specs/specs/10.1/node-traits.html [2] https://specs.openstack.org/openstack/ironic-specs/specs/11.1/deployment-steps-framework.html [3] https://etherpad.openstack.org/p/ironic-stein-ptg-goals [4] https://review.openstack.org/#/c/504952/ [5] http://eavesdrop.openstack.org/irclogs/%23openstack-ironic/%23openstack-ironic.2018-09-28.log.html#t2018-09-28T14:22:57 [6] https://docs.openstack.org/glance/pike/user/metadefs-concepts.html On Sat, 29 Sep 2018 at 03:15, Alex Xu wrote: > Sorry for append another email for something I missed to say. > > Alex Xu 于2018年9月29日周六 上午10:01写道： > >> >> >> Jay Pipes 于2018年9月29日周六 上午5:51写道： >> >>> On 09/28/2018 04:42 PM, Eric Fried wrote: >>> > On 09/28/2018 09:41 AM, Balázs Gibizer wrote: >>> >> On Fri, Sep 28, 2018 at 3:25 PM, Eric Fried >>> wrote: >>> >>> It's time somebody said this. >>> >>> >>> >>> Every time we turn a corner or look under a rug, we find another use >>> >>> case for provider traits in placement. But every time we have to have >>> >>> the argument about whether that use case satisfies the original >>> >>> "intended purpose" of traits. >>> >>> >>> >>> That's only reason I've ever been able to glean: that it (whatever >>> "it" >>> >>> is) wasn't what the architects had in mind when they came up with the >>> >>> idea of traits. We're not even talking about anything that would >>> require >>> >>> changes to the placement API. Just, "Oh, that's not a *capability* - >>> >>> shut it down." >>> >>> >>> >>> Bubble wrap was originally intended as a textured wallpaper and a >>> >>> greenhouse insulator. Can we accept the fact that traits have (many, >>> >>> many) uses beyond marking capabilities, and quit with the arbitrary >>> >>> restrictions? >>> >> >>> >> How far are we willing to go? Does an arbitrary (key: value) pair >>> >> encoded in a trait name like key_`str(value)` (e.g. >>> CURRENT_TEMPERATURE: >>> >> 85 encoded as CUSTOM_TEMPERATURE_85) something we would be OK to see >>> in >>> >> placement? >>> > >>> > Great question. Perhaps TEMPERATURE_DANGEROUSLY_HIGH is okay, but >>> > TEMPERATURE_ is not. >>> >>> That's correct, because you're encoding >1 piece of information into the >>> single string (the fact that it's a temperature *and* the value of that >>> temperature are the two pieces of information encoded into the single >>> string). >>> >>> Now that there's multiple pieces of information encoded in the string >>> the reader of the trait string needs to know
Re: [openstack-dev] [placement] The "intended purpose" of traits
Sorry for append another email for something I missed to say. Alex Xu 于2018年9月29日周六 上午10:01写道： > > > Jay Pipes 于2018年9月29日周六 上午5:51写道： > >> On 09/28/2018 04:42 PM, Eric Fried wrote: >> > On 09/28/2018 09:41 AM, Balázs Gibizer wrote: >> >> On Fri, Sep 28, 2018 at 3:25 PM, Eric Fried >> wrote: >> >>> It's time somebody said this. >> >>> >> >>> Every time we turn a corner or look under a rug, we find another use >> >>> case for provider traits in placement. But every time we have to have >> >>> the argument about whether that use case satisfies the original >> >>> "intended purpose" of traits. >> >>> >> >>> That's only reason I've ever been able to glean: that it (whatever >> "it" >> >>> is) wasn't what the architects had in mind when they came up with the >> >>> idea of traits. We're not even talking about anything that would >> require >> >>> changes to the placement API. Just, "Oh, that's not a *capability* - >> >>> shut it down." >> >>> >> >>> Bubble wrap was originally intended as a textured wallpaper and a >> >>> greenhouse insulator. Can we accept the fact that traits have (many, >> >>> many) uses beyond marking capabilities, and quit with the arbitrary >> >>> restrictions? >> >> >> >> How far are we willing to go? Does an arbitrary (key: value) pair >> >> encoded in a trait name like key_`str(value)` (e.g. >> CURRENT_TEMPERATURE: >> >> 85 encoded as CUSTOM_TEMPERATURE_85) something we would be OK to see in >> >> placement? >> > >> > Great question. Perhaps TEMPERATURE_DANGEROUSLY_HIGH is okay, but >> > TEMPERATURE_ is not. >> >> That's correct, because you're encoding >1 piece of information into the >> single string (the fact that it's a temperature *and* the value of that >> temperature are the two pieces of information encoded into the single >> string). >> >> Now that there's multiple pieces of information encoded in the string >> the reader of the trait string needs to know how to decode those bits of >> information, which is exactly what we're trying to avoid doing (because >> we can see from the ComputeCapabilitiesFilter, the extra_specs mess, and >> the giant hairball that is the NUMA and CPU pinning "metadata requests" >> how that turns out). >> > > May I understand the one of Jay's complain is about metadata API > undiscoverable? That is extra_spec mess and ComputeCapabilitiesFilter mess? > If yes, then we resolve the discoverable by the "/Traits" API. > > Another complain is about the information in the string. Agree with that > TEMPERATURE_ is terriable. > I prefer the way I used in nvdimm proposal now, I don't want to use Trait > NVDIMM_DEVICE_500GB, NVDIMM_DEVICE_1024GB. I want to put them into the > different resource provider, and use min_size, max_size limit the > allocation. And the user will request with resource class like > RC_NVDIMM_GB=512. > TEMPERATURE_ is wrong, as the way using it. But I don't thing the version of BIOS is wrong, I don't expect the end user to ready the information from the trait directly, there should document from the admin to explain more. The version of BIOS should be a thing understand by the admin, then it is enough. > >> >> > This thread isn't about setting these parameters; it's about getting >> > us to a point where we can discuss a question just like this one >> > without running up against: > >> > "That's a hard no, because you shouldn't encode key/value pairs in >> traits." >> > >> > "Oh, why's that?" >> > >> > "Because that's not what we intended when we created traits." >> > >> > "But it would work, and the alternatives are way harder." >> > >> > "-1" >> > >> > "But..." >> > >> > "-I >> >> I believe I've articulated a number of times why traits should remain >> unary pieces of information, and not just said "because that's what we >> intended when we created traits". >> >> I'm tough on this because I've seen the garbage code and unmaintainable >> mess that not having structurally sound data modeling concepts and >> information interpretation rules leads to in Nova and I don't want to >> encourage any more of it. >> >> -jay >> >> >> __ >> OpenStack Development Mailing List (not for usage questions) >> Unsubscribe: >> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >> > __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [placement] The "intended purpose" of traits
Jay Pipes 于2018年9月29日周六 上午5:51写道： > On 09/28/2018 04:42 PM, Eric Fried wrote: > > On 09/28/2018 09:41 AM, Balázs Gibizer wrote: > >> On Fri, Sep 28, 2018 at 3:25 PM, Eric Fried wrote: > >>> It's time somebody said this. > >>> > >>> Every time we turn a corner or look under a rug, we find another use > >>> case for provider traits in placement. But every time we have to have > >>> the argument about whether that use case satisfies the original > >>> "intended purpose" of traits. > >>> > >>> That's only reason I've ever been able to glean: that it (whatever "it" > >>> is) wasn't what the architects had in mind when they came up with the > >>> idea of traits. We're not even talking about anything that would > require > >>> changes to the placement API. Just, "Oh, that's not a *capability* - > >>> shut it down." > >>> > >>> Bubble wrap was originally intended as a textured wallpaper and a > >>> greenhouse insulator. Can we accept the fact that traits have (many, > >>> many) uses beyond marking capabilities, and quit with the arbitrary > >>> restrictions? > >> > >> How far are we willing to go? Does an arbitrary (key: value) pair > >> encoded in a trait name like key_`str(value)` (e.g. CURRENT_TEMPERATURE: > >> 85 encoded as CUSTOM_TEMPERATURE_85) something we would be OK to see in > >> placement? > > > > Great question. Perhaps TEMPERATURE_DANGEROUSLY_HIGH is okay, but > > TEMPERATURE_ is not. > > That's correct, because you're encoding >1 piece of information into the > single string (the fact that it's a temperature *and* the value of that > temperature are the two pieces of information encoded into the single > string). > > Now that there's multiple pieces of information encoded in the string > the reader of the trait string needs to know how to decode those bits of > information, which is exactly what we're trying to avoid doing (because > we can see from the ComputeCapabilitiesFilter, the extra_specs mess, and > the giant hairball that is the NUMA and CPU pinning "metadata requests" > how that turns out). > May I understand the one of Jay's complain is about metadata API undiscoverable? That is extra_spec mess and ComputeCapabilitiesFilter mess? Another complain is about the information in the string. Agree with that TEMPERATURE_ is terriable. I prefer the way I used in nvdimm proposal now, I don't want to use Trait NVDIMM_DEVICE_500GB, NVDIMM_DEVICE_1024GB. I want to put them into the different resource provider, and use min_size, max_size limit the allocation. And the user will request with resource class like RC_NVDIMM_GB=512. > > > This thread isn't about setting these parameters; it's about getting > > us to a point where we can discuss a question just like this one > > without running up against: > > > "That's a hard no, because you shouldn't encode key/value pairs in > traits." > > > > "Oh, why's that?" > > > > "Because that's not what we intended when we created traits." > > > > "But it would work, and the alternatives are way harder." > > > > "-1" > > > > "But..." > > > > "-I > > I believe I've articulated a number of times why traits should remain > unary pieces of information, and not just said "because that's what we > intended when we created traits". > > I'm tough on this because I've seen the garbage code and unmaintainable > mess that not having structurally sound data modeling concepts and > information interpretation rules leads to in Nova and I don't want to > encourage any more of it. > > -jay > > > __ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [placement] The "intended purpose" of traits
Chris Dent 于2018年9月29日周六 上午1:19写道： > On Fri, 28 Sep 2018, Jay Pipes wrote: > > > On 09/28/2018 09:25 AM, Eric Fried wrote: > >> It's time somebody said this. > > Yes, a useful topic, I think. > ++, I'm interesting this topic also, since it confuses me for a long time... > > >> Every time we turn a corner or look under a rug, we find another use > >> case for provider traits in placement. But every time we have to have > >> the argument about whether that use case satisfies the original > >> "intended purpose" of traits. > >> > >> That's only reason I've ever been able to glean: that it (whatever "it" > >> is) wasn't what the architects had in mind when they came up with the > >> idea of traits. > > > > Don't pussyfoot around things. It's me you're talking about, Eric. You > could > > just ask me instead of passive-aggressively posting to the list like > this. > > It's not just you. Ed and I have also expressed some fairly strong > statement about how traits are "supposed" to be used and I would > guess that from Eric's perspective all three of us (amongst others) > have some form of architectural influence. Since it takes a village > and all that. > > > They aren't arbitrary. They are there for a reason: a trait is a boolean > > capability. It describes something that either a provider is capable of > > supporting or it isn't. > > This is somewhat (maybe even only slightly) different from what I > think the definition of a trait is, and that nuance may be relevant. > > I describe a trait as a "quality that a resource provider has" (the > car is blue). This contrasts with a resource class which is a > "quantity that a resource provider has" (the car has 4 doors). > > Yes, this is what I'm thinking when I propose the Trait. Basically, I'm trying to match two points in the proposal: #1 we need qualitative of resource, #2 we don't want another metadata API, since metadata API isn't discoverable and wild place, people put anything to it. Nobody knows what metadata available in the code except deep into the code. For #1, just as Chris said. For #2, You have to create Trait before using it, and we have API to query traits, make it discoverable in the API. And standard trait make its naming has rule, then as Jay suggested, we have os-traits library to store all the standard traits. But we have to have custom trait, since there have use-case for managing resource out of OpenStack. > Our implementation is pretty much exactly that ^. We allow > clients to ask "give me things that have qualities x, y, z, not > qualities a, b, c, and quanities of G of 5 and H of 7". > > Add in aggregates and we have exactly what you say: > > > * Does the provider have *capacity* for the requested resources? > > * Does the provider have the required (or forbidden) *capabilities*? > > * Does the provider belong to some group? > > The nuance of difference is that your description of *capabilities* > seems more narrow than my description of *qualities* (aka > characteristics). You've got something fairly specific in mind, as a > way of constraining the profusion of noise that has happened with > how various kinds of information about resources of all sorts is > managed in OpenStack, as you describe in your message. > > I do not think it should be placement's job to control that noise. > It should be placement's job to provide a very strict contract about > what you can do with a trait: > > * create it, if necessary > * assign it to one or more resource providers > * ask for providers that either have it > * ... or do not have it > > That's all. Placement _code_ should _never_ be aware of the value of > a trait (except for the magical MISC_SHARES...). It should never > become possible to regex on traits or do comparisons > (required= ++ > > > If we want to add further constraints to the placement allocation > candidates > > request that ask things like: > > > > * Does the provider have version 1.22.61821 of BIOS firmware from > Marvell > > installed on it? > > That's a quality of the provider in a moment. > > > * Does the provider support an FPGA that has had an OVS program flashed > to it > > in the last 20 days? > > If you squint, so is this. > > > * Does the provider belong to physical network "corpnet" and also > support > > creation of virtual NICs of type either "DIRECT" or "NORMAL"? > > And these. > > But at least some of them are dynamic rather than some kind of > platonic ideal associated with the resource provider. > > I don't think placement should be concerned about temporal aspects > of traits. If we can't write a web service that can handle setting > lots of traits every second of every day, we should go home. If > clients of placement want to set weird traits, more power to them. > > However, if clients of placement (such as nova) which are being the > orchestrator of resource providers manipulated by multiple systems > (neutron, cinder, ironic, cyborg, etc) wish to set some constraints > on how and what traits
Re: [openstack-dev] [placement] The "intended purpose" of traits
On Fri, Sep 28, 2018 at 7:17 PM Chris Dent wrote: > > On Fri, 28 Sep 2018, melanie witt wrote: > > > I'm concerned about a lot of repetition here and maintenance headache for > > operators. That's where the thoughts about whether we should provide > > something like a key-value construct to API callers where they can instead > > say: > > > > * OWNER=CINDER > > * RAID=10 > > * NUMA_CELL=0 > > > > for each resource provider. > > > > If I'm off base with my example, please let me know. I'm not a placement > > expert. > > > > Anyway, I hope that gives an idea of what I'm thinking about in this > > discussion. I agree we need to pick a direction and go with it. I'm just > > trying to look out for the experience operators are going to be using this > > and maintaining it in their deployments. > > Despite saying "let's never do this" with regard to having formal > support for key/values in placement, if we did choose to do it (if > that's what we chose, I'd live with it), when would we do it? We > have a very long backlog of features that are not yet done. I > believe (I hope obviously) that we will be able to accelerate > placement's velocity with it being extracted, but that won't be > enough to suddenly be able to do quickly do all the things we have > on the plate. > > Are we going to make people wait for some unknown amount of time, > in the meantime? While there is a grammar that could do some of > these things? > > Unless additional resources come on the scene I don't think is > either feasible or reasonable for us to considering doing any model > extending at this time (irrespective of the merit of the idea). > > In some kind of weird belief way I'd really prefer we keep the > grammar placement exposes simple, because my experience with HTTP > APIs strongly suggests that's very important, and that experience is > effectively why I am here, but I have no interest in being a > fundamentalist about it. We should argue about it strongly to make > sure we get the right result, but it's not a huge deal either way. Is there a spec up for this should anyone want to implement it? > -- > Chris Dent ٩◔̯◔۶ https://anticdent.org/ > freenode: cdent tw: > @anticdent__ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev -- Mohammed Naser — vexxhost - D. 514-316-8872 D. 800-910-1726 ext. 200 E. mna...@vexxhost.com W. http://vexxhost.com __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [placement] The "intended purpose" of traits
On Fri, 28 Sep 2018, melanie witt wrote: I'm concerned about a lot of repetition here and maintenance headache for operators. That's where the thoughts about whether we should provide something like a key-value construct to API callers where they can instead say: * OWNER=CINDER * RAID=10 * NUMA_CELL=0 for each resource provider. If I'm off base with my example, please let me know. I'm not a placement expert. Anyway, I hope that gives an idea of what I'm thinking about in this discussion. I agree we need to pick a direction and go with it. I'm just trying to look out for the experience operators are going to be using this and maintaining it in their deployments. Despite saying "let's never do this" with regard to having formal support for key/values in placement, if we did choose to do it (if that's what we chose, I'd live with it), when would we do it? We have a very long backlog of features that are not yet done. I believe (I hope obviously) that we will be able to accelerate placement's velocity with it being extracted, but that won't be enough to suddenly be able to do quickly do all the things we have on the plate. Are we going to make people wait for some unknown amount of time, in the meantime? While there is a grammar that could do some of these things? Unless additional resources come on the scene I don't think is either feasible or reasonable for us to considering doing any model extending at this time (irrespective of the merit of the idea). In some kind of weird belief way I'd really prefer we keep the grammar placement exposes simple, because my experience with HTTP APIs strongly suggests that's very important, and that experience is effectively why I am here, but I have no interest in being a fundamentalist about it. We should argue about it strongly to make sure we get the right result, but it's not a huge deal either way. -- Chris Dent ٩◔̯◔۶ https://anticdent.org/ freenode: cdent tw: @anticdent__ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [placement] The "intended purpose" of traits
On 09/28/2018 04:42 PM, Eric Fried wrote: On 09/28/2018 09:41 AM, Balázs Gibizer wrote: On Fri, Sep 28, 2018 at 3:25 PM, Eric Fried wrote: It's time somebody said this. Every time we turn a corner or look under a rug, we find another use case for provider traits in placement. But every time we have to have the argument about whether that use case satisfies the original "intended purpose" of traits. That's only reason I've ever been able to glean: that it (whatever "it" is) wasn't what the architects had in mind when they came up with the idea of traits. We're not even talking about anything that would require changes to the placement API. Just, "Oh, that's not a *capability* - shut it down." Bubble wrap was originally intended as a textured wallpaper and a greenhouse insulator. Can we accept the fact that traits have (many, many) uses beyond marking capabilities, and quit with the arbitrary restrictions? How far are we willing to go? Does an arbitrary (key: value) pair encoded in a trait name like key_`str(value)` (e.g. CURRENT_TEMPERATURE: 85 encoded as CUSTOM_TEMPERATURE_85) something we would be OK to see in placement? Great question. Perhaps TEMPERATURE_DANGEROUSLY_HIGH is okay, but TEMPERATURE_ is not. That's correct, because you're encoding >1 piece of information into the single string (the fact that it's a temperature *and* the value of that temperature are the two pieces of information encoded into the single string). Now that there's multiple pieces of information encoded in the string the reader of the trait string needs to know how to decode those bits of information, which is exactly what we're trying to avoid doing (because we can see from the ComputeCapabilitiesFilter, the extra_specs mess, and the giant hairball that is the NUMA and CPU pinning "metadata requests" how that turns out). This thread isn't about setting these parameters; it's about getting us to a point where we can discuss a question just like this one without running up against: > "That's a hard no, because you shouldn't encode key/value pairs in traits." "Oh, why's that?" "Because that's not what we intended when we created traits." "But it would work, and the alternatives are way harder." "-1" "But..." "-I I believe I've articulated a number of times why traits should remain unary pieces of information, and not just said "because that's what we intended when we created traits". I'm tough on this because I've seen the garbage code and unmaintainable mess that not having structurally sound data modeling concepts and information interpretation rules leads to in Nova and I don't want to encourage any more of it. -jay __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [placement] The "intended purpose" of traits
On Fri, 28 Sep 2018 15:42:23 -0500, Eric Fried wrote: On 09/28/2018 09:41 AM, Balázs Gibizer wrote: On Fri, Sep 28, 2018 at 3:25 PM, Eric Fried wrote: It's time somebody said this. Every time we turn a corner or look under a rug, we find another use case for provider traits in placement. But every time we have to have the argument about whether that use case satisfies the original "intended purpose" of traits. That's only reason I've ever been able to glean: that it (whatever "it" is) wasn't what the architects had in mind when they came up with the idea of traits. We're not even talking about anything that would require changes to the placement API. Just, "Oh, that's not a *capability* - shut it down." Bubble wrap was originally intended as a textured wallpaper and a greenhouse insulator. Can we accept the fact that traits have (many, many) uses beyond marking capabilities, and quit with the arbitrary restrictions? How far are we willing to go? Does an arbitrary (key: value) pair encoded in a trait name like key_`str(value)` (e.g. CURRENT_TEMPERATURE: 85 encoded as CUSTOM_TEMPERATURE_85) something we would be OK to see in placement? Great question. Perhaps TEMPERATURE_DANGEROUSLY_HIGH is okay, but TEMPERATURE_ is not. This thread isn't about setting these parameters; it's about getting us to a point where we can discuss a question just like this one without running up against: "That's a hard no, because you shouldn't encode key/value pairs in traits." "Oh, why's that?" "Because that's not what we intended when we created traits." "But it would work, and the alternatives are way harder." "-1" "But..." "-1" I think it's not so much about the intention when traits were created and more about what UX callers of the API are left with, if we were to recommend representing everything with traits and not providing another API for key-value use cases. We need to think about what the maintenance of their deployments will look like if traits are the only tool we provide. I get that we don't want to put artificial restrictions on how API callers can and can't use the traits API, but will they be left with a manageable experience if that's all that's available? I don't have time right now to come up with a really great example, but I'm thinking along the lines of, can this get out of hand (a la "flavor explosion") for an operator using traits to model what their compute hosts can do? Please forgive the oversimplified example I'm going to try to use to illustrate my concern: We all agree we can have traits for resource providers like: * HAS_SSD * HAS_GPU * HAS_WINDOWS But things get less straightforward when we think of traits like: * HAS_OWNER_CINDER * HAS_OWNER_NEUTRON * HAS_OWNER_CYBORG * HAS_RAID_0 * HAS_RAID_1 * HAS_RAID_5 * HAS_RAID_6 * HAS_RAID_10 * HAS_NUMA_CELL_0 * HAS_NUMA_CELL_1 * HAS_NUMA_CELL_2 * HAS_NUMA_CELL_3 I'm concerned about a lot of repetition here and maintenance headache for operators. That's where the thoughts about whether we should provide something like a key-value construct to API callers where they can instead say: * OWNER=CINDER * RAID=10 * NUMA_CELL=0 for each resource provider. If I'm off base with my example, please let me know. I'm not a placement expert. Anyway, I hope that gives an idea of what I'm thinking about in this discussion. I agree we need to pick a direction and go with it. I'm just trying to look out for the experience operators are going to be using this and maintaining it in their deployments. Cheers, -melanie __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [placement] The "intended purpose" of traits
On 09/28/2018 09:41 AM, Balázs Gibizer wrote: > > > On Fri, Sep 28, 2018 at 3:25 PM, Eric Fried wrote: >> It's time somebody said this. >> >> Every time we turn a corner or look under a rug, we find another use >> case for provider traits in placement. But every time we have to have >> the argument about whether that use case satisfies the original >> "intended purpose" of traits. >> >> That's only reason I've ever been able to glean: that it (whatever "it" >> is) wasn't what the architects had in mind when they came up with the >> idea of traits. We're not even talking about anything that would require >> changes to the placement API. Just, "Oh, that's not a *capability* - >> shut it down." >> >> Bubble wrap was originally intended as a textured wallpaper and a >> greenhouse insulator. Can we accept the fact that traits have (many, >> many) uses beyond marking capabilities, and quit with the arbitrary >> restrictions? > > How far are we willing to go? Does an arbitrary (key: value) pair > encoded in a trait name like key_`str(value)` (e.g. CURRENT_TEMPERATURE: > 85 encoded as CUSTOM_TEMPERATURE_85) something we would be OK to see in > placement? Great question. Perhaps TEMPERATURE_DANGEROUSLY_HIGH is okay, but TEMPERATURE_ is not. This thread isn't about setting these parameters; it's about getting us to a point where we can discuss a question just like this one without running up against: "That's a hard no, because you shouldn't encode key/value pairs in traits." "Oh, why's that?" "Because that's not what we intended when we created traits." "But it would work, and the alternatives are way harder." "-1" "But..." "-1" > > Cheers, > gibi > >> >> __ >> >> OpenStack Development Mailing List (not for usage questions) >> Unsubscribe: >> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > > > __ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [placement] The "intended purpose" of traits
On 09/28/2018 12:19 PM, Chris Dent wrote: > On Fri, 28 Sep 2018, Jay Pipes wrote: > >> On 09/28/2018 09:25 AM, Eric Fried wrote: >>> It's time somebody said this. > > Yes, a useful topic, I think. > >>> Every time we turn a corner or look under a rug, we find another use >>> case for provider traits in placement. But every time we have to have >>> the argument about whether that use case satisfies the original >>> "intended purpose" of traits. >>> >>> That's only reason I've ever been able to glean: that it (whatever "it" >>> is) wasn't what the architects had in mind when they came up with the >>> idea of traits. >> >> Don't pussyfoot around things. It's me you're talking about, Eric. You >> could just ask me instead of passive-aggressively posting to the list >> like this. > > It's not just you. Ed and I have also expressed some fairly strong > statement about how traits are "supposed" to be used and I would > guess that from Eric's perspective all three of us (amongst others) > have some form of architectural influence. Since it takes a village > and all that. Correct. I certainly wasn't talking about Jay specifically. I also wanted people other than placement cores/architects to participate in the discussion (thanks Julia and Zane). >> They aren't arbitrary. They are there for a reason: a trait is a >> boolean capability. It describes something that either a provider is >> capable of supporting or it isn't. > > This is somewhat (maybe even only slightly) different from what I > think the definition of a trait is, and that nuance may be relevant. > > I describe a trait as a "quality that a resource provider has" (the > car is blue). This contrasts with a resource class which is a > "quantity that a resource provider has" (the car has 4 doors). Yes, this. I don't want us to go off in the weeds about the reason or relevance of the choice of name, but "trait" is a superset of "capability" and easily encompasses "BLUE" or "PHYSNET_PUBLIC" or "OWNED_BY_NEUTRON" or "XYZ_BITSTREAM" or "PCI_ADDRESS_01_AB_23_CD" or "RAID5". > Our implementation is pretty much exactly that ^. We allow > clients to ask "give me things that have qualities x, y, z, not > qualities a, b, c, and quanities of G of 5 and H of 7". > > Add in aggregates and we have exactly what you say: > >> * Does the provider have *capacity* for the requested resources? >> * Does the provider have the required (or forbidden) *capabilities*? >> * Does the provider belong to some group? > > The nuance of difference is that your description of *capabilities* > seems more narrow than my description of *qualities* (aka > characteristics). You've got something fairly specific in mind, as a > way of constraining the profusion of noise that has happened with > how various kinds of information about resources of all sorts is > managed in OpenStack, as you describe in your message. > > I do not think it should be placement's job to control that noise. > It should be placement's job to provide a very strict contract about > what you can do with a trait: > > * create it, if necessary > * assign it to one or more resource providers > * ask for providers that either have it > * ... or do not have it > > That's all. Placement _code_ should _never_ be aware of the value of > a trait (except for the magical MISC_SHARES...). It should never > become possible to regex on traits or do comparisons > (required= >> If we want to add further constraints to the placement allocation >> candidates request that ask things like: >> >> * Does the provider have version 1.22.61821 of BIOS firmware from >> Marvell installed on it? > > That's a quality of the provider in a moment. > >> * Does the provider support an FPGA that has had an OVS program >> flashed to it in the last 20 days? > > If you squint, so is this. > >> * Does the provider belong to physical network "corpnet" and also >> support creation of virtual NICs of type either "DIRECT" or "NORMAL"? > > And these. > > But at least some of them are dynamic rather than some kind of > platonic ideal associated with the resource provider. > > I don't think placement should be concerned about temporal aspects > of traits. If we can't write a web service that can handle setting > lots of traits every second of every day, we should go home. If > clients of placement want to set weird traits, more power to them. > > However, if clients of placement (such as nova) which are being the > orchestrator of resource providers manipulated by multiple systems > (neutron, cinder, ironic, cyborg, etc) wish to set some constraints > on how and what traits can do and mean, then that is up to them. > > nova-scheduler is the thing that is doing `GET > /allocation_candidates` for those multiple system. It presumably > should have some say in what traits it is willing to express and > use. Right, this is where it's getting sticky. I feel like the push-back comes from people wearing their placement hats saying "you can't
Re: [openstack-dev] [placement] The "intended purpose" of traits
On Fri, 28 Sep 2018, Jay Pipes wrote:
On 09/28/2018 09:25 AM, Eric Fried wrote:
It's time somebody said this.
Yes, a useful topic, I think.
Every time we turn a corner or look under a rug, we find another use
case for provider traits in placement. But every time we have to have
the argument about whether that use case satisfies the original
"intended purpose" of traits.
That's only reason I've ever been able to glean: that it (whatever "it"
is) wasn't what the architects had in mind when they came up with the
idea of traits.
Don't pussyfoot around things. It's me you're talking about, Eric. You could
just ask me instead of passive-aggressively posting to the list like this.
It's not just you. Ed and I have also expressed some fairly strong
statement about how traits are "supposed" to be used and I would
guess that from Eric's perspective all three of us (amongst others)
have some form of architectural influence. Since it takes a village
and all that.
They aren't arbitrary. They are there for a reason: a trait is a boolean
capability. It describes something that either a provider is capable of
supporting or it isn't.
This is somewhat (maybe even only slightly) different from what I
think the definition of a trait is, and that nuance may be relevant.
I describe a trait as a "quality that a resource provider has" (the
car is blue). This contrasts with a resource class which is a
"quantity that a resource provider has" (the car has 4 doors).
Our implementation is pretty much exactly that ^. We allow
clients to ask "give me things that have qualities x, y, z, not
qualities a, b, c, and quanities of G of 5 and H of 7".
Add in aggregates and we have exactly what you say:
* Does the provider have *capacity* for the requested resources?
* Does the provider have the required (or forbidden) *capabilities*?
* Does the provider belong to some group?
The nuance of difference is that your description of *capabilities*
seems more narrow than my description of *qualities* (aka
characteristics). You've got something fairly specific in mind, as a
way of constraining the profusion of noise that has happened with
how various kinds of information about resources of all sorts is
managed in OpenStack, as you describe in your message.
I do not think it should be placement's job to control that noise.
It should be placement's job to provide a very strict contract about
what you can do with a trait:
* create it, if necessary
* assign it to one or more resource providers
* ask for providers that either have it
* ... or do not have it
That's all. Placement _code_ should _never_ be aware of the value of
a trait (except for the magical MISC_SHARES...). It should never
become possible to regex on traits or do comparisons
(required=If we want to add further constraints to the placement allocation candidates
request that ask things like:
* Does the provider have version 1.22.61821 of BIOS firmware from Marvell
installed on it?
That's a quality of the provider in a moment.
* Does the provider support an FPGA that has had an OVS program flashed to it
in the last 20 days?
If you squint, so is this.
* Does the provider belong to physical network "corpnet" and also support
creation of virtual NICs of type either "DIRECT" or "NORMAL"?
And these.
But at least some of them are dynamic rather than some kind of
platonic ideal associated with the resource provider.
I don't think placement should be concerned about temporal aspects
of traits. If we can't write a web service that can handle setting
lots of traits every second of every day, we should go home. If
clients of placement want to set weird traits, more power to them.
However, if clients of placement (such as nova) which are being the
orchestrator of resource providers manipulated by multiple systems
(neutron, cinder, ironic, cyborg, etc) wish to set some constraints
on how and what traits can do and mean, then that is up to them.
nova-scheduler is the thing that is doing `GET
/allocation_candidates` for those multiple system. It presumably
should have some say in what traits it is willing to express and
use.
But the placement service doesn't and shouldn't care.
Then we should add a data model that allow providers to be decorated with
key/value (or more complex than key/value) information where we can query for
those kinds of constraints without needing to encode all sorts of non-binary
bits of information into a capability string.
Let's never do this, please. The three capabilities (ha!) of
placement that you listed above ("Does the...") are very powerful as
is and have a conceptual integrity that's really quite awesome. I
think keeping it contained and constrained in very "simple" concepts
like that was stroke of genius you (Jay) made and I'd hope we can
keep it clean like that.
If we weren't a multiple-service oriented system, and instead had
some kind of k8s-like etcd-like
keeper-of-all-the-info-about-everything, then sure, having
Re: [openstack-dev] [placement] The "intended purpose" of traits
On 09/28/2018 09:25 AM, Eric Fried wrote: It's time somebody said this. Every time we turn a corner or look under a rug, we find another use case for provider traits in placement. But every time we have to have the argument about whether that use case satisfies the original "intended purpose" of traits. That's only reason I've ever been able to glean: that it (whatever "it" is) wasn't what the architects had in mind when they came up with the idea of traits. Don't pussyfoot around things. It's me you're talking about, Eric. You could just ask me instead of passive-aggressively posting to the list like this. We're not even talking about anything that would require changes to the placement API. Just, "Oh, that's not a *capability* - shut it down." That's precisely the attitude that got the Nova scheduler into the unmaintainable and convoluted mess that it is now: "well, who cares if a concept was originally intended to describe X, it's just *easier* for us to re-use this random piece of data in ways it wasn't intended because that way we don't have to change anything about our docs or our API". And *this* is the kind of stuff you end up with: https://github.com/openstack/nova/blob/99bf62e42701397690fe2b4987ce4fd7879355b8/nova/scheduler/filters/compute_capabilities_filter.py#L35-L107 Which is a pile of unreadable, unintelligible garbage; nobody knows how it works, how it originally was intended to work, or how to really clean it up. Bubble wrap was originally intended as a textured wallpaper and a greenhouse insulator. Can we accept the fact that traits have (many, many) uses beyond marking capabilities, and quit with the arbitrary restrictions? They aren't arbitrary. They are there for a reason: a trait is a boolean capability. It describes something that either a provider is capable of supporting or it isn't. Conceptually, having boolean traits/capabilities is important because it allows the user to reason simply about how a provider meets the requested constraints for scheduling. Currently, those constraints include the following: * Does the provider have *capacity* for the requested resources? * Does the provider have the required (or forbidden) *capabilities*? * Does the provider belong to some group? If we want to add further constraints to the placement allocation candidates request that ask things like: * Does the provider have version 1.22.61821 of BIOS firmware from Marvell installed on it? * Does the provider support an FPGA that has had an OVS program flashed to it in the last 20 days? * Does the provider belong to physical network "corpnet" and also support creation of virtual NICs of type either "DIRECT" or "NORMAL"? Then we should add a data model that allow providers to be decorated with key/value (or more complex than key/value) information where we can query for those kinds of constraints without needing to encode all sorts of non-binary bits of information into a capability string. Propose such a thing and I'll gladly support it. But I won't support bastardizing the simple concept of a boolean capability just because we don't want to change the API or database schema. -jay __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [placement] The "intended purpose" of traits
On 28/09/18 9:25 AM, Eric Fried wrote:
It's time somebody said this.
Every time we turn a corner or look under a rug, we find another use
case for provider traits in placement. But every time we have to have
the argument about whether that use case satisfies the original
"intended purpose" of traits.
That's only reason I've ever been able to glean: that it (whatever "it"
is) wasn't what the architects had in mind when they came up with the
idea of traits. We're not even talking about anything that would require
changes to the placement API. Just, "Oh, that's not a *capability* -
shut it down."
So I have no idea what traits or capabilities are (in this context), but
I have a bit of experience with running a busy project where everyone
wants to get their pet feature in, so I'd like to offer a couple of
observations if I may:
* Conceptual integrity *is* important.
* 'Everything we could think of before we had a chance to try it' is not
an especially compelling concept, and using it in place of one will tend
to result in a lot of repeated arguments.
Both extremes ('that's how we've always done it' vs. 'free-for-all') are
probably undesirable. I'd recommend trying to document traits in
conceptual, rather than historical, terms. What are they good at? What
are they not good at? Is there a limit to how many there can be while
still remaining manageable? Are there other potential concepts that
would map better to certain borderline use cases? That won't make the
arguments go away, but it should help make them easier to resolve.
cheers,
Zane.
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [placement] The "intended purpose" of traits
On Fri, Sep 28, 2018 at 3:25 PM, Eric Fried wrote: It's time somebody said this. Every time we turn a corner or look under a rug, we find another use case for provider traits in placement. But every time we have to have the argument about whether that use case satisfies the original "intended purpose" of traits. That's only reason I've ever been able to glean: that it (whatever "it" is) wasn't what the architects had in mind when they came up with the idea of traits. We're not even talking about anything that would require changes to the placement API. Just, "Oh, that's not a *capability* - shut it down." Bubble wrap was originally intended as a textured wallpaper and a greenhouse insulator. Can we accept the fact that traits have (many, many) uses beyond marking capabilities, and quit with the arbitrary restrictions? How far are we willing to go? Does an arbitrary (key: value) pair encoded in a trait name like key_`str(value)` (e.g. CURRENT_TEMPERATURE: 85 encoded as CUSTOM_TEMPERATURE_85) something we would be OK to see in placement? Cheers, gibi __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [placement] The "intended purpose" of traits
Eric, Very well said, I completely agree with you. We should not hold ourselves back based upon perceptions of original intended purpose. Things do change. We have to accept that. We must normalize this fact in our actions moving forward. That being said, I'm not entirely sure I'm personally fully aware of the arbitrary restrictions you speak of. Is there thread or a discussion out there that I can gain further context with? Thanks! -Julia On Fri, Sep 28, 2018 at 6:25 AM Eric Fried wrote: > > It's time somebody said this. > > Every time we turn a corner or look under a rug, we find another use > case for provider traits in placement. But every time we have to have > the argument about whether that use case satisfies the original > "intended purpose" of traits. > > That's only reason I've ever been able to glean: that it (whatever "it" > is) wasn't what the architects had in mind when they came up with the > idea of traits. We're not even talking about anything that would require > changes to the placement API. Just, "Oh, that's not a *capability* - > shut it down." > > Bubble wrap was originally intended as a textured wallpaper and a > greenhouse insulator. Can we accept the fact that traits have (many, > many) uses beyond marking capabilities, and quit with the arbitrary > restrictions? > > __ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [placement] The "intended purpose" of traits
It's time somebody said this. Every time we turn a corner or look under a rug, we find another use case for provider traits in placement. But every time we have to have the argument about whether that use case satisfies the original "intended purpose" of traits. That's only reason I've ever been able to glean: that it (whatever "it" is) wasn't what the architects had in mind when they came up with the idea of traits. We're not even talking about anything that would require changes to the placement API. Just, "Oh, that's not a *capability* - shut it down." Bubble wrap was originally intended as a textured wallpaper and a greenhouse insulator. Can we accept the fact that traits have (many, many) uses beyond marking capabilities, and quit with the arbitrary restrictions? __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
