Re: [openstack-dev] [nova] [scheduler] Use ResourceProviderTags instead of ResourceClass?
On Mon, 8 Aug 2016, Ed Leafe wrote: Me too. I think that this is one case where thinking in SQL messes you up. Sure, you can probably make it work by hacking in the concept of infinity into the code, but there will still be a conceptual disconnect. And later, when we inevitably need to enhance resource providers and their capabilities, we will end up creating another hack to work around what is an actual inventory and what is this new infinite inventory thing. /me shrugs This has been an interesting exercise because it does get people to express their ideas and their concerns a bit more. Which even if it doesn't change this round of stuff that we create helps for the next round. In my case my interest in exploring the model I've been describing is largely driven by wanting to _not_ think in SQL but in a way that is more simple math but with a very constrained grammar. Part of the goal here is to make "inevitable enhancement" which involves a different conceptual model something that is constrained to such an extent that it might be impossible and thus has to happen in a separate service; so that we can avoid monoliths This constrained grammar thing is really important. Oh, and think of the person coming in new to the placement engine, and having to explain what an infinite inventory is. Imagine their face. Weird, I wonder what experience I had that makes that so natural for me. I conceded a long time ago, but you guys keep saying things that make me want to keep talking about it. -- Chris Dent ┬─┬ノ( º _ ºノ) http://anticdent.org/ freenode: cdent tw: @anticdent__ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova] [scheduler] Use ResourceProviderTags instead of ResourceClass?
On Aug 8, 2016, at 9:42 AM, Jay Pipeswrote: >> When some ssd-ness is consumed, all of it (infinity) is still left >> over. >> >> For me it is easier: a resource provider has just one way of >> describing what it can do: classes of inventory (that provide >> gigabytes of disk that are ssd). When we add tags/capabilities >> we have another mode of description. > > I could not disagree more. :) Me too. I think that this is one case where thinking in SQL messes you up. Sure, you can probably make it work by hacking in the concept of infinity into the code, but there will still be a conceptual disconnect. And later, when we inevitably need to enhance resource providers and their capabilities, we will end up creating another hack to work around what is an actual inventory and what is this new infinite inventory thing. Oh, and think of the person coming in new to the placement engine, and having to explain what an infinite inventory is. Imagine their face. -- Ed Leafe __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova] [scheduler] Use ResourceProviderTags instead of ResourceClass?
On 08/08/2016 06:14 AM, Chris Dent wrote: On Mon, 8 Aug 2016, Alex Xu wrote: Chris, thanks for the blog to explain your idea! It helps me understand your idea better. Thanks for reading it. As I think I've mentioned a few times I'm not really trying to sell the idea, just make sure it is clear enough to be evaluated. I agree the goal for API interface design in your blog. But one point I guess you also agree, that is "The interface is easy to understand for API user". So look at the example of API request flow with gabbi, it is pretty clear for me even I didn't spend any time to learn the gabbi. That means: gabbi is cool and the interface is clear! But the only confuse is "total: ∞". And the related ResourceClass is "ssd", does it mean disk size is infinite? For a user, he is learning our API, he needs to search the document, due to he want to know "what is this special usage way means to". If user can understand our API without any document, so that is prefect. I think the main source of confusion is that where I find it pretty easy to think of qualitative characteristics as being an "infinity of a quantity" that's not an easy concept in general. In the "ssd" example what it means is not that disk size is infinite but taht the ssd-ness of the resource provider is infinite. So, for example a resource provider which provides disk space that is hosted on ssd has (at least) two resource classes: DISK_GB: SSD: When some ssd-ness is consumed, all of it (infinity) is still left over. For me it is easier: a resource provider has just one way of describing what it can do: classes of inventory (that provide gigabytes of disk that are ssd). When we add tags/capabilities we have another mode of description. I could not disagree more. :) You don't have an "inventory" of SSD. You have an inventory of DISK_GB. Whether the resource provider of that DISK_GB resource uses SSD or HDD disks is an *adjective* -- i.e. a capability -- that describes the provider. Saying capabilities are just resources with infinite inventory just makes the API more confusing IMHO. It's like saying "hey, give me two apples and an infinite amount of red." Just doesn't make sense. My two pesos, -jay __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova] [scheduler] Use ResourceProviderTags instead of ResourceClass?
On Mon, 8 Aug 2016, Alex Xu wrote: Chris, thanks for the blog to explain your idea! It helps me understand your idea better. Thanks for reading it. As I think I've mentioned a few times I'm not really trying to sell the idea, just make sure it is clear enough to be evaluated. I agree the goal for API interface design in your blog. But one point I guess you also agree, that is "The interface is easy to understand for API user". So look at the example of API request flow with gabbi, it is pretty clear for me even I didn't spend any time to learn the gabbi. That means: gabbi is cool and the interface is clear! But the only confuse is "total: ∞". And the related ResourceClass is "ssd", does it mean disk size is infinite? For a user, he is learning our API, he needs to search the document, due to he want to know "what is this special usage way means to". If user can understand our API without any document, so that is prefect. I think the main source of confusion is that where I find it pretty easy to think of qualitative characteristics as being an "infinity of a quantity" that's not an easy concept in general. In the "ssd" example what it means is not that disk size is infinite but taht the ssd-ness of the resource provider is infinite. So, for example a resource provider which provides disk space that is hosted on ssd has (at least) two resource classes: DISK_GB: SSD: When some ssd-ness is consumed, all of it (infinity) is still left over. For me it is easier: a resource provider has just one way of describing what it can do: classes of inventory (that provide gigabytes of disk that are ssd). When we add tags/capabilities we have another mode of description. /me yields -- Chris Dent ┬─┬ノ( º _ ºノ) http://anticdent.org/ freenode: cdent tw: @anticdent__ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova] [scheduler] Use ResourceProviderTags instead of ResourceClass?
Chris, thanks for the blog to explain your idea! It helps me understand your idea better. I agree the goal for API interface design in your blog. But one point I guess you also agree, that is "The interface is easy to understand for API user". So look at the example of API request flow with gabbi, it is pretty clear for me even I didn't spend any time to learn the gabbi. That means: gabbi is cool and the interface is clear! But the only confuse is "total: ∞". And the related ResourceClass is "ssd", does it mean disk size is infinite? For a user, he is learning our API, he needs to search the document, due to he want to know "what is this special usage way means to". If user can understand our API without any document, so that is prefect. I agree all of other point you said, limit resource, unified concept. If we want to finish that goal, I think the way is "Use ResourceProviderTags instead of ResourceClass", not "Use ResourceClass instead of ResourceClass" 2016-08-05 21:16 GMT+08:00 Chris Dent: > On Tue, 2 Aug 2016, Alex Xu wrote: > > Chris have a thought about using ResourceClass to describe Capabilities >> with an infinite inventory. In the beginning we brain storming the idea of >> Tags, Tan Lin have same thought, but we say no very quickly, due to the >> ResourceClass is really about Quantitative stuff. But Chris give very good >> point about simplify the ResourceProvider model and the API. >> > > I'm still leaning in this direction. I realized I wasn't explaining > myself very well and "because I like it" isn't really a good enough > for doing anything, so I wrote something up about it: > >https://anticdent.org/simple-resource-provision.html > > -- > Chris Dent ┬─┬ノ( º _ ºノ) http://anticdent.org/ > freenode: cdent tw: @anticdent > __ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > > __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova] [scheduler] Use ResourceProviderTags instead of ResourceClass?
2016-08-05 21:16 GMT+08:00 Chris Dent: > On Tue, 2 Aug 2016, Alex Xu wrote: > > Chris have a thought about using ResourceClass to describe Capabilities >> with an infinite inventory. In the beginning we brain storming the idea of >> Tags, Tan Lin have same thought, but we say no very quickly, due to the >> ResourceClass is really about Quantitative stuff. But Chris give very good >> point about simplify the ResourceProvider model and the API. >> > > I'm still leaning in this direction. I realized I wasn't explaining > myself very well and "because I like it" isn't really a good enough > for doing anything, so I wrote something up about it: > >https://anticdent.org/simple-resource-provision.html > > Reusing the existing infrastructure of resource classes, inventories and allocations does make implementation easier with capabilities as well as their calculations and representations, at least at the beginning. But I'm still not convinced by this direction, because it introduces unnecessary reuses as well as overhead for capabilities. Instead of attaching a capability directly to a resource provider, it needs to create an inventory and assign the capability to inventory, indirectly. Moreover, it reuses allocations and even the "compare-and-swap" strategy with the implementation of "generation" field in the resource provider. And it introduces further complexities and obscurities if we decide to disable the unnecessary consumable features for capabilities. The existing architecture of resource provider is mainly for consumable resources. And we don't want capabilities to be consumable by mistake. It is an inherently different implementation for non consumable capabilities, so I tend to agree to implement qualitative part of resource provider from a fresher start to keep it simple and direct. And add features incrementally if they are thought necessary. --- Regards Yingxin __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova] [scheduler] Use ResourceProviderTags instead of ResourceClass?
On Tue, 2 Aug 2016, Alex Xu wrote: Chris have a thought about using ResourceClass to describe Capabilities with an infinite inventory. In the beginning we brain storming the idea of Tags, Tan Lin have same thought, but we say no very quickly, due to the ResourceClass is really about Quantitative stuff. But Chris give very good point about simplify the ResourceProvider model and the API. I'm still leaning in this direction. I realized I wasn't explaining myself very well and "because I like it" isn't really a good enough for doing anything, so I wrote something up about it: https://anticdent.org/simple-resource-provision.html -- Chris Dent ┬─┬ノ( º _ ºノ) http://anticdent.org/ freenode: cdent tw: @anticdent__ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova] [scheduler] Use ResourceProviderTags instead of ResourceClass?
2016-08-03 2:12 GMT+08:00 Jay Pipes: > On 08/02/2016 08:19 AM, Alex Xu wrote: > >> Chris have a thought about using ResourceClass to describe Capabilities >> with an infinite inventory. In the beginning we brain storming the idea >> of Tags, Tan Lin have same thought, but we say no very quickly, due to >> the ResourceClass is really about Quantitative stuff. But Chris give >> very good point about simplify the ResourceProvider model and the API. >> >> After rethinking about those idea, I like simplify the ResourceProvider >> model and the API. But I think the direction is opposite. ResourceClass >> with infinite inventory is really hacky. The Placement API is simple, >> but the usage of those API isn't simple for user, they need create a >> ResourceClass, then create an infinite inventory. And ResourceClass >> isn't managable like Tags, look at the Tags API, there are many query >> parameter. >> >> But look at the ResourceClass and ResourceProviderTags, they are totally >> same, two columns: one is integer id, another one is string. >> ResourceClass is just for naming the quantitative stuff. So what we need >> is thing used to 'naming'. ResourceProviderTags is higher abstract, Tags >> is generic thing to name something, we totally can use Tag instead of >> ResourceClass. So user can create inventory with tags, also user can >> create ResourceProvider with tags. >> > > No, this sounds like actually way more complexity than is needed and will > make the schema less explicit. No, it simplify the ResourceProvider model and the API? maybe the complexity you pointed at other place. Yes, it make the schema less explicit. Using higher layer abstraction, it will lose some characteristic. That is thing we have to pay. Anyway let me put this in the alternative section... > > > But yes, there may still have problem isn't resolved, one of problem is >> pointed out when I discuss with YingXin about how to distinguish the Tag >> is about quantitative or qualitative. He think we need attribute for Tag >> to distinguish it. But the attribute isn't thing I like, I prefer leave >> that alone due to the user of placement API is admin-user. >> >> Any thought? or I'm too crazy at here...maybe I just need put this in >> the alternative section in the spec... >> > > A resource class is not a capability, though. It's an indication of a type > of quantitative consumable that is exposed on a resource provider. > > A capability is a string that indicates a feature that a resource provider > offers. A capability isn't "consumed". > Agree about the definition of resource class and capability. I think it is pretty clear for us. What I want to say is the Placement Engine really don't know what is ResourceClass and Capability. They just need an indication for something. You can think about ResourceClass and Capability is sub-class, the Tag is the base-class for them. And think about a case, user can input 'cookie' as the name of ResourceClass, but Placement Engine won't say no to user. Because Placement Engine really don't care about the meaning of ResourceClass' name. Placement Engine just needs a 'tag' to distinguish the ResourceClass and Capability. > BTW, this is why I continue to think that using the term "tags" in the > placement API is wrong. The placement API should clearly indicate that a > resource provider has a set of capabilities. Tags, in Nova at least, are > end-user-defined simple categorization strings that have no standardization > and no cataloguing or collation to them. > Yes, but we don't have standard string for all the capabilities. For shared storage, this is setup by deployer, not the OpenStack, so the capabilities of shared storage won't be defined by OpenStack, it is defined by deployer. > > Capabilities are not end-user-defined -- they can be defined by an > operator but they are not things that a normal end-user can simply create. > And capabilities are specifically *not* for categorization purposes. They > are an indication of a set of features that a resource provider exposes. > I totally see your point. But there is one question I can't get answer. If we call Capabilities instead of tags, but user still can free to input any string. User can input 'fish' as a capabilities, and placement API won't say no. Is this OK? Why this is OK when user input non-capability into a capability field? Actually same question for ResourceClass. (ah, this is back to ResourceClass and Capability have same base-class Tags) I think this is only question I can't pass through, otherwise I will update the spec :) > > This is why I think the placement API for capabilities should use the term > "capabilities" and not "tags". > > Best, > -jay > > __ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe >
Re: [openstack-dev] [nova] [scheduler] Use ResourceProviderTags instead of ResourceClass?
On 08/02/2016 08:19 AM, Alex Xu wrote: Chris have a thought about using ResourceClass to describe Capabilities with an infinite inventory. In the beginning we brain storming the idea of Tags, Tan Lin have same thought, but we say no very quickly, due to the ResourceClass is really about Quantitative stuff. But Chris give very good point about simplify the ResourceProvider model and the API. After rethinking about those idea, I like simplify the ResourceProvider model and the API. But I think the direction is opposite. ResourceClass with infinite inventory is really hacky. The Placement API is simple, but the usage of those API isn't simple for user, they need create a ResourceClass, then create an infinite inventory. And ResourceClass isn't managable like Tags, look at the Tags API, there are many query parameter. But look at the ResourceClass and ResourceProviderTags, they are totally same, two columns: one is integer id, another one is string. ResourceClass is just for naming the quantitative stuff. So what we need is thing used to 'naming'. ResourceProviderTags is higher abstract, Tags is generic thing to name something, we totally can use Tag instead of ResourceClass. So user can create inventory with tags, also user can create ResourceProvider with tags. No, this sounds like actually way more complexity than is needed and will make the schema less explicit. But yes, there may still have problem isn't resolved, one of problem is pointed out when I discuss with YingXin about how to distinguish the Tag is about quantitative or qualitative. He think we need attribute for Tag to distinguish it. But the attribute isn't thing I like, I prefer leave that alone due to the user of placement API is admin-user. Any thought? or I'm too crazy at here...maybe I just need put this in the alternative section in the spec... A resource class is not a capability, though. It's an indication of a type of quantitative consumable that is exposed on a resource provider. A capability is a string that indicates a feature that a resource provider offers. A capability isn't "consumed". BTW, this is why I continue to think that using the term "tags" in the placement API is wrong. The placement API should clearly indicate that a resource provider has a set of capabilities. Tags, in Nova at least, are end-user-defined simple categorization strings that have no standardization and no cataloguing or collation to them. Capabilities are not end-user-defined -- they can be defined by an operator but they are not things that a normal end-user can simply create. And capabilities are specifically *not* for categorization purposes. They are an indication of a set of features that a resource provider exposes. This is why I think the placement API for capabilities should use the term "capabilities" and not "tags". Best, -jay __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [nova] [scheduler] Use ResourceProviderTags instead of ResourceClass?
Chris have a thought about using ResourceClass to describe Capabilities with an infinite inventory. In the beginning we brain storming the idea of Tags, Tan Lin have same thought, but we say no very quickly, due to the ResourceClass is really about Quantitative stuff. But Chris give very good point about simplify the ResourceProvider model and the API. After rethinking about those idea, I like simplify the ResourceProvider model and the API. But I think the direction is opposite. ResourceClass with infinite inventory is really hacky. The Placement API is simple, but the usage of those API isn't simple for user, they need create a ResourceClass, then create an infinite inventory. And ResourceClass isn't managable like Tags, look at the Tags API, there are many query parameter. But look at the ResourceClass and ResourceProviderTags, they are totally same, two columns: one is integer id, another one is string. ResourceClass is just for naming the quantitative stuff. So what we need is thing used to 'naming'. ResourceProviderTags is higher abstract, Tags is generic thing to name something, we totally can use Tag instead of ResourceClass. So user can create inventory with tags, also user can create ResourceProvider with tags. But yes, there may still have problem isn't resolved, one of problem is pointed out when I discuss with YingXin about how to distinguish the Tag is about quantitative or qualitative. He think we need attribute for Tag to distinguish it. But the attribute isn't thing I like, I prefer leave that alone due to the user of placement API is admin-user. Any thought? or I'm too crazy at here...maybe I just need put this in the alternative section in the spec... Thanks Alex __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev