Re: [openstack-dev] [nova] [scheduler] Use ResourceProviderTags instead of ResourceClass?

[Chris Dent]

On Mon, 8 Aug 2016, Ed Leafe wrote:


Me too. I think that this is one case where thinking in SQL messes you
up. Sure, you can probably make it work by hacking in the concept of
infinity into the code, but there will still be a conceptual
disconnect. And later, when we inevitably need to enhance resource
providers and their capabilities, we will end up creating another hack
to work around what is an actual inventory and what is this new
infinite inventory thing.


/me shrugs

This has been an interesting exercise because it does get people to
express their ideas and their concerns a bit more. Which even if it
doesn't change this round of stuff that we create helps for the next
round.

In my case my interest in exploring the model I've been describing is
largely driven by wanting to _not_ think in SQL but in a way that is
more simple math but with a very constrained grammar.

Part of the goal here is to make "inevitable enhancement" which
involves a different conceptual model something that is constrained
to such an extent that it might be impossible and thus has to happen
in a separate service; so that we can avoid monoliths

This constrained grammar thing is really important.


Oh, and think of the person coming in new to the placement engine, and
having to explain what an infinite inventory is. Imagine their face.


Weird, I wonder what experience I had that makes that so natural for me.

I conceded a long time ago, but you guys keep saying things that
make me want to keep talking about it.

--
Chris Dent   ┬─┬ノ( º _ ºノ) http://anticdent.org/
freenode: cdent tw: @anticdent__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [nova] [scheduler] Use ResourceProviderTags instead of ResourceClass?

[Ed Leafe]

On Aug 8, 2016, at 9:42 AM, Jay Pipes  wrote:

>> When some ssd-ness is consumed, all of it (infinity) is still left
>> over.
>> 
>> For me it is easier: a resource provider has just one way of
>> describing what it can do: classes of inventory (that provide
>> gigabytes of disk that are ssd). When we add tags/capabilities
>> we have another mode of description.
> 
> I could not disagree more. :)

Me too. I think that this is one case where thinking in SQL messes you up. 
Sure, you can probably make it work by hacking in the concept of infinity into 
the code, but there will still be a conceptual disconnect. And later, when we 
inevitably need to enhance resource providers and their capabilities, we will 
end up creating another hack to work around what is an actual inventory and 
what is this new infinite inventory thing.

Oh, and think of the person coming in new to the placement engine, and having 
to explain what an infinite inventory is. Imagine their face.


-- Ed Leafe






__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [nova] [scheduler] Use ResourceProviderTags instead of ResourceClass?

[Jay Pipes]

On 08/08/2016 06:14 AM, Chris Dent wrote:

On Mon, 8 Aug 2016, Alex Xu wrote:


Chris, thanks for the blog to explain your idea! It helps me understand
your idea better.


Thanks for reading it. As I think I've mentioned a few times I'm not
really trying to sell the idea, just make sure it is clear enough to
be evaluated.


I agree the goal for API interface design in your blog. But one point I
guess you also agree, that is "The interface is easy to understand for
API
user". So look at the example of API request flow with gabbi,  it is
pretty
clear for me even I didn't spend any time to learn the gabbi. That means:
gabbi is cool and the interface is clear! But the only confuse is "total:
∞". And the related ResourceClass is "ssd", does it mean disk size is
infinite? For a user, he is learning our API, he needs to search the
document, due to he want to know "what is this special usage way means
to".
If user can understand our API without any document, so that is prefect.


I think the main source of confusion is that where I find it pretty
easy to think of qualitative characteristics as being an "infinity
of a quantity" that's not an easy concept in general.

In the "ssd" example what it means is not that disk size is infinite
but taht the ssd-ness of the resource provider is infinite. So, for
example a resource provider which provides disk space that is hosted
on ssd has (at least) two resource classes:

   DISK_GB: 
   SSD: 

When some ssd-ness is consumed, all of it (infinity) is still left
over.

For me it is easier: a resource provider has just one way of
describing what it can do: classes of inventory (that provide
gigabytes of disk that are ssd). When we add tags/capabilities
we have another mode of description.


I could not disagree more. :)

You don't have an "inventory" of SSD. You have an inventory of DISK_GB. 
Whether the resource provider of that DISK_GB resource uses SSD or HDD 
disks is an *adjective* -- i.e. a capability -- that describes the provider.


Saying capabilities are just resources with infinite inventory just 
makes the API more confusing IMHO.


It's like saying "hey, give me two apples and an infinite amount of 
red." Just doesn't make sense.


My two pesos,
-jay

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [nova] [scheduler] Use ResourceProviderTags instead of ResourceClass?

[Chris Dent]

On Mon, 8 Aug 2016, Alex Xu wrote:


Chris, thanks for the blog to explain your idea! It helps me understand
your idea better.


Thanks for reading it. As I think I've mentioned a few times I'm not
really trying to sell the idea, just make sure it is clear enough to
be evaluated.


I agree the goal for API interface design in your blog. But one point I
guess you also agree, that is "The interface is easy to understand for API
user". So look at the example of API request flow with gabbi,  it is pretty
clear for me even I didn't spend any time to learn the gabbi. That means:
gabbi is cool and the interface is clear! But the only confuse is "total:
∞". And the related ResourceClass is "ssd", does it mean disk size is
infinite? For a user, he is learning our API, he needs to search the
document, due to he want to know "what is this special usage way means to".
If user can understand our API without any document, so that is prefect.


I think the main source of confusion is that where I find it pretty
easy to think of qualitative characteristics as being an "infinity
of a quantity" that's not an easy concept in general.

In the "ssd" example what it means is not that disk size is infinite
but taht the ssd-ness of the resource provider is infinite. So, for
example a resource provider which provides disk space that is hosted
on ssd has (at least) two resource classes:

   DISK_GB: 
   SSD: 

When some ssd-ness is consumed, all of it (infinity) is still left
over.

For me it is easier: a resource provider has just one way of
describing what it can do: classes of inventory (that provide
gigabytes of disk that are ssd). When we add tags/capabilities
we have another mode of description.

/me yields

--
Chris Dent   ┬─┬ノ( º _ ºノ) http://anticdent.org/
freenode: cdent tw: @anticdent__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [nova] [scheduler] Use ResourceProviderTags instead of ResourceClass?

[Alex Xu]

Chris, thanks for the blog to explain your idea! It helps me understand
your idea better.

I agree the goal for API interface design in your blog. But one point I
guess you also agree, that is "The interface is easy to understand for API
user". So look at the example of API request flow with gabbi,  it is pretty
clear for me even I didn't spend any time to learn the gabbi. That means:
gabbi is cool and the interface is clear! But the only confuse is "total:
∞". And the related ResourceClass is "ssd", does it mean disk size is
infinite? For a user, he is learning our API, he needs to search the
document, due to he want to know "what is this special usage way means to".
If user can understand our API without any document, so that is prefect.

I agree all of other point you said, limit resource, unified concept. If we
want to finish that goal, I think the way is "Use ResourceProviderTags
instead of ResourceClass", not "Use ResourceClass instead of ResourceClass"

2016-08-05 21:16 GMT+08:00 Chris Dent :

> On Tue, 2 Aug 2016, Alex Xu wrote:
>
> Chris have a thought about using ResourceClass to describe Capabilities
>> with an infinite inventory. In the beginning we brain storming the idea of
>> Tags, Tan Lin have same thought, but we say no very quickly, due to the
>> ResourceClass is really about Quantitative stuff. But Chris give very good
>> point about simplify the ResourceProvider model and the API.
>>
>
> I'm still leaning in this direction. I realized I wasn't explaining
> myself very well and "because I like it" isn't really a good enough
> for doing anything, so I wrote something up about it:
>
>https://anticdent.org/simple-resource-provision.html
>
> --
> Chris Dent   ┬─┬ノ( º _ ºノ) http://anticdent.org/
> freenode: cdent tw: @anticdent
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [nova] [scheduler] Use ResourceProviderTags instead of ResourceClass?

[Yingxin Cheng]

2016-08-05 21:16 GMT+08:00 Chris Dent :

> On Tue, 2 Aug 2016, Alex Xu wrote:
>
> Chris have a thought about using ResourceClass to describe Capabilities
>> with an infinite inventory. In the beginning we brain storming the idea of
>> Tags, Tan Lin have same thought, but we say no very quickly, due to the
>> ResourceClass is really about Quantitative stuff. But Chris give very good
>> point about simplify the ResourceProvider model and the API.
>>
>
> I'm still leaning in this direction. I realized I wasn't explaining
> myself very well and "because I like it" isn't really a good enough
> for doing anything, so I wrote something up about it:
>
>https://anticdent.org/simple-resource-provision.html
>
>
Reusing the existing infrastructure of resource classes, inventories and
allocations does make implementation easier with capabilities as well as
their calculations and representations, at least at the beginning.

But I'm still not convinced by this direction, because it introduces
unnecessary reuses as well as overhead for capabilities. Instead of
attaching a capability directly to a resource provider, it needs to create
an inventory and assign the capability to inventory, indirectly. Moreover,
it reuses allocations and even the "compare-and-swap" strategy with the
implementation of "generation" field in the resource provider. And it
introduces further complexities and obscurities if we decide to disable the
unnecessary consumable features for capabilities.

The existing architecture of resource provider is mainly for consumable
resources. And we don't want capabilities to be consumable by mistake. It
is an inherently different implementation for non consumable capabilities,
so I tend to agree to implement qualitative part of resource provider from
a fresher start to keep it simple and direct. And add features
incrementally if they are thought necessary.


---
Regards
Yingxin
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [nova] [scheduler] Use ResourceProviderTags instead of ResourceClass?

[Chris Dent]

On Tue, 2 Aug 2016, Alex Xu wrote:


Chris have a thought about using ResourceClass to describe Capabilities
with an infinite inventory. In the beginning we brain storming the idea of
Tags, Tan Lin have same thought, but we say no very quickly, due to the
ResourceClass is really about Quantitative stuff. But Chris give very good
point about simplify the ResourceProvider model and the API.


I'm still leaning in this direction. I realized I wasn't explaining
myself very well and "because I like it" isn't really a good enough
for doing anything, so I wrote something up about it:

   https://anticdent.org/simple-resource-provision.html

--
Chris Dent   ┬─┬ノ( º _ ºノ) http://anticdent.org/
freenode: cdent tw: @anticdent__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [nova] [scheduler] Use ResourceProviderTags instead of ResourceClass?

[Alex Xu]

2016-08-03 2:12 GMT+08:00 Jay Pipes :

> On 08/02/2016 08:19 AM, Alex Xu wrote:
>
>> Chris have a thought about using ResourceClass to describe Capabilities
>> with an infinite inventory. In the beginning we brain storming the idea
>> of Tags, Tan Lin have same thought, but we say no very quickly, due to
>> the ResourceClass is really about Quantitative stuff. But Chris give
>> very good point about simplify the ResourceProvider model and the API.
>>
>> After rethinking about those idea, I like simplify the ResourceProvider
>> model and the API. But I think the direction is opposite. ResourceClass
>> with infinite inventory is really hacky. The Placement API is simple,
>> but the usage of those API isn't simple for user, they need create a
>> ResourceClass, then create an infinite inventory. And ResourceClass
>> isn't managable like Tags, look at the Tags API, there are many query
>> parameter.
>>
>> But look at the ResourceClass and ResourceProviderTags, they are totally
>> same, two columns: one is integer id, another one is string.
>> ResourceClass is just for naming the quantitative stuff. So what we need
>> is thing used to 'naming'. ResourceProviderTags is higher abstract, Tags
>> is generic thing to name something, we totally can use Tag instead of
>> ResourceClass. So user can create inventory with tags, also user can
>> create ResourceProvider with tags.
>>
>
> No, this sounds like actually way more complexity than is needed and will
> make the schema less explicit.


No, it simplify the ResourceProvider model and the API? maybe the
complexity you pointed at other place.

Yes, it make the schema less explicit. Using higher layer abstraction, it
will lose some characteristic. That is thing we have to pay.

Anyway let me put this in the alternative section...


>
>
> But yes, there may still have problem isn't resolved, one of problem is
>> pointed out when I discuss with YingXin about how to distinguish the Tag
>> is about quantitative or qualitative. He think we need attribute for Tag
>> to distinguish it. But the attribute isn't thing I like, I prefer leave
>> that alone due to the user of placement API is admin-user.
>>
>> Any thought? or I'm too crazy at here...maybe I just need put this in
>> the alternative section in the spec...
>>
>
> A resource class is not a capability, though. It's an indication of a type
> of quantitative consumable that is exposed on a resource provider.
>
> A capability is a string that indicates a feature that a resource provider
> offers. A capability isn't "consumed".
>

Agree about the definition of resource class and capability. I think it is
pretty clear for us.

What I want to say is the Placement Engine really don't know what is
ResourceClass and Capability. They just need an indication for something.
You can think about ResourceClass and Capability is sub-class, the Tag is
the base-class for them. And think about a case, user can input 'cookie' as
the name of ResourceClass, but Placement Engine won't say no to user.
Because Placement Engine really don't care about the meaning of
ResourceClass' name. Placement Engine just needs a 'tag' to distinguish the
ResourceClass and Capability.


> BTW, this is why I continue to think that using the term "tags" in the
> placement API is wrong. The placement API should clearly indicate that a
> resource provider has a set of capabilities. Tags, in Nova at least, are
> end-user-defined simple categorization strings that have no standardization
> and no cataloguing or collation to them.
>

Yes, but we don't have standard string for all the capabilities. For shared
storage, this is setup by deployer, not the OpenStack, so the capabilities
of shared storage won't be defined by OpenStack, it is defined by deployer.


>
> Capabilities are not end-user-defined -- they can be defined by an
> operator but they are not things that a normal end-user can simply create.
> And capabilities are specifically *not* for categorization purposes. They
> are an indication of a set of features that a resource provider exposes.
>

I totally see your point. But there is one question I can't get answer. If
we call Capabilities instead of tags, but user still can free to input any
string. User can input 'fish' as a capabilities, and placement API won't
say no. Is this OK? Why this is OK when user input non-capability into a
capability field? Actually same question for ResourceClass. (ah, this is
back to ResourceClass and Capability have same base-class Tags)

I think this is only question I can't pass through, otherwise I will update
the spec :)


>
> This is why I think the placement API for capabilities should use the term
> "capabilities" and not "tags".
>
> Best,
> -jay
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> 

Re: [openstack-dev] [nova] [scheduler] Use ResourceProviderTags instead of ResourceClass?

[Jay Pipes]

On 08/02/2016 08:19 AM, Alex Xu wrote:

Chris have a thought about using ResourceClass to describe Capabilities
with an infinite inventory. In the beginning we brain storming the idea
of Tags, Tan Lin have same thought, but we say no very quickly, due to
the ResourceClass is really about Quantitative stuff. But Chris give
very good point about simplify the ResourceProvider model and the API.

After rethinking about those idea, I like simplify the ResourceProvider
model and the API. But I think the direction is opposite. ResourceClass
with infinite inventory is really hacky. The Placement API is simple,
but the usage of those API isn't simple for user, they need create a
ResourceClass, then create an infinite inventory. And ResourceClass
isn't managable like Tags, look at the Tags API, there are many query
parameter.

But look at the ResourceClass and ResourceProviderTags, they are totally
same, two columns: one is integer id, another one is string.
ResourceClass is just for naming the quantitative stuff. So what we need
is thing used to 'naming'. ResourceProviderTags is higher abstract, Tags
is generic thing to name something, we totally can use Tag instead of
ResourceClass. So user can create inventory with tags, also user can
create ResourceProvider with tags.


No, this sounds like actually way more complexity than is needed and 
will make the schema less explicit.



But yes, there may still have problem isn't resolved, one of problem is
pointed out when I discuss with YingXin about how to distinguish the Tag
is about quantitative or qualitative. He think we need attribute for Tag
to distinguish it. But the attribute isn't thing I like, I prefer leave
that alone due to the user of placement API is admin-user.

Any thought? or I'm too crazy at here...maybe I just need put this in
the alternative section in the spec...


A resource class is not a capability, though. It's an indication of a 
type of quantitative consumable that is exposed on a resource provider.


A capability is a string that indicates a feature that a resource 
provider offers. A capability isn't "consumed".


BTW, this is why I continue to think that using the term "tags" in the 
placement API is wrong. The placement API should clearly indicate that a 
resource provider has a set of capabilities. Tags, in Nova at least, are 
end-user-defined simple categorization strings that have no 
standardization and no cataloguing or collation to them.


Capabilities are not end-user-defined -- they can be defined by an 
operator but they are not things that a normal end-user can simply 
create. And capabilities are specifically *not* for categorization 
purposes. They are an indication of a set of features that a resource 
provider exposes.


This is why I think the placement API for capabilities should use the 
term "capabilities" and not "tags".


Best,
-jay

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [nova] [scheduler] Use ResourceProviderTags instead of ResourceClass?

[Alex Xu]

Chris have a thought about using ResourceClass to describe Capabilities
with an infinite inventory. In the beginning we brain storming the idea of
Tags, Tan Lin have same thought, but we say no very quickly, due to the
ResourceClass is really about Quantitative stuff. But Chris give very good
point about simplify the ResourceProvider model and the API.

After rethinking about those idea, I like simplify the ResourceProvider
model and the API. But I think the direction is opposite. ResourceClass
with infinite inventory is really hacky. The Placement API is simple, but
the usage of those API isn't simple for user, they need create a
ResourceClass, then create an infinite inventory. And ResourceClass isn't
managable like Tags, look at the Tags API, there are many query parameter.

But look at the ResourceClass and ResourceProviderTags, they are totally
same, two columns: one is integer id, another one is string. ResourceClass
is just for naming the quantitative stuff. So what we need is thing used to
'naming'. ResourceProviderTags is higher abstract, Tags is generic thing to
name something, we totally can use Tag instead of ResourceClass. So user
can create inventory with tags, also user can create ResourceProvider with
tags.

But yes, there may still have problem isn't resolved, one of problem is
pointed out when I discuss with YingXin about how to distinguish the Tag is
about quantitative or qualitative. He think we need attribute for Tag to
distinguish it. But the attribute isn't thing I like, I prefer leave that
alone due to the user of placement API is admin-user.

Any thought? or I'm too crazy at here...maybe I just need put this in the
alternative section in the spec...

Thanks
Alex
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev