Re: [openstack-dev] Dynamically adding Extra Specs

[Joshua Harlow]

Dhvanan Shah wrote:

Hey Jay!

I was looking at implementing a few scheduling algorithms of my own
natively into OpenStack, and for that I went through the nova-scheduler.
After going through the scheduler, I felt that it was not very easy to
implement or extend and add new scheduling algorithms to the scheduler.
The only things that I felt that I could change or maybe was provisioned
for adding or extending were the filters and weighers and implementing
new scheduling algorithms with just these 2 knobs was a little hard. I
did change the code in the filter_scheduler to get some basic algorithms
running like the first and next fit apart from the spreading and
stacking which was already present. But to go beyond and to implement
more complex algorithms was much harder and I would have to change a lot
of code in different places that could as a side effect also break
things and didn't seem clean. I might be wrong and might have not
understood things right, please correct me if so.

To give an example of what I mean by a little complex scheduling
algorithms: a subset matching algorithm - that schedules multiple
heterogeneous requests by picking out a subset from the requests that
best fit a host/s, so this would improve the utilization. The
prerequisite for this is that I have multiple heterogeneous requests
lined up to be scheduled. So for this kind of an algorithm it isnt easy
to implement into OpenStack.

So a workaround that I'm working on for implementing different
scheduling algorithms is by building a scheduling wrapper outside of the
OpenStack architecture, where the user interacts with this wrapper and
in the wrapper I get the host details from the database and based on the
algorithm I want, the scheduler chooses the host for the request and
gives out a VM : Host mapping (The wrapper does the sanity checks that
the filters do to check if the host can accommodate or handle the
request). Along with the request, I also want to pass this mapping that
the scheduler can use to assign the request to the host passed in the
mapping. I've written a filter that filters all the hosts apart from the
host that I sent and this is how I make sure that the request gets
placed on the host that I had passed. I have come up with a hack to pass
the host to the scheduler, but it is not quite elegant.


Why use the filter mechanism at all?

Just provide a whole new scheduler that replaces the plugabble point 
that already exists (a class with ``def select_destinations(self, 
context, spec_obj)``)?


Although it seems u want to put a request into SCHEDULING_WAIT (or 
something like that) and then when u have enough requests to batch up u 
will move from SCHEDULING_WAIT -> SCHEDULING (something akin to delayed 
scheduling when you have reached your batch size). Is something like 
that correct?




Would be great to have your input on the same!

On Mon, Feb 8, 2016 at 12:51 AM, Jay Pipes > wrote:

Apologies for the delayed responses. Comments inline.

On 01/27/2016 02:29 AM, Dhvanan Shah wrote:

Hey Jay!

Thanks for the clarification. There was another thing that I
wanted to
know, is there any provision to pass extra arguments or some extra
specifications along with the VM request to nova. To give you some
context, I wanted to pass a host:vm mapping to the nova
scheduler for
its host selection process, and I'm providing this mapping from
outside
of the openstack architecture.


Why do you want to do this? The scheduler is the thing that sets the
host -> vm mapping -- that's what the process of scheduling does.

>  So I need to send this information along

with the request to the scheduler. One way of doing this was
creating
new flavors with their extra specification as different hosts,
but that
would lead to as you pointed out earlier a "flavor explosion"
problem.

So is there a way to pass some extra arguments or some additional
information to nova.


Depends what exactly you are trying to pass to Nova. Could you give
some more information about your use case?

Thanks!
-jay




--
Dhvanan Shah

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] Dynamically adding Extra Specs

[Tripp, Travis S]

Dhvanan,

I admittedly only just saw the subject and skimmed the thread (so might be 
missing something), but have you looked into taking advantage of scheduler 
hints from a custom filter from the API / CLI?  As FYI, horizon has a patch up 
adding support for them [1].

[1] https://review.openstack.org/#/c/272635/

Travis

From: Dhvanan Shah <dhva...@gmail.com<mailto:dhva...@gmail.com>>
Reply-To: OpenStack List 
<openstack-dev@lists.openstack.org<mailto:openstack-dev@lists.openstack.org>>
Date: Sunday, February 7, 2016 at 8:34 PM
To: "Jay com>" <jaypi...@gmail.com<mailto:jaypi...@gmail.com>>, OpenStack List 
<openstack-dev@lists.openstack.org<mailto:openstack-dev@lists.openstack.org>>
Subject: Re: [openstack-dev] Dynamically adding Extra Specs

Hey Jay!

I was looking at implementing a few scheduling algorithms of my own natively 
into OpenStack, and for that I went through the nova-scheduler. After going 
through the scheduler, I felt that it was not very easy to implement or extend 
and add new scheduling algorithms to the scheduler. The only things that I felt 
that I could change or maybe was provisioned for adding or extending were the 
filters and weighers and implementing new scheduling algorithms with just these 
2 knobs was a little hard. I did change the code in the filter_scheduler to get 
some basic algorithms running like the first and next fit apart from the 
spreading and stacking which was already present. But to go beyond and to 
implement more complex algorithms was much harder and I would have to change a 
lot of code in different places that could as a side effect also break things 
and didn't seem clean. I might be wrong and might have not understood things 
right, please correct me if so.

To give an example of what I mean by a little complex scheduling algorithms: a 
subset matching algorithm - that schedules multiple heterogeneous requests by 
picking out a subset from the requests that best fit a host/s, so this would 
improve the utilization. The prerequisite for this is that I have multiple 
heterogeneous requests lined up to be scheduled. So for this kind of an 
algorithm it isnt easy to implement into OpenStack.

So a workaround that I'm working on for implementing different scheduling 
algorithms is by building a scheduling wrapper outside of the OpenStack 
architecture, where the user interacts with this wrapper and in the wrapper I 
get the host details from the database and based on the algorithm I want, the 
scheduler chooses the host for the request and gives out a VM : Host mapping 
(The wrapper does the sanity checks that the filters do to check if the host 
can accommodate or handle the request). Along with the request, I also want to 
pass this mapping that the scheduler can use to assign the request to the host 
passed in the mapping. I've written a filter that filters all the hosts apart 
from the host that I sent and this is how I make sure that the request gets 
placed on the host that I had passed. I have come up with a hack to pass the 
host to the scheduler, but it is not quite elegant.

Would be great to have your input on the same!

On Mon, Feb 8, 2016 at 12:51 AM, Jay Pipes 
<jaypi...@gmail.com<mailto:jaypi...@gmail.com>> wrote:
Apologies for the delayed responses. Comments inline.

On 01/27/2016 02:29 AM, Dhvanan Shah wrote:
Hey Jay!

Thanks for the clarification. There was another thing that I wanted to
know, is there any provision to pass extra arguments or some extra
specifications along with the VM request to nova. To give you some
context, I wanted to pass a host:vm mapping to the nova scheduler for
its host selection process, and I'm providing this mapping from outside
of the openstack architecture.

Why do you want to do this? The scheduler is the thing that sets the host -> vm 
mapping -- that's what the process of scheduling does.

> So I need to send this information along
with the request to the scheduler. One way of doing this was creating
new flavors with their extra specification as different hosts, but that
would lead to as you pointed out earlier a "flavor explosion" problem.

So is there a way to pass some extra arguments or some additional
information to nova.

Depends what exactly you are trying to pass to Nova. Could you give some more 
information about your use case?

Thanks!
-jay



--
Dhvanan Shah
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] Dynamically adding Extra Specs

[Jay Pipes]

Apologies for the delayed responses. Comments inline.

On 01/27/2016 02:29 AM, Dhvanan Shah wrote:

Hey Jay!

Thanks for the clarification. There was another thing that I wanted to
know, is there any provision to pass extra arguments or some extra
specifications along with the VM request to nova. To give you some
context, I wanted to pass a host:vm mapping to the nova scheduler for
its host selection process, and I'm providing this mapping from outside
of the openstack architecture.


Why do you want to do this? The scheduler is the thing that sets the 
host -> vm mapping -- that's what the process of scheduling does.


> So I need to send this information along

with the request to the scheduler. One way of doing this was creating
new flavors with their extra specification as different hosts, but that
would lead to as you pointed out earlier a "flavor explosion" problem.

So is there a way to pass some extra arguments or some additional
information to nova.


Depends what exactly you are trying to pass to Nova. Could you give some 
more information about your use case?


Thanks!
-jay

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] Dynamically adding Extra Specs

[Dhvanan Shah]

Hey Jay!

I was looking at implementing a few scheduling algorithms of my own
natively into OpenStack, and for that I went through the nova-scheduler.
After going through the scheduler, I felt that it was not very easy to
implement or extend and add new scheduling algorithms to the scheduler. The
only things that I felt that I could change or maybe was provisioned for
adding or extending were the filters and weighers and implementing new
scheduling algorithms with just these 2 knobs was a little hard. I did
change the code in the filter_scheduler to get some basic algorithms
running like the first and next fit apart from the spreading and stacking
which was already present. But to go beyond and to implement more complex
algorithms was much harder and I would have to change a lot of code in
different places that could as a side effect also break things and didn't
seem clean. I might be wrong and might have not understood things right,
please correct me if so.

To give an example of what I mean by a little complex scheduling
algorithms: a subset matching algorithm - that schedules multiple
heterogeneous requests by picking out a subset from the requests that best
fit a host/s, so this would improve the utilization. The prerequisite for
this is that I have multiple heterogeneous requests lined up to be
scheduled. So for this kind of an algorithm it isnt easy to implement into
OpenStack.

So a workaround that I'm working on for implementing different scheduling
algorithms is by building a scheduling wrapper outside of the OpenStack
architecture, where the user interacts with this wrapper and in the wrapper
I get the host details from the database and based on the algorithm I want,
the scheduler chooses the host for the request and gives out a VM : Host
mapping (The wrapper does the sanity checks that the filters do to check if
the host can accommodate or handle the request). Along with the request, I
also want to pass this mapping that the scheduler can use to assign the
request to the host passed in the mapping. I've written a filter that
filters all the hosts apart from the host that I sent and this is how I
make sure that the request gets placed on the host that I had passed. I
have come up with a hack to pass the host to the scheduler, but it is not
quite elegant.

Would be great to have your input on the same!

On Mon, Feb 8, 2016 at 12:51 AM, Jay Pipes  wrote:

> Apologies for the delayed responses. Comments inline.
>
> On 01/27/2016 02:29 AM, Dhvanan Shah wrote:
>
>> Hey Jay!
>>
>> Thanks for the clarification. There was another thing that I wanted to
>> know, is there any provision to pass extra arguments or some extra
>> specifications along with the VM request to nova. To give you some
>> context, I wanted to pass a host:vm mapping to the nova scheduler for
>> its host selection process, and I'm providing this mapping from outside
>> of the openstack architecture.
>>
>
> Why do you want to do this? The scheduler is the thing that sets the host
> -> vm mapping -- that's what the process of scheduling does.
>
> > So I need to send this information along
>
>> with the request to the scheduler. One way of doing this was creating
>> new flavors with their extra specification as different hosts, but that
>> would lead to as you pointed out earlier a "flavor explosion" problem.
>>
>> So is there a way to pass some extra arguments or some additional
>> information to nova.
>>
>
> Depends what exactly you are trying to pass to Nova. Could you give some
> more information about your use case?
>
> Thanks!
> -jay
>



-- 
Dhvanan Shah
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] Dynamically adding Extra Specs

[Dhvanan Shah]

Hey Jay!

Thanks for the clarification. There was another thing that I wanted to
know, is there any provision to pass extra arguments or some extra
specifications along with the VM request to nova. To give you some context,
I wanted to pass a host:vm mapping to the nova scheduler for its host
selection process, and I'm providing this mapping from outside of the
openstack architecture. So I need to send this information along with the
request to the scheduler. One way of doing this was creating new flavors
with their extra specification as different hosts, but that would lead to
as you pointed out earlier a "flavor explosion" problem.

So is there a way to pass some extra arguments or some additional
information to nova.

On Fri, Jan 22, 2016 at 7:17 PM, Jay Pipes  wrote:

> Hi! Comments inline...
>
> On 01/22/2016 01:26 AM, Dhvanan Shah wrote:
>
>> Hi,
>>
>> I had a few queries regarding adding extra specs for VM requests.
>>
>> According to my understanding if I want to add extra specs to requests
>> then I need to change that in different flavors adding those
>> capabilities by setting them in the flavors. But if the requests that I
>> get have varying values for those extra capabilities then it seems to
>> create an issue as the values in the flavors are static. Please correct
>> me if I'm wrong.
>>
>
> Right, flavors are static blobs of both resources (amounts of things
> requested by the user) and capabilities (free-form key/value pairs in
> extra_specs). For every variation you may want to offer your cloud users,
> you need to create a new flavor in the system, and your cloud users must
> select that flavor when booting a VM.
>
> Don Dugger from Intel has called this problem "flavor explosion", which I
> have comically re-labeled "the Skittles problem". We have a long-term plan
> to allow for the ability to list a set of capabilities present in the
> deployment and allow a cloud user to "mix and match" resource amounts and
> required capabilities in a more flexible way, but that work is likely 9-12
> months out in reality.
>
> So I wanted to know as to how I could dynamically add those extra specs
>> best suiting each request. Is there a way of mentioning the extra specs
>> everytime I spawn a VM through the nova cli? Setting and unsetting the
>> extra specs everytime I spawn VM's according to my need would be quite
>> inefficient as it makes changes to the database.
>>
>
> Yes, it is a clunky and inflexible API right now, I agree. The genesis of
> the flavor concept comes from Rackspace Cloud Servers, which modeled its
> flavor concept on the Amazon EC2 instance types concept. This model works
> well for public clouds, because a) they use the flavor as a stock-keeping
> unit (SKU) so therefore this system makes billing easier, and b) it reduces
> the number of hardware configuration options that public cloud operators
> need to provide at scale to users.
>
> If you are interested in this area, follow conversations and blueprints
> with the terms "flavor decomposition", "host capabilities", and "resource
> representation".
>
> Best,
> -jay
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>



-- 
Dhvanan Shah
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] Dynamically adding Extra Specs

[Jay Pipes]

Hi! Comments inline...

On 01/22/2016 01:26 AM, Dhvanan Shah wrote:

Hi,

I had a few queries regarding adding extra specs for VM requests.

According to my understanding if I want to add extra specs to requests
then I need to change that in different flavors adding those
capabilities by setting them in the flavors. But if the requests that I
get have varying values for those extra capabilities then it seems to
create an issue as the values in the flavors are static. Please correct
me if I'm wrong.


Right, flavors are static blobs of both resources (amounts of things 
requested by the user) and capabilities (free-form key/value pairs in 
extra_specs). For every variation you may want to offer your cloud 
users, you need to create a new flavor in the system, and your cloud 
users must select that flavor when booting a VM.


Don Dugger from Intel has called this problem "flavor explosion", which 
I have comically re-labeled "the Skittles problem". We have a long-term 
plan to allow for the ability to list a set of capabilities present in 
the deployment and allow a cloud user to "mix and match" resource 
amounts and required capabilities in a more flexible way, but that work 
is likely 9-12 months out in reality.



So I wanted to know as to how I could dynamically add those extra specs
best suiting each request. Is there a way of mentioning the extra specs
everytime I spawn a VM through the nova cli? Setting and unsetting the
extra specs everytime I spawn VM's according to my need would be quite
inefficient as it makes changes to the database.


Yes, it is a clunky and inflexible API right now, I agree. The genesis 
of the flavor concept comes from Rackspace Cloud Servers, which modeled 
its flavor concept on the Amazon EC2 instance types concept. This model 
works well for public clouds, because a) they use the flavor as a 
stock-keeping unit (SKU) so therefore this system makes billing easier, 
and b) it reduces the number of hardware configuration options that 
public cloud operators need to provide at scale to users.


If you are interested in this area, follow conversations and blueprints 
with the terms "flavor decomposition", "host capabilities", and 
"resource representation".


Best,
-jay

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] Dynamically adding Extra Specs

[Dhvanan Shah]

Hi,

I had a few queries regarding adding extra specs for VM requests.

According to my understanding if I want to add extra specs to requests then
I need to change that in different flavors adding those capabilities by
setting them in the flavors. But if the requests that I get have varying
values for those extra capabilities then it seems to create an issue as the
values in the flavors are static. Please correct me if I'm wrong.

So I wanted to know as to how I could dynamically add those extra specs
best suiting each request. Is there a way of mentioning the extra specs
everytime I spawn a VM through the nova cli? Setting and unsetting the
extra specs everytime I spawn VM's according to my need would be quite
inefficient as it makes changes to the database.


Thanks,
Dhvanan Shah
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev