Re: [openstack-dev] Compute API (Was Re: [nova][cinder] how to handle AZ bug 1496235?)

2015-09-28 Thread Sylvain Bauza 



Le 28/09/2015 16:19, Sean Dague a écrit :

On 09/28/2015 10:11 AM, Andrew Laski wrote:

On 09/28/15 at 08:50am, Monty Taylor wrote:

On 09/28/2015 07:58 AM, Sylvain Bauza wrote:



Specifically, I want "nova boot" to get me a VM with an IP address. I
don't want it to do fancy orchestration - I want it to not need fancy
orchestration, because needing fancy orchestration to get a VM  on a
network is not a feature.

In the networking case there is a minimum of orchestration because the
time required to allocate a port is small.  What has been requiring
orchestration is the creation of volumes because of the requirement of
Cinder to download an image, or be on a backend that support fast
cloning and rely on a cache hit.  So the question under discussion is
when booting an instance relies on another service performing a long
running operation where is a good place to handle that.

My thinking for a while has been that we could use another API that
could manage those things.  And be the central place you're looking for
to pass a simple "nova boot" with whatever options are required so you
don't have to manage the complexities of calls to
Neutron/Cinder/Nova(current API).  What's become clear to me from this
thread is that people don't seem to oppose that idea, however they don't
want their users/clients to need to switch what API they're currently
using(Nova).

The right way to proceed with this idea seems to be to by evolving the
Nova API and potentially creating a split down the road.  And by split I
more mean architectural within Nova, and not necessarily a split API.
What I imagine is that we follow the model of git and have a plumbing
and porcelain API and each can focus on doing the right things.

Right, and I think that's a fine approach. Nova's job is "give me a
working VM". Working includes networking, persistent storage. The API
semantics for "give me a working VM" should exist in Nova.

It is also fine if there are lower level calls that tweak parts of that,
but nova boot shouldn't have to be a multi step API process for the
user. Building one working VM you can do something with is really the
entire point of Nova.


I'm all for a request with some network and volume semantics in it, 
which would imply that the Nova scheduler would be able to do some 
instance placement based on cross-project resources.


What is a grey line to me is the fact that "give me a volume-backed 
instance from this image" requires some volume creation to get it done. 
So, if we consider that Nova is the best API for it (and I can 
understand the motivation for it), then we need some clear architectural 
segmentation between Nova and the other projects in Nova, like Andrew 
said (sorry if you feel I'm paraphrasing). For example, the move to 
os-brick is one of the efforts we should do, but it's just one step, 
since we should decouple all the tasks involved in the instance creation 
to isolate those in a better way).


-Sylvain


-Sean




__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] Compute API (Was Re: [nova][cinder] how to handle AZ bug 1496235?)

2015-09-28 Thread Andrew Laski 

On 09/28/15 at 10:19am, Sean Dague wrote:

On 09/28/2015 10:11 AM, Andrew Laski wrote:

On 09/28/15 at 08:50am, Monty Taylor wrote:

On 09/28/2015 07:58 AM, Sylvain Bauza wrote:




Specifically, I want "nova boot" to get me a VM with an IP address. I
don't want it to do fancy orchestration - I want it to not need fancy
orchestration, because needing fancy orchestration to get a VM  on a
network is not a feature.


In the networking case there is a minimum of orchestration because the
time required to allocate a port is small.  What has been requiring
orchestration is the creation of volumes because of the requirement of
Cinder to download an image, or be on a backend that support fast
cloning and rely on a cache hit.  So the question under discussion is
when booting an instance relies on another service performing a long
running operation where is a good place to handle that.

My thinking for a while has been that we could use another API that
could manage those things.  And be the central place you're looking for
to pass a simple "nova boot" with whatever options are required so you
don't have to manage the complexities of calls to
Neutron/Cinder/Nova(current API).  What's become clear to me from this
thread is that people don't seem to oppose that idea, however they don't
want their users/clients to need to switch what API they're currently
using(Nova).

The right way to proceed with this idea seems to be to by evolving the
Nova API and potentially creating a split down the road.  And by split I
more mean architectural within Nova, and not necessarily a split API.
What I imagine is that we follow the model of git and have a plumbing
and porcelain API and each can focus on doing the right things.


Right, and I think that's a fine approach. Nova's job is "give me a
working VM". Working includes networking, persistent storage. The API
semantics for "give me a working VM" should exist in Nova.

It is also fine if there are lower level calls that tweak parts of that,
but nova boot shouldn't have to be a multi step API process for the
user. Building one working VM you can do something with is really the
entire point of Nova.


What I'm struggling with is where do we draw the line in this model?  
For instance we don't allow a user to boot an instance from a disk image 
on their local machine via the Nova API, that is a multi step process.  
And which parameters do we expose that can influence network and volume 
creation, if not all of them?  It would be helpful to establish 
guidelines on what is a good candidate for inclusion in Nova.


I see a clear line between something that handles the creation of all 
ancillary resources needed to boot a VM and then the creation of the VM 
itself.  I don't understand why the creation of the other resources 
should live within Nova but as long as we can get to a good split 
between responsibilities that's a secondary concern.




-Sean

--
Sean Dague
http://dague.net

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] Compute API (Was Re: [nova][cinder] how to handle AZ bug 1496235?)

2015-09-28 Thread Mike Spreitzer 
> From: Monty Taylor <mo...@inaugust.com>
> To: Sylvain Bauza <sba...@redhat.com>, "OpenStack Development 
> Mailing List (not for usage questions)" 
<openstack-dev@lists.openstack.org>
> Date: 09/28/2015 09:54 AM
> Subject: Re: [openstack-dev] Compute API (Was Re: [nova][cinder] how
> to handle AZ bug 1496235?)
>
> ...
> Specifically, I want "nova boot" to get me a VM with an IP address. I 
> don't want it to do fancy orchestration - I want it to not need fancy 
> orchestration, because needing fancy orchestration to get a VM  on a 
> network is not a feature.
> 
> I also VERY MUCH do not want to need Heat to get a VM. I want to use 
> Heat to do something complex. Getting a VM is not complex. It should not 

> be complex. What it's complex and to the level of needing Heat, we've 
> failed somewhere else.
> 
> Also, people should stop deploying clouds that require people to use 
> floating IPs to get basic internet access. It's a misuse of the 
construct.
> 
> Public Network "ext-net" -> shared / directly attachable
> Per-tenant Network "private" -> private network, not shared, not 
routable
> 
> If the user chooses, a router can be added with gateway set to ext-net.
> 
> This way:
> 
> nova boot --network=ext-net  -> vm dhcp'd on the public network
> nova boot --network=private  -> vm dhcp'd on the private network
> nova floating-ip-attach  -> vm gets a floating ip attached to their 
> vm from the ext-net network
> 
> All of the use cases are handled, basic things are easy (boot a vm on 
> the network works in one step) and for the 5% of cases where a floating 
> IP is actually needed (a long-lived service on a single vm that wants to 

> keep the IP and not just a DNS name across VM migrations and isn't using 

> a load-balancer) can use that.
> 
> This is, btw, the most common public cloud deployment model.
> 
> Let's stop making things harder than they need to be and serve our 
users.

As an operator, +1

Mike



__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] Compute API (Was Re: [nova][cinder] how to handle AZ bug 1496235?)

2015-09-28 Thread Ryan Brown 

On 09/26/2015 12:04 AM, Joshua Harlow wrote:

+1 from me, although I thought heat was supposed to be this thing?

Maybe there should be a 'warm' project or something ;)

Or we can call it 'bbs' for 'building block service' (obviously not
bulletin board system); ask said service to build a set of blocks into
well defined structures and let it figure out how to make that happen...

This though most definitely requires cross-project agreement though so
I'd hope we can reach that somehow (before creating a halfway done new
orchestration thing that is halfway integrated with a bunch of other
apis that do one quarter of the work in ten different ways).


Indeed, I don't think I understand what need heat is failing to fulfill 
here? A user can easily have a template that contains a single server 
and a volume.


Heat's job is to be an API that lets you define a result[1] and then 
calls the APIs of whatever projects provide those things.


1: in this case, the result is "a working server with network and storage"


Duncan Thomas wrote:

I think there's a place for yet another service breakout from nova -
some sort of like-weight platform orchestration piece, nothing as
complicated or complete as heat, nothing that touches the inside of a
VM, just something that can talk to cinder, nova and neutron (plus I
guess ironic and whatever the container thing is called) and work
through long running / cross-project tasks. I'd probably expect it to
provide a task style interface, e.g. a boot-from-new-volume call returns
a request-id that can then be polled for detailed status.

The existing nova API for this (and any other nova APIs where this makes
sense) can then become a proxy for the new service, so that tenants are
not affected. The nova apis can then be deprecated in slow time.

Anybody else think this could be useful?


--
Ryan Brown / Senior Software Engineer, Openstack / Red Hat, Inc.

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] Compute API (Was Re: [nova][cinder] how to handle AZ bug 1496235?)

2015-09-28 Thread Andrew Laski 

On 09/28/15 at 08:50am, Monty Taylor wrote:

On 09/28/2015 07:58 AM, Sylvain Bauza wrote:



Le 28/09/2015 12:35, Duncan Thomas a écrit :



On 28 September 2015 at 12:35, Sylvain Bauza > wrote:

   About the maintenance burden, I also consider that patching
   clients is far more easier than patching an API unless I missed
   something.


I think I very much disagree there - patching a central installation
is much, much easier than getting N customers to patch M different
libraries, even assuming the fix is available for any significant
subset of the M libraries, plus making sure that new customers use the
correct libraries, plus helping any customers who have some sort of
roll-your-own library do the new right thing...



Well, having N versions of clients against one single API version is
just something we manage since the beginning. I don't really see why it
suddently becomes so difficult to manage it.



I think there's a definite place for a simple API to do infrastructure
level orchestration without needing the complexities of heat - these
APIs are in nova because they're useful - there's clear operator
desire for them and a couple of operators have been quite vocal about
their desire for them not to be removed. Great, let's keep them, but
form a team of people interested in getting them right (get rid of
fixed timeouts, etc), add any missing pieces (like floating IPs for
new VMs) and generally focus on getting this piece of the puzzle
right. Breaking another small piece off nova and polishing it has been
a generally successful pattern.


I don't want to overthink what could be the right scope of that future
API but given the Heat mission statement [1] and its service name
'orchestration', I don't see why this API endpoint should land in the
Nova codebase and couldn't be rather provided by the Heat API. Oh sure,
it would perhaps require another endpoint behind the same service, but
isn't that better than having another endpoint in Nova ?

[1]
https://github.com/openstack/governance/blob/master/reference/projects.yaml#L482-L484




I remember Monty Taylor (copied) having a rant about the lack of the
perfect 'give me a VM with all its stuff sorted' API. Care to comment,
Monty?


Sounds you misunderstood me. I'm not against implementing this excellent
usecase, I just think the best place is not in Nova and should be done
elsewhere.



Specifically, I want "nova boot" to get me a VM with an IP address. I 
don't want it to do fancy orchestration - I want it to not need fancy 
orchestration, because needing fancy orchestration to get a VM  on a 
network is not a feature.


In the networking case there is a minimum of orchestration because the 
time required to allocate a port is small.  What has been requiring 
orchestration is the creation of volumes because of the requirement of 
Cinder to download an image, or be on a backend that support fast 
cloning and rely on a cache hit.  So the question under discussion is 
when booting an instance relies on another service performing a long 
running operation where is a good place to handle that.


My thinking for a while has been that we could use another API that 
could manage those things.  And be the central place you're looking for 
to pass a simple "nova boot" with whatever options are required so you 
don't have to manage the complexities of calls to 
Neutron/Cinder/Nova(current API).  What's become clear to me from this 
thread is that people don't seem to oppose that idea, however they don't 
want their users/clients to need to switch what API they're currently 
using(Nova).


The right way to proceed with this idea seems to be to by evolving the 
Nova API and potentially creating a split down the road.  And by split I 
more mean architectural within Nova, and not necessarily a split API.  
What I imagine is that we follow the model of git and have a plumbing 
and porcelain API and each can focus on doing the right things.





I also VERY MUCH do not want to need Heat to get a VM. I want to use 
Heat to do something complex. Getting a VM is not complex. It should 
not be complex. What it's complex and to the level of needing Heat, 
we've failed somewhere else.


Also, people should stop deploying clouds that require people to use 
floating IPs to get basic internet access. It's a misuse of the 
construct.


Public Network "ext-net" -> shared / directly attachable
Per-tenant Network "private" -> private network, not shared, not routable

If the user chooses, a router can be added with gateway set to ext-net.

This way:

nova boot --network=ext-net  -> vm dhcp'd on the public network
nova boot --network=private  -> vm dhcp'd on the private network
nova floating-ip-attach  -> vm gets a floating ip attached to 
their vm from the ext-net network


All of the use cases are handled, basic things are easy (boot a vm on 
the network works in one step) and for the 5% of cases where a 
floating IP is actually

Re: [openstack-dev] Compute API (Was Re: [nova][cinder] how to handle AZ bug 1496235?)

2015-09-28 Thread Sean Dague 
On 09/28/2015 10:11 AM, Andrew Laski wrote:
> On 09/28/15 at 08:50am, Monty Taylor wrote:
>> On 09/28/2015 07:58 AM, Sylvain Bauza wrote:

>>
>> Specifically, I want "nova boot" to get me a VM with an IP address. I
>> don't want it to do fancy orchestration - I want it to not need fancy
>> orchestration, because needing fancy orchestration to get a VM  on a
>> network is not a feature.
> 
> In the networking case there is a minimum of orchestration because the
> time required to allocate a port is small.  What has been requiring
> orchestration is the creation of volumes because of the requirement of
> Cinder to download an image, or be on a backend that support fast
> cloning and rely on a cache hit.  So the question under discussion is
> when booting an instance relies on another service performing a long
> running operation where is a good place to handle that.
> 
> My thinking for a while has been that we could use another API that
> could manage those things.  And be the central place you're looking for
> to pass a simple "nova boot" with whatever options are required so you
> don't have to manage the complexities of calls to
> Neutron/Cinder/Nova(current API).  What's become clear to me from this
> thread is that people don't seem to oppose that idea, however they don't
> want their users/clients to need to switch what API they're currently
> using(Nova).
> 
> The right way to proceed with this idea seems to be to by evolving the
> Nova API and potentially creating a split down the road.  And by split I
> more mean architectural within Nova, and not necessarily a split API. 
> What I imagine is that we follow the model of git and have a plumbing
> and porcelain API and each can focus on doing the right things.

Right, and I think that's a fine approach. Nova's job is "give me a
working VM". Working includes networking, persistent storage. The API
semantics for "give me a working VM" should exist in Nova.

It is also fine if there are lower level calls that tweak parts of that,
but nova boot shouldn't have to be a multi step API process for the
user. Building one working VM you can do something with is really the
entire point of Nova.

-Sean

-- 
Sean Dague
http://dague.net

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] Compute API (Was Re: [nova][cinder] how to handle AZ bug 1496235?)

2015-09-28 Thread Sylvain Bauza 



Le 25/09/2015 16:12, Andrew Laski a écrit :

On 09/24/15 at 03:13pm, James Penick wrote:



At risk of getting too offtopic I think there's an alternate 
solution to
doing this in Nova or on the client side.  I think we're missing 
some sort

of OpenStack API and service that can handle this.  Nova is a low level
infrastructure API and service, it is not designed to handle these
orchestrations.  I haven't checked in on Heat in a while but perhaps 
this

is a role that it could fill.

I think that too many people consider Nova to be *the* OpenStack API 
when
considering instances/volumes/networking/images and that's not 
something I

would like to see continue.  Or at the very least I would like to see a
split between the orchestration/proxy pieces and the "manage my
VM/container/baremetal" bits



(new thread)
You've hit on one of my biggest issues right now: As far as many 
deployers

and consumers are concerned (and definitely what I tell my users within
Yahoo): The value of an OpenStack value-stream (compute, network, 
storage)

is to provide a single consistent API for abstracting and managing those
infrastructure resources.

Take networking: I can manage Firewalls, switches, IP selection, SDN, 
etc

through Neutron. But for compute, If I want VM I go through Nova, for
Baremetal I can -mostly- go through Nova, and for containers I would 
talk

to Magnum or use something like the nova docker driver.

This means that, by default, Nova -is- the closest thing to a top level
abstraction layer for compute. But if that is explicitly against Nova's
charter, and Nova isn't going to be the top level abstraction for all
things Compute, then something else needs to fill that space. When that
happens, all things common to compute provisioning should come out of 
Nova

and move into that new API. Availability zones, Quota, etc.


I do think Nova is the top level abstraction layer for compute. My 
issue is when Nova is asked to manage other resources.  There's no API 
call to tell Cinder "create a volume and attach it to this instance, 
and create that instance if it doesn't exist."  And I'm not sure why 
the reverse isn't true.


I want Nova to be the absolute best API for managing compute 
resources.  It's when someone is managing compute and volumes and 
networks together that I don't feel that Nova is the best place for 
that.  Most importantly right now it seems that not everyone is on the 
same page on this and I think it would be beneficial to come together 
and figure out what sort of workloads the Nova API is intending to 
provide.


I totally agree with you on those points :
 - nova API should be only supporting CRUD operations for compute VMs 
and should no longer manage neither volumes nor networks IMHO, because 
it creates more problems than it resolves
 - given the above, nova API could possibly accept resources from 
networks or volumes but only for placement decisions related to instances.


Tho, I can also understand that operators sometimes just want a single 
tool for creating this kind of relationship between a volume and an 
instance (and not provide a YAML file), but IMHO, it doesn't perhaps 
need a top-level API, just a python client able to do some very simple 
orchestration between services, something like openstack-client.


I don't really see a uber-value for getting a proxy API calling Nova or 
Neutron. IMHO, that should still be done by clients, not services.


-Sylvain





-James


__ 


OpenStack Development Mailing List (not for usage questions)
Unsubscribe: 
openstack-dev-requ...@lists.openstack.org?subject:unsubscribe

http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



__ 


OpenStack Development Mailing List (not for usage questions)
Unsubscribe: 
openstack-dev-requ...@lists.openstack.org?subject:unsubscribe

http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] Compute API (Was Re: [nova][cinder] how to handle AZ bug 1496235?)

2015-09-28 Thread Sylvain Bauza 



Le 28/09/2015 11:23, Duncan Thomas a écrit :


The trouble with putting more intelligence in the clients is that 
there are more clients than just the one we provide, and the more 
smarts we require in the clients, the more divergence of functionality 
we're likely to see. Also, bugs and slowly percolating bug fixes.




That's why I consider the layer of orchestration in the client just 
being as identical as what we have in Nova, not more than that. If we 
require more than just a volume creation when asking to boot from a 
volume with source=image, then I agree with you, it has nothing to do in 
the client, but rather in Heat.


The same goes with networks. What is done with Nova for managing CRUD 
operations can be done in python clients, but that's the limit.


About the maintenance burden, I also consider that patching clients is 
far more easier than patching an API unless I missed something.


-Sylvain


On 28 Sep 2015 11:27, "Sylvain Bauza" > wrote:




Le 25/09/2015 16:12, Andrew Laski a écrit :

On 09/24/15 at 03:13pm, James Penick wrote:



At risk of getting too offtopic I think there's an
alternate solution to
doing this in Nova or on the client side.  I think
we're missing some sort
of OpenStack API and service that can handle this. 
Nova is a low level

infrastructure API and service, it is not designed to
handle these
orchestrations.  I haven't checked in on Heat in a
while but perhaps this
is a role that it could fill.

I think that too many people consider Nova to be *the*
OpenStack API when
considering instances/volumes/networking/images and
that's not something I
would like to see continue.  Or at the very least I
would like to see a
split between the orchestration/proxy pieces and the
"manage my
VM/container/baremetal" bits



(new thread)
You've hit on one of my biggest issues right now: As far
as many deployers
and consumers are concerned (and definitely what I tell my
users within
Yahoo): The value of an OpenStack value-stream (compute,
network, storage)
is to provide a single consistent API for abstracting and
managing those
infrastructure resources.

Take networking: I can manage Firewalls, switches, IP
selection, SDN, etc
through Neutron. But for compute, If I want VM I go
through Nova, for
Baremetal I can -mostly- go through Nova, and for
containers I would talk
to Magnum or use something like the nova docker driver.

This means that, by default, Nova -is- the closest thing
to a top level
abstraction layer for compute. But if that is explicitly
against Nova's
charter, and Nova isn't going to be the top level
abstraction for all
things Compute, then something else needs to fill that
space. When that
happens, all things common to compute provisioning should
come out of Nova
and move into that new API. Availability zones, Quota, etc.


I do think Nova is the top level abstraction layer for
compute. My issue is when Nova is asked to manage other
resources.  There's no API call to tell Cinder "create a
volume and attach it to this instance, and create that
instance if it doesn't exist."  And I'm not sure why the
reverse isn't true.

I want Nova to be the absolute best API for managing compute
resources.  It's when someone is managing compute and volumes
and networks together that I don't feel that Nova is the best
place for that.  Most importantly right now it seems that not
everyone is on the same page on this and I think it would be
beneficial to come together and figure out what sort of
workloads the Nova API is intending to provide.


I totally agree with you on those points :
 - nova API should be only supporting CRUD operations for compute
VMs and should no longer manage neither volumes nor networks IMHO,
because it creates more problems than it resolves
 - given the above, nova API could possibly accept resources from
networks or volumes but only for placement decisions related to
instances.

Tho, I can also understand that operators sometimes just want a
single tool for creating this kind of relationship between a
volume and an instance (and not provide a YAML file), but IMHO, it
doesn't perhaps need a top-level API, just a

Re: [openstack-dev] Compute API (Was Re: [nova][cinder] how to handle AZ bug 1496235?)

2015-09-28 Thread Duncan Thomas 
On 28 September 2015 at 12:35, Sylvain Bauza  wrote:

> About the maintenance burden, I also consider that patching clients is far
> more easier than patching an API unless I missed something.
>
>
I think I very much disagree there - patching a central installation is
much, much easier than getting N customers to patch M different libraries,
even assuming the fix is available for any significant subset of the M
libraries, plus making sure that new customers use the correct libraries,
plus helping any customers who have some sort of roll-your-own library do
the new right thing...

I think there's a definite place for a simple API to do infrastructure
level orchestration without needing the complexities of heat - these APIs
are in nova because they're useful - there's clear operator desire for them
and a couple of operators have been quite vocal about their desire for them
not to be removed. Great, let's keep them, but form a team of people
interested in getting them right (get rid of fixed timeouts, etc), add any
missing pieces (like floating IPs for new VMs) and generally focus on
getting this piece of the puzzle right. Breaking another small piece off
nova and polishing it has been a generally successful pattern.

I remember Monty Taylor (copied) having a rant about the lack of the
perfect 'give me a VM with all its stuff sorted' API. Care to comment,
Monty?
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] Compute API (Was Re: [nova][cinder] how to handle AZ bug 1496235?)

2015-09-28 Thread Duncan Thomas 
The trouble with putting more intelligence in the clients is that there are
more clients than just the one we provide, and the more smarts we require
in the clients, the more divergence of functionality we're likely to see.
Also, bugs and slowly percolating bug fixes.
On 28 Sep 2015 11:27, "Sylvain Bauza"  wrote:

>
>
> Le 25/09/2015 16:12, Andrew Laski a écrit :
>
>> On 09/24/15 at 03:13pm, James Penick wrote:
>>
>>>

 At risk of getting too offtopic I think there's an alternate solution to
 doing this in Nova or on the client side.  I think we're missing some
 sort
 of OpenStack API and service that can handle this.  Nova is a low level
 infrastructure API and service, it is not designed to handle these
 orchestrations.  I haven't checked in on Heat in a while but perhaps
 this
 is a role that it could fill.

 I think that too many people consider Nova to be *the* OpenStack API
 when
 considering instances/volumes/networking/images and that's not
 something I
 would like to see continue.  Or at the very least I would like to see a
 split between the orchestration/proxy pieces and the "manage my
 VM/container/baremetal" bits

>>>
>>>
>>> (new thread)
>>> You've hit on one of my biggest issues right now: As far as many
>>> deployers
>>> and consumers are concerned (and definitely what I tell my users within
>>> Yahoo): The value of an OpenStack value-stream (compute, network,
>>> storage)
>>> is to provide a single consistent API for abstracting and managing those
>>> infrastructure resources.
>>>
>>> Take networking: I can manage Firewalls, switches, IP selection, SDN, etc
>>> through Neutron. But for compute, If I want VM I go through Nova, for
>>> Baremetal I can -mostly- go through Nova, and for containers I would talk
>>> to Magnum or use something like the nova docker driver.
>>>
>>> This means that, by default, Nova -is- the closest thing to a top level
>>> abstraction layer for compute. But if that is explicitly against Nova's
>>> charter, and Nova isn't going to be the top level abstraction for all
>>> things Compute, then something else needs to fill that space. When that
>>> happens, all things common to compute provisioning should come out of
>>> Nova
>>> and move into that new API. Availability zones, Quota, etc.
>>>
>>
>> I do think Nova is the top level abstraction layer for compute. My issue
>> is when Nova is asked to manage other resources.  There's no API call to
>> tell Cinder "create a volume and attach it to this instance, and create
>> that instance if it doesn't exist."  And I'm not sure why the reverse isn't
>> true.
>>
>> I want Nova to be the absolute best API for managing compute resources.
>> It's when someone is managing compute and volumes and networks together
>> that I don't feel that Nova is the best place for that.  Most importantly
>> right now it seems that not everyone is on the same page on this and I
>> think it would be beneficial to come together and figure out what sort of
>> workloads the Nova API is intending to provide.
>>
>
> I totally agree with you on those points :
>  - nova API should be only supporting CRUD operations for compute VMs and
> should no longer manage neither volumes nor networks IMHO, because it
> creates more problems than it resolves
>  - given the above, nova API could possibly accept resources from networks
> or volumes but only for placement decisions related to instances.
>
> Tho, I can also understand that operators sometimes just want a single
> tool for creating this kind of relationship between a volume and an
> instance (and not provide a YAML file), but IMHO, it doesn't perhaps need a
> top-level API, just a python client able to do some very simple
> orchestration between services, something like openstack-client.
>
> I don't really see a uber-value for getting a proxy API calling Nova or
> Neutron. IMHO, that should still be done by clients, not services.
>
> -Sylvain
>
>
>>
>>> -James
>>>
>>
>> __
>>>
>>> OpenStack Development Mailing List (not for usage questions)
>>> Unsubscribe:
>>> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>
>>
>>
>> __
>>
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe:
>> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>

Re: [openstack-dev] Compute API (Was Re: [nova][cinder] how to handle AZ bug 1496235?)

2015-09-28 Thread James Penick 
>I see a clear line between something that handles the creation of all
ancillary resources needed to boot a VM and then the creation of the VM
itself.

I agree. To me the line is the difference between creating a top level
resource, and adding a host to that resource.

For example, I do expect a top level compute API to:
-Request an IP from Neutron
-Associate an instance with a volume
-Add an instance to a network security group in Neutron
-Add a real to a vip in neutron

But I don't expect Nova to:
-Create tenant/provider networks in Neutron
-Create a volume (boot from volume is a weird case)
-Create a neutron security group
-Create a load balancer

Also, if Nova is the API for all things compute, then there are some things
it will need to support that are not specific to VMs. For example, with
Ironic my users expect to use the Nova API/CLI to boot a baremetal compute
resource, and specify raid configuration as well as drive layout. My
understanding is there has been pushback on adding that to Nova, since it
doesn't make sense to have RAID config when building a VM. But, if Nova is
the compute abstraction layer, then we'll need a graceful way to express
this.

-James



On Mon, Sep 28, 2015 at 9:12 AM, Andrew Laski  wrote:

> On 09/28/15 at 10:19am, Sean Dague wrote:
>
>> On 09/28/2015 10:11 AM, Andrew Laski wrote:
>>
>>> On 09/28/15 at 08:50am, Monty Taylor wrote:
>>>
 On 09/28/2015 07:58 AM, Sylvain Bauza wrote:

>>> 
>>
>>>
 Specifically, I want "nova boot" to get me a VM with an IP address. I
 don't want it to do fancy orchestration - I want it to not need fancy
 orchestration, because needing fancy orchestration to get a VM  on a
 network is not a feature.

>>>
>>> In the networking case there is a minimum of orchestration because the
>>> time required to allocate a port is small.  What has been requiring
>>> orchestration is the creation of volumes because of the requirement of
>>> Cinder to download an image, or be on a backend that support fast
>>> cloning and rely on a cache hit.  So the question under discussion is
>>> when booting an instance relies on another service performing a long
>>> running operation where is a good place to handle that.
>>>
>>> My thinking for a while has been that we could use another API that
>>> could manage those things.  And be the central place you're looking for
>>> to pass a simple "nova boot" with whatever options are required so you
>>> don't have to manage the complexities of calls to
>>> Neutron/Cinder/Nova(current API).  What's become clear to me from this
>>> thread is that people don't seem to oppose that idea, however they don't
>>> want their users/clients to need to switch what API they're currently
>>> using(Nova).
>>>
>>> The right way to proceed with this idea seems to be to by evolving the
>>> Nova API and potentially creating a split down the road.  And by split I
>>> more mean architectural within Nova, and not necessarily a split API.
>>> What I imagine is that we follow the model of git and have a plumbing
>>> and porcelain API and each can focus on doing the right things.
>>>
>>
>> Right, and I think that's a fine approach. Nova's job is "give me a
>> working VM". Working includes networking, persistent storage. The API
>> semantics for "give me a working VM" should exist in Nova.
>>
>> It is also fine if there are lower level calls that tweak parts of that,
>> but nova boot shouldn't have to be a multi step API process for the
>> user. Building one working VM you can do something with is really the
>> entire point of Nova.
>>
>
> What I'm struggling with is where do we draw the line in this model?  For
> instance we don't allow a user to boot an instance from a disk image on
> their local machine via the Nova API, that is a multi step process.  And
> which parameters do we expose that can influence network and volume
> creation, if not all of them?  It would be helpful to establish guidelines
> on what is a good candidate for inclusion in Nova.
>
> I see a clear line between something that handles the creation of all
> ancillary resources needed to boot a VM and then the creation of the VM
> itself.  I don't understand why the creation of the other resources should
> live within Nova but as long as we can get to a good split between
> responsibilities that's a secondary concern.
>
>
>
>> -Sean
>>
>> --
>> Sean Dague
>> http://dague.net
>>
>> __
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe:
>> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
>

Re: [openstack-dev] Compute API (Was Re: [nova][cinder] how to handle AZ bug 1496235?)

2015-09-28 Thread Sylvain Bauza 



Le 28/09/2015 12:35, Duncan Thomas a écrit :



On 28 September 2015 at 12:35, Sylvain Bauza > wrote:


About the maintenance burden, I also consider that patching
clients is far more easier than patching an API unless I missed
something.


I think I very much disagree there - patching a central installation 
is much, much easier than getting N customers to patch M different 
libraries, even assuming the fix is available for any significant 
subset of the M libraries, plus making sure that new customers use the 
correct libraries, plus helping any customers who have some sort of 
roll-your-own library do the new right thing...




Well, having N versions of clients against one single API version is 
just something we manage since the beginning. I don't really see why it 
suddently becomes so difficult to manage it.



I think there's a definite place for a simple API to do infrastructure 
level orchestration without needing the complexities of heat - these 
APIs are in nova because they're useful - there's clear operator 
desire for them and a couple of operators have been quite vocal about 
their desire for them not to be removed. Great, let's keep them, but 
form a team of people interested in getting them right (get rid of 
fixed timeouts, etc), add any missing pieces (like floating IPs for 
new VMs) and generally focus on getting this piece of the puzzle 
right. Breaking another small piece off nova and polishing it has been 
a generally successful pattern.


I don't want to overthink what could be the right scope of that future 
API but given the Heat mission statement [1] and its service name 
'orchestration', I don't see why this API endpoint should land in the 
Nova codebase and couldn't be rather provided by the Heat API. Oh sure, 
it would perhaps require another endpoint behind the same service, but 
isn't that better than having another endpoint in Nova ?


[1] 
https://github.com/openstack/governance/blob/master/reference/projects.yaml#L482-L484





I remember Monty Taylor (copied) having a rant about the lack of the 
perfect 'give me a VM with all its stuff sorted' API. Care to comment, 
Monty?


Sounds you misunderstood me. I'm not against implementing this excellent 
usecase, I just think the best place is not in Nova and should be done 
elsewhere.





__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] Compute API (Was Re: [nova][cinder] how to handle AZ bug 1496235?)

2015-09-28 Thread Monty Taylor 

On 09/28/2015 07:58 AM, Sylvain Bauza wrote:



Le 28/09/2015 12:35, Duncan Thomas a écrit :



On 28 September 2015 at 12:35, Sylvain Bauza > wrote:

About the maintenance burden, I also consider that patching
clients is far more easier than patching an API unless I missed
something.


I think I very much disagree there - patching a central installation
is much, much easier than getting N customers to patch M different
libraries, even assuming the fix is available for any significant
subset of the M libraries, plus making sure that new customers use the
correct libraries, plus helping any customers who have some sort of
roll-your-own library do the new right thing...



Well, having N versions of clients against one single API version is
just something we manage since the beginning. I don't really see why it
suddently becomes so difficult to manage it.



I think there's a definite place for a simple API to do infrastructure
level orchestration without needing the complexities of heat - these
APIs are in nova because they're useful - there's clear operator
desire for them and a couple of operators have been quite vocal about
their desire for them not to be removed. Great, let's keep them, but
form a team of people interested in getting them right (get rid of
fixed timeouts, etc), add any missing pieces (like floating IPs for
new VMs) and generally focus on getting this piece of the puzzle
right. Breaking another small piece off nova and polishing it has been
a generally successful pattern.


I don't want to overthink what could be the right scope of that future
API but given the Heat mission statement [1] and its service name
'orchestration', I don't see why this API endpoint should land in the
Nova codebase and couldn't be rather provided by the Heat API. Oh sure,
it would perhaps require another endpoint behind the same service, but
isn't that better than having another endpoint in Nova ?

[1]
https://github.com/openstack/governance/blob/master/reference/projects.yaml#L482-L484




I remember Monty Taylor (copied) having a rant about the lack of the
perfect 'give me a VM with all its stuff sorted' API. Care to comment,
Monty?


Sounds you misunderstood me. I'm not against implementing this excellent
usecase, I just think the best place is not in Nova and should be done
elsewhere.



Specifically, I want "nova boot" to get me a VM with an IP address. I 
don't want it to do fancy orchestration - I want it to not need fancy 
orchestration, because needing fancy orchestration to get a VM  on a 
network is not a feature.


I also VERY MUCH do not want to need Heat to get a VM. I want to use 
Heat to do something complex. Getting a VM is not complex. It should not 
be complex. What it's complex and to the level of needing Heat, we've 
failed somewhere else.


Also, people should stop deploying clouds that require people to use 
floating IPs to get basic internet access. It's a misuse of the construct.


Public Network "ext-net" -> shared / directly attachable
Per-tenant Network "private" -> private network, not shared, not routable

If the user chooses, a router can be added with gateway set to ext-net.

This way:

nova boot --network=ext-net  -> vm dhcp'd on the public network
nova boot --network=private  -> vm dhcp'd on the private network
nova floating-ip-attach  -> vm gets a floating ip attached to their 
vm from the ext-net network


All of the use cases are handled, basic things are easy (boot a vm on 
the network works in one step) and for the 5% of cases where a floating 
IP is actually needed (a long-lived service on a single vm that wants to 
keep the IP and not just a DNS name across VM migrations and isn't using 
a load-balancer) can use that.


This is, btw, the most common public cloud deployment model.

Let's stop making things harder than they need to be and serve our users.

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] Compute API (Was Re: [nova][cinder] how to handle AZ bug 1496235?)

2015-09-26 Thread Duncan Thomas 
On 26 September 2015 at 07:04, Joshua Harlow  wrote:

> Maybe there should be a 'warm' project or something ;)
>
> Or we can call it 'bbs' for 'building block service' (obviously not
> bulletin board system); ask said service to build a set of blocks into well
> defined structures and let it figure out how to make that happen...
>
>
I was looking for a pun on herding cats for the name, but I've not yet come
up with one.

More seriously though, I don't see this as directly heat - heat is a
project with much  greater scope. What I'm thinking of is, like
cinder/neutron, just breaking out a small piece of the nova API, cleaning
it up and adding any obviously missing features. It is a project that could
easily be ruined by scope creep.
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] Compute API (Was Re: [nova][cinder] how to handle AZ bug 1496235?)

2015-09-25 Thread Duncan Thomas 
I think there's a place for yet another service breakout from nova - some
sort of like-weight platform orchestration piece, nothing as complicated or
complete as heat, nothing that touches the inside of a VM, just something
that can talk to cinder, nova and neutron (plus I guess ironic and whatever
the container thing is called) and work through long running /
cross-project tasks. I'd probably expect it to provide a task style
interface, e.g. a boot-from-new-volume call returns a request-id that can
then be polled for detailed status.

The existing nova API for this (and any other nova APIs where this makes
sense) can then become a proxy for the new service, so that tenants are not
affected. The nova apis can then be deprecated in slow time.

Anybody else think this could be useful?

On 25 September 2015 at 17:12, Andrew Laski  wrote:

> On 09/24/15 at 03:13pm, James Penick wrote:
>
>>
>>>
>>> At risk of getting too offtopic I think there's an alternate solution to
>>> doing this in Nova or on the client side.  I think we're missing some
>>> sort
>>> of OpenStack API and service that can handle this.  Nova is a low level
>>> infrastructure API and service, it is not designed to handle these
>>> orchestrations.  I haven't checked in on Heat in a while but perhaps this
>>> is a role that it could fill.
>>>
>>> I think that too many people consider Nova to be *the* OpenStack API when
>>> considering instances/volumes/networking/images and that's not something
>>> I
>>> would like to see continue.  Or at the very least I would like to see a
>>> split between the orchestration/proxy pieces and the "manage my
>>> VM/container/baremetal" bits
>>>
>>
>>
>> (new thread)
>> You've hit on one of my biggest issues right now: As far as many deployers
>> and consumers are concerned (and definitely what I tell my users within
>> Yahoo): The value of an OpenStack value-stream (compute, network, storage)
>> is to provide a single consistent API for abstracting and managing those
>> infrastructure resources.
>>
>> Take networking: I can manage Firewalls, switches, IP selection, SDN, etc
>> through Neutron. But for compute, If I want VM I go through Nova, for
>> Baremetal I can -mostly- go through Nova, and for containers I would talk
>> to Magnum or use something like the nova docker driver.
>>
>> This means that, by default, Nova -is- the closest thing to a top level
>> abstraction layer for compute. But if that is explicitly against Nova's
>> charter, and Nova isn't going to be the top level abstraction for all
>> things Compute, then something else needs to fill that space. When that
>> happens, all things common to compute provisioning should come out of Nova
>> and move into that new API. Availability zones, Quota, etc.
>>
>
> I do think Nova is the top level abstraction layer for compute.  My issue
> is when Nova is asked to manage other resources.  There's no API call to
> tell Cinder "create a volume and attach it to this instance, and create
> that instance if it doesn't exist."  And I'm not sure why the reverse isn't
> true.
>
> I want Nova to be the absolute best API for managing compute resources.
> It's when someone is managing compute and volumes and networks together
> that I don't feel that Nova is the best place for that.  Most importantly
> right now it seems that not everyone is on the same page on this and I
> think it would be beneficial to come together and figure out what sort of
> workloads the Nova API is intending to provide.
>
>
>
>> -James
>>
>
> __
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe:
>> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>



-- 
-- 
Duncan Thomas
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] Compute API (Was Re: [nova][cinder] how to handle AZ bug 1496235?)

2015-09-25 Thread Joshua Harlow 

+1 from me, although I thought heat was supposed to be this thing?

Maybe there should be a 'warm' project or something ;)

Or we can call it 'bbs' for 'building block service' (obviously not 
bulletin board system); ask said service to build a set of blocks into 
well defined structures and let it figure out how to make that happen...


This though most definitely requires cross-project agreement though so 
I'd hope we can reach that somehow (before creating a halfway done new 
orchestration thing that is halfway integrated with a bunch of other 
apis that do one quarter of the work in ten different ways).


Duncan Thomas wrote:

I think there's a place for yet another service breakout from nova -
some sort of like-weight platform orchestration piece, nothing as
complicated or complete as heat, nothing that touches the inside of a
VM, just something that can talk to cinder, nova and neutron (plus I
guess ironic and whatever the container thing is called) and work
through long running / cross-project tasks. I'd probably expect it to
provide a task style interface, e.g. a boot-from-new-volume call returns
a request-id that can then be polled for detailed status.

The existing nova API for this (and any other nova APIs where this makes
sense) can then become a proxy for the new service, so that tenants are
not affected. The nova apis can then be deprecated in slow time.

Anybody else think this could be useful?

On 25 September 2015 at 17:12, Andrew Laski > wrote:

On 09/24/15 at 03:13pm, James Penick wrote:



At risk of getting too offtopic I think there's an alternate
solution to
doing this in Nova or on the client side.  I think we're
missing some sort
of OpenStack API and service that can handle this.  Nova is
a low level
infrastructure API and service, it is not designed to handle
these
orchestrations.  I haven't checked in on Heat in a while but
perhaps this
is a role that it could fill.

I think that too many people consider Nova to be *the*
OpenStack API when
considering instances/volumes/networking/images and that's
not something I
would like to see continue.  Or at the very least I would
like to see a
split between the orchestration/proxy pieces and the "manage my
VM/container/baremetal" bits



(new thread)
You've hit on one of my biggest issues right now: As far as many
deployers
and consumers are concerned (and definitely what I tell my users
within
Yahoo): The value of an OpenStack value-stream (compute,
network, storage)
is to provide a single consistent API for abstracting and
managing those
infrastructure resources.

Take networking: I can manage Firewalls, switches, IP selection,
SDN, etc
through Neutron. But for compute, If I want VM I go through
Nova, for
Baremetal I can -mostly- go through Nova, and for containers I
would talk
to Magnum or use something like the nova docker driver.

This means that, by default, Nova -is- the closest thing to a
top level
abstraction layer for compute. But if that is explicitly against
Nova's
charter, and Nova isn't going to be the top level abstraction
for all
things Compute, then something else needs to fill that space.
When that
happens, all things common to compute provisioning should come
out of Nova
and move into that new API. Availability zones, Quota, etc.


I do think Nova is the top level abstraction layer for compute.  My
issue is when Nova is asked to manage other resources.  There's no
API call to tell Cinder "create a volume and attach it to this
instance, and create that instance if it doesn't exist."  And I'm
not sure why the reverse isn't true.

I want Nova to be the absolute best API for managing compute
resources.  It's when someone is managing compute and volumes and
networks together that I don't feel that Nova is the best place for
that.  Most importantly right now it seems that not everyone is on
the same page on this and I think it would be beneficial to come
together and figure out what sort of workloads the Nova API is
intending to provide.



-James



__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe:
openstack-dev-requ...@lists.openstack.org?subject:unsubscribe

http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] Compute API (Was Re: [nova][cinder] how to handle AZ bug 1496235?)

2015-09-25 Thread Sylvain Bauza 



Le 25/09/2015 00:13, James Penick a écrit :



At risk of getting too offtopic I think there's an alternate
solution to doing this in Nova or on the client side.  I think
we're missing some sort of OpenStack API and service that can
handle this.  Nova is a low level infrastructure API and service,
it is not designed to handle these orchestrations.  I haven't
checked in on Heat in a while but perhaps this is a role that it
could fill.

I think that too many people consider Nova to be *the* OpenStack
API when considering instances/volumes/networking/images and
that's not something I would like to see continue.  Or at the very
least I would like to see a split between the orchestration/proxy
pieces and the "manage my VM/container/baremetal" bits


(new thread)
 You've hit on one of my biggest issues right now: As far as many 
deployers and consumers are concerned (and definitely what I tell my 
users within Yahoo): The value of an OpenStack value-stream (compute, 
network, storage) is to provide a single consistent API for 
abstracting and managing those infrastructure resources.


 Take networking: I can manage Firewalls, switches, IP selection, SDN, 
etc through Neutron. But for compute, If I want VM I go through Nova, 
for Baremetal I can -mostly- go through Nova, and for containers I 
would talk to Magnum or use something like the nova docker driver.


 This means that, by default, Nova -is- the closest thing to a top 
level abstraction layer for compute. But if that is explicitly against 
Nova's charter, and Nova isn't going to be the top level abstraction 
for all things Compute, then something else needs to fill that space. 
When that happens, all things common to compute provisioning should 
come out of Nova and move into that new API. Availability zones, 
Quota, etc.




There is an old story that I would like to see where a nova boot could 
have some affinity with volumes and networks. That means that Neutron 
and Cinder could provide some resources to the nova scheduler (or nova 
could call the projects) so that we could use either filters (for hard 
limits) or weighers (for soft limits) in order to say "eh, Nova, please 
create me an instance with that flavor and that image close to this 
volume or this network".


That said, we still have lots of work to do in Nova to help those 
projects giving resources and we agreed to first work on the scheduler 
interfaces (for providing resources and for getting a destination) 
before working on cross-project resources. That's still an on-going work 
but we hope to land the new interfaces by Mitaka.


Not sure if we could have some discussion at Tokyo between Cinder, 
Neutron and Nova about how to provide resources to the nova scheduler 
given we haven't yet finished the interface reworking, but we could at 
least get feedback from the Neutron and Cinder teams about what kind of 
resources they'd like to provide so that an user could ask for.


Thoughts on that ?
-Sylvain


-James


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] Compute API (Was Re: [nova][cinder] how to handle AZ bug 1496235?)

2015-09-25 Thread Andrew Laski 

On 09/24/15 at 03:13pm, James Penick wrote:



At risk of getting too offtopic I think there's an alternate solution to
doing this in Nova or on the client side.  I think we're missing some sort
of OpenStack API and service that can handle this.  Nova is a low level
infrastructure API and service, it is not designed to handle these
orchestrations.  I haven't checked in on Heat in a while but perhaps this
is a role that it could fill.

I think that too many people consider Nova to be *the* OpenStack API when
considering instances/volumes/networking/images and that's not something I
would like to see continue.  Or at the very least I would like to see a
split between the orchestration/proxy pieces and the "manage my
VM/container/baremetal" bits



(new thread)
You've hit on one of my biggest issues right now: As far as many deployers
and consumers are concerned (and definitely what I tell my users within
Yahoo): The value of an OpenStack value-stream (compute, network, storage)
is to provide a single consistent API for abstracting and managing those
infrastructure resources.

Take networking: I can manage Firewalls, switches, IP selection, SDN, etc
through Neutron. But for compute, If I want VM I go through Nova, for
Baremetal I can -mostly- go through Nova, and for containers I would talk
to Magnum or use something like the nova docker driver.

This means that, by default, Nova -is- the closest thing to a top level
abstraction layer for compute. But if that is explicitly against Nova's
charter, and Nova isn't going to be the top level abstraction for all
things Compute, then something else needs to fill that space. When that
happens, all things common to compute provisioning should come out of Nova
and move into that new API. Availability zones, Quota, etc.


I do think Nova is the top level abstraction layer for compute.  My 
issue is when Nova is asked to manage other resources.  There's no API 
call to tell Cinder "create a volume and attach it to this instance, and 
create that instance if it doesn't exist."  And I'm not sure why the 
reverse isn't true.


I want Nova to be the absolute best API for managing compute resources.  
It's when someone is managing compute and volumes and networks together 
that I don't feel that Nova is the best place for that.  Most 
importantly right now it seems that not everyone is on the same page on 
this and I think it would be beneficial to come together and figure out 
what sort of workloads the Nova API is intending to provide.




-James



__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] Compute API (Was Re: [nova][cinder] how to handle AZ bug 1496235?)

2015-09-24 Thread James Penick 
>
>
> At risk of getting too offtopic I think there's an alternate solution to
> doing this in Nova or on the client side.  I think we're missing some sort
> of OpenStack API and service that can handle this.  Nova is a low level
> infrastructure API and service, it is not designed to handle these
> orchestrations.  I haven't checked in on Heat in a while but perhaps this
> is a role that it could fill.
>
> I think that too many people consider Nova to be *the* OpenStack API when
> considering instances/volumes/networking/images and that's not something I
> would like to see continue.  Or at the very least I would like to see a
> split between the orchestration/proxy pieces and the "manage my
> VM/container/baremetal" bits


(new thread)
 You've hit on one of my biggest issues right now: As far as many deployers
and consumers are concerned (and definitely what I tell my users within
Yahoo): The value of an OpenStack value-stream (compute, network, storage)
is to provide a single consistent API for abstracting and managing those
infrastructure resources.

 Take networking: I can manage Firewalls, switches, IP selection, SDN, etc
through Neutron. But for compute, If I want VM I go through Nova, for
Baremetal I can -mostly- go through Nova, and for containers I would talk
to Magnum or use something like the nova docker driver.

 This means that, by default, Nova -is- the closest thing to a top level
abstraction layer for compute. But if that is explicitly against Nova's
charter, and Nova isn't going to be the top level abstraction for all
things Compute, then something else needs to fill that space. When that
happens, all things common to compute provisioning should come out of Nova
and move into that new API. Availability zones, Quota, etc.

-James
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev