Re: [Openstack] Providing packages for stable releases of OpenStack

2011-12-07 Thread Mark McLoughlin
On Tue, 2011-12-06 at 14:12 -0800, Duncan McGreggor wrote:
 On 06 Dec 2011 - 13:52, Duncan McGreggor wrote:
  On 06 Dec 2011 - 21:14, Thierry Carrez wrote:
   Tim Bell wrote:
I'm not clear on who will be maintaining the stable/diablo  branch.
The people such as EPEL for RedHat systems need to have something
with the appropriate bug fixes back ported.
   
There are an increasing number of sites looking to deploy in
production and cannot follow the latest development version.
  
   Agreed on the need, we discussed this at length during the design
   summit. The stable branches have been established and are maintained by
   the OpenStack Stable Branch Maintainers team. Currently this team is
   mostly made of distribution members (Ubuntu and Fedora/RedHat, mostly)
   collaborating on a single branch to avoid duplication of effort.
  
   See:
   https://launchpad.net/~openstack-stable-maint
   http://wiki.openstack.org/StableBranch
 
  Okay, I think this mostly addresses item #4 that I wanted to add to your
  summary, Thierry.
 
  I do have the following minor concerns, though:
 
   * that wiki page's summary (intro sentence) only specifically mentions
 Diablo; I'd like to see something along the lines of currently
 focusing on Diablo. If these processes evolve into a successful
 model, they will be applied to all future releases.

Added.

   * the discussion on the page treats this as an experiment (this is
 good!), but I'd like to see a phrase alone the lines of if this
 experiment is successful, we will do X to ensure these processes
 become an official part of the workflow.

Cleaned up the this is an experiment text a bit, it's gone beyond an
experiment now I think.

  These are tiny things, but I think they will better set expectations and
  give more warm fuzzies to organizations thinking about deploying
  OpenStack in production environments, seeing that we're considering the
  long-term (given success of the experiment).
 
  In addition, I would like to emphasize Tim's point from earlier, though:
  it's not just packaging...

Note that the stable-maint team has no involvement with upstream
packaging, if that's what you're talking about.

Cheers,
Mark.


___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp


Re: [Openstack] Providing packages for stable releases of OpenStack

2011-12-07 Thread Mark McLoughlin
On Tue, 2011-12-06 at 10:11 -0800, Duncan McGreggor wrote:
 On 06 Dec 2011 - 14:28, Thierry Carrez wrote:
  So the general consensus so far on this discussion seems to be:
 
  (0) The 2011.3 release PPA bears false expectations and should be
  removed now. In the future, we should not provide such PPAs: 0-day
  packages for the release should be available from the last milestone
  PPA anyway.
 
  (1) OpenStack, as an upstream project, should focus on development
  rather than on providing a production-ready distribution.
 
  (2) We could provide daily builds from the stable/diablo branch for a
  variety of releases (much like what we do for the master branch), but
  those should be clearly marked not for production use and be
  best-effort only (like our master branch builds).
 
  (3) This should not prevent a group in the community from working on a
  project providing an openstack on Lucid production-ready distribution
  if they so wishes. This project would just be another distribution of
  OpenStack.
 
 This doesn't seem like enough to me. OpenStack isn't just a library;
 it's a fairly substantial collection of software and services, intended
 to be used as a product. If it can't be used as a product, what's the
 use?
 
 Someone made the suggestion that a new OpenStack group be started, one
 whose focus is on producing a production-ready, distribution-ready,
 release of the software. So can we add one more (need some help with
 wording, here...):

This paragraph makes it sound like you think OpenStack upstream should
produce production-ready binary packages? This is what we've agreed not
to do, at least until some group of volunteers show that they can
sustain the work involved.

 (4) OpenStack will accept and foster a new project, one that is not
 focused on development, but rather the distribution and it's general
 stability. This distro project will be responsible for advocating on
 behalf of various operating systems/distros/sponsoring vendors for bugs
 that affect performance and stability of OpenStack, or prevent an
 operating system from running OpenStack.

This sounds like you want want OpenStack to focus more on the quality of
its releases. This is generally accepted and you can join the QA team to
help out:

  https://launchpad.net/~openstack-qa-team

It also sounds like you want more co-ordination upstream between
downstream distributions of OpenStack. I think there's already pretty
good co-ordination. Is there any specific problem you've seen on that
front?

Cheers,
Mark.


___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp


Re: [Openstack] Nova Subteam Changes

2011-12-07 Thread Mark McLoughlin
On Tue, 2011-12-06 at 19:54 -0800, Vishvananda Ishaya wrote:
 Hello Everyone,
 
 The Nova subteams have now been active for a month and a half.  Some
 things are going very well, and others could use a little improvement.
 To keep things moving forward, I'd like to make the following changes:
 
 1) Weekly meeting for team leads. [..]
 
 2) Closing down the team mailinglists. [..]
 
 3) Closing teams. Some of the teams haven't really started having much
 activity. [..]

Sounds good to me. In future, perhaps new teams should only be formed
when there are two or more people actively working on stuff and need to
co-ordinate?

It was a little troubling at the summit that some discussions ended with
agreement to start a team, but no obvious sense that anyone was going to
step up and actively do the work.

Cheers,
Mark.


___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp


Re: [Openstack] Nova Subteam Changes

2011-12-07 Thread Thierry Carrez
Vishvananda Ishaya wrote:

 2) *Closing down the team mailinglists.*  Some of the lists have been a
 bit active, but I think the approach that soren has been using of
 sending messages to the regular list with a team [header] is a better
 approach. Examples:
 [db] Should we use zookeeper?
 [scaling] Plans for bursting
 I realize that this will lead to a little more noise on some of the
 channels, but I think it makes sure that we don't isolate knowledge too much

Could we have a standardized list of headers ? Those would double as
tags for Nova bugs, so the scaling team could see a list of bugs in
their area by searching for the scaling tag.

 3) *Closing teams.* Some of the teams haven't really started having much
 activity.  I'm going to move these teams into a WANTED section on the
 teams page here:
 http://wiki.openstack.org/Teams
 For now, people can just discuss stuff on the main mailing list, but we
 won't target blueprints to those teams specifically. We can make the
 teams official again once they have some activity and a clear person to
 be responsible for running them.  Please keep in mind that I'm not
 saying that this stuff isn't important, just that there is no need to
 separate out if there isn't a lot of activity.
 Specifically the teams which haven't really been very active are:
 
  1. Nova Upgrades Team http://wiki.openstack.org/Teams#Nova_Upgrades_Team
  2. Nova Auth Team http://wiki.openstack.org/Teams#Nova_Auth_Team
  3. Nova Security Improvements Team
 http://wiki.openstack.org/Teams#Nova_Security_Improvements_Team

Agreed, so far it's mostly been individual efforts, nothing coordinated.

  4. Nova Operational Support Team
 http://wiki.openstack.org/Teams#Nova_Operational_Support_Team
  5. Nova EC2 API Team http://wiki.openstack.org/Teams#Nova_EC2_API_Team
 
 Im going to leave the ec2 team for now, because it is relatively new.
  If anyone feels that the other teams above should not be folded back
 in, please let me know.

The EC2 team has been meeting a few times already, and started triaging
and working on EC2 bugs. I think it belongs to the active category :)

Regards,

-- 
Thierry Carrez (ttx)
Release Manager, OpenStack

___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp


[Openstack] Openstack with XenServer

2011-12-07 Thread jeffrey coho
Hi,all,
   I am trying to install OpenStack with XenServer.I followed what this
page says,but with no luck.
(http://wiki.openstack.org/XenServerDevelopment)  Is there any one who
successfully installed this?
Is there any guides available beyond the page above?
  What's more, I tried this way:
https://github.com/cloudbuilders/devstack/tree/master/tools/xen
and it doesn't seem  to work either.Any thoughts? Thanks very much.

Yours,
Jeff
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp


Re: [Openstack] Providing packages for stable releases of OpenStack

2011-12-07 Thread John Garbutt
 On 06 Dec 2011 - 13:52, Duncan McGreggor wrote:
 Yikes! I forgot an incredibly important one:
  * What is the migration path story (diablo to essex, essex to f, etc.)?

I think it was going to be the Upgrades Team?

___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp


[Openstack] [Orchestration] Handling error events ... explicit vs. implicit

2011-12-07 Thread Sandy Walsh
For orchestration (and now the scheduler improvements) we need to know when an 
operation fails ... and specifically, which resource was involved. In the 
majority of the cases it's an instance_uuid we're looking for, but it could be 
a security group id or a reservation id.

With most of the compute.manager calls the resource id is the third parameter 
in the call (after self  context), but there are some oddities. And sometimes 
we need to know the additional parameters (like a migration id related to an 
instance uuid). So simply enforcing parameter orders may be insufficient and 
impossible to enforce programmatically.

A little background:

In nova, exceptions are generally handled in the RPC or middleware layers as a 
logged event and life goes on. In an attempt to tie this into the notification 
system, a while ago I added stuff to the wrap_exception decorator. I'm sure 
you've seen this nightmare scattered around the code:
@exception.wrap_exception(notifier=notifier, publisher_id=publisher_id())

What started as a simple decorator now takes parameters and the code has become 
nasty. 

But it works ... no matter where the exception was generated, the notifier gets:
*   compute.host_id
*   method name
*   and whatever arguments the method takes.

So, we know what operation failed and the host it failed on, but someone needs 
to crack the argument nut to get the goodies. It's a fragile coupling from 
publisher to receiver.

One, less fragile, alternative is to put a try/except block inside every 
top-level nova.compute.manager method and send meaningful exceptions right from 
the source. More fidelity, but messier code. Although explicit is better than 
implicit keeps ringing in my head.

Or, we make a general event parser that anyone can use ... but again, the link 
between the actual method and the parser is fragile. The developers have to 
remember to update both. 

Opinions?

-S








___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp


Re: [Openstack] [Orchestration] Handling error events ... explicit vs. implicit

2011-12-07 Thread Mark Washenberger
Can you talk a little more about how you want to apply this failure 
notification? That is, what is the case where you are going to use the 
information that an operation failed? In my head I have an idea of getting code 
simplicity dividends from an everything succeeds approach to some of our 
operations. But it might not really apply to the case you're working on.

Sandy Walsh sandy.wa...@rackspace.com said:

 For orchestration (and now the scheduler improvements) we need to know when an
 operation fails ... and specifically, which resource was involved. In the 
 majority
 of the cases it's an instance_uuid we're looking for, but it could be a 
 security
 group id or a reservation id.
 
 With most of the compute.manager calls the resource id is the third parameter 
 in
 the call (after self  context), but there are some oddities. And sometimes we
 need to know the additional parameters (like a migration id related to an 
 instance
 uuid). So simply enforcing parameter orders may be insufficient and 
 impossible to
 enforce programmatically.
 
 A little background:
 
 In nova, exceptions are generally handled in the RPC or middleware layers as a
 logged event and life goes on. In an attempt to tie this into the notification
 system, a while ago I added stuff to the wrap_exception decorator. I'm sure 
 you've
 seen this nightmare scattered around the code:
 @exception.wrap_exception(notifier=notifier, publisher_id=publisher_id())
 
 What started as a simple decorator now takes parameters and the code has 
 become
 nasty.
 
 But it works ... no matter where the exception was generated, the notifier 
 gets:
 *   compute.host_id
 *   method name
 *   and whatever arguments the method takes.
 
 So, we know what operation failed and the host it failed on, but someone 
 needs to
 crack the argument nut to get the goodies. It's a fragile coupling from 
 publisher
 to receiver.
 
 One, less fragile, alternative is to put a try/except block inside every 
 top-level
 nova.compute.manager method and send meaningful exceptions right from the 
 source.
 More fidelity, but messier code. Although explicit is better than implicit 
 keeps
 ringing in my head.
 
 Or, we make a general event parser that anyone can use ... but again, the link
 between the actual method and the parser is fragile. The developers have to
 remember to update both.
 
 Opinions?
 
 -S
 
 
 
 
 
 
 
 
 ___
 Mailing list: https://launchpad.net/~openstack
 Post to : openstack@lists.launchpad.net
 Unsubscribe : https://launchpad.net/~openstack
 More help   : https://help.launchpad.net/ListHelp
 



___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp


Re: [Openstack] [Orchestration] Handling error events ... explicit vs. implicit

2011-12-07 Thread Sandy Walsh
Sure, the problem I'm immediately facing is reclaiming resources from the 
Capacity table when something fails. (we claim them immediately in the 
scheduler when the host is selected to lessen the latency).

The other situation is Orchestration needs it for retries, rescheduling, 
rollbacks and cross-service timeouts.

I think it's needed core functionality. I like Fail-Fast for the same reasons, 
but it can get in the way.

-S


From: openstack-bounces+sandy.walsh=rackspace@lists.launchpad.net 
[openstack-bounces+sandy.walsh=rackspace@lists.launchpad.net] on behalf of 
Mark Washenberger [mark.washenber...@rackspace.com]
Sent: Wednesday, December 07, 2011 11:53 AM
To: openstack@lists.launchpad.net
Subject: Re: [Openstack] [Orchestration] Handling error events ... explicit 
vs. implicit

Can you talk a little more about how you want to apply this failure 
notification? That is, what is the case where you are going to use the 
information that an operation failed? In my head I have an idea of getting code 
simplicity dividends from an everything succeeds approach to some of our 
operations. But it might not really apply to the case you're working on.

___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp


[Openstack] [QA] Team meeting at 12 EST today

2011-12-07 Thread Jay Pipes
Hey all,

A quick reminder that the QA team has our weekly meeting on
#openstack-meeting in about 30 minutes.

12:00 EST
09:00 PST
17:00 UTC

See you there,
-jay

___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp


Re: [Openstack] [Orchestration] Handling error events ... explicit vs. implicit

2011-12-07 Thread Mark Washenberger
Gotcha.

So the way this might work is, for example, when a run_instance fails on 
compute node, it would publish a run_instance for uuid=blah failed event. 
There would be a subscriber associated with the scheduler listening for such 
events--when it receives one it would go check the capacity table and update it 
to reflect the failure. Does that sound about right?

Sandy Walsh sandy.wa...@rackspace.com said:

 Sure, the problem I'm immediately facing is reclaiming resources from the 
 Capacity
 table when something fails. (we claim them immediately in the scheduler when 
 the
 host is selected to lessen the latency).
 
 The other situation is Orchestration needs it for retries, rescheduling, 
 rollbacks
 and cross-service timeouts.
 
 I think it's needed core functionality. I like Fail-Fast for the same 
 reasons, but
 it can get in the way.
 
 -S
 
 
 From: openstack-bounces+sandy.walsh=rackspace@lists.launchpad.net
 [openstack-bounces+sandy.walsh=rackspace@lists.launchpad.net] on behalf of
 Mark Washenberger [mark.washenber...@rackspace.com]
 Sent: Wednesday, December 07, 2011 11:53 AM
 To: openstack@lists.launchpad.net
 Subject: Re: [Openstack] [Orchestration] Handling error events ... explicit   
  
 vs. implicit
 
 Can you talk a little more about how you want to apply this failure 
 notification?
 That is, what is the case where you are going to use the information that an
 operation failed? In my head I have an idea of getting code simplicity 
 dividends
 from an everything succeeds approach to some of our operations. But it 
 might not
 really apply to the case you're working on.
 



___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp


Re: [Openstack] [Orchestration] Handling error events ... explicit vs. implicit

2011-12-07 Thread Sandy Walsh
Exactly! ... or it could be handled in the notifier itself.


From: openstack-bounces+sandy.walsh=rackspace@lists.launchpad.net 
[openstack-bounces+sandy.walsh=rackspace@lists.launchpad.net] on behalf of 
Mark Washenberger [mark.washenber...@rackspace.com]
Sent: Wednesday, December 07, 2011 12:36 PM
To: openstack@lists.launchpad.net
Subject: Re: [Openstack] [Orchestration] Handling error events ... explicit 
vs. implicit

Gotcha.

So the way this might work is, for example, when a run_instance fails on 
compute node, it would publish a run_instance for uuid=blah failed event. 
There would be a subscriber associated with the scheduler listening for such 
events--when it receives one it would go check the capacity table and update it 
to reflect the failure. Does that sound about right?


___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp


Re: [Openstack] [Orchestration] Handling error events ... explicit vs. implicit

2011-12-07 Thread Yun Mao
Hi Sandy,

I'm wondering if it is possible to change the scheduler's rpc cast to
rpc call. This way the exceptions should be magically propagated back
to the scheduler, right? Naturally the scheduler can find another node
to retry or decide to give up and report failure. If we need to
provision many instances, we can spawn a few green threads for that.

Yun

On Wed, Dec 7, 2011 at 10:26 AM, Sandy Walsh sandy.wa...@rackspace.com wrote:
 For orchestration (and now the scheduler improvements) we need to know when 
 an operation fails ... and specifically, which resource was involved. In the 
 majority of the cases it's an instance_uuid we're looking for, but it could 
 be a security group id or a reservation id.

 With most of the compute.manager calls the resource id is the third parameter 
 in the call (after self  context), but there are some oddities. And 
 sometimes we need to know the additional parameters (like a migration id 
 related to an instance uuid). So simply enforcing parameter orders may be 
 insufficient and impossible to enforce programmatically.

 A little background:

 In nova, exceptions are generally handled in the RPC or middleware layers as 
 a logged event and life goes on. In an attempt to tie this into the 
 notification system, a while ago I added stuff to the wrap_exception 
 decorator. I'm sure you've seen this nightmare scattered around the code:
 @exception.wrap_exception(notifier=notifier, publisher_id=publisher_id())

 What started as a simple decorator now takes parameters and the code has 
 become nasty.

 But it works ... no matter where the exception was generated, the notifier 
 gets:
 *   compute.host_id
 *   method name
 *   and whatever arguments the method takes.

 So, we know what operation failed and the host it failed on, but someone 
 needs to crack the argument nut to get the goodies. It's a fragile coupling 
 from publisher to receiver.

 One, less fragile, alternative is to put a try/except block inside every 
 top-level nova.compute.manager method and send meaningful exceptions right 
 from the source. More fidelity, but messier code. Although explicit is 
 better than implicit keeps ringing in my head.

 Or, we make a general event parser that anyone can use ... but again, the 
 link between the actual method and the parser is fragile. The developers have 
 to remember to update both.

 Opinions?

 -S








 ___
 Mailing list: https://launchpad.net/~openstack
 Post to     : openstack@lists.launchpad.net
 Unsubscribe : https://launchpad.net/~openstack
 More help   : https://help.launchpad.net/ListHelp

___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp


Re: [Openstack] [Orchestration] Handling error events ... explicit vs. implicit

2011-12-07 Thread Sandy Walsh
True ... this idea has come up before (and is still being kicked around). My 
biggest concern is what happens if that scheduler dies? We need a mechanism 
that can live outside of a single scheduler service. 

The more of these long-running processes we leave in a service the greater the 
impact when something fails. Shouldn't we let the queue provide the resiliency 
and not depend on the worker staying alive? Personally I'm not a fan of 
removing our synchronous nature.


From: Yun Mao [yun...@gmail.com]
Sent: Wednesday, December 07, 2011 1:03 PM
To: Sandy Walsh
Cc: openstack@lists.launchpad.net
Subject: Re: [Openstack] [Orchestration] Handling error events ... explicit vs. 
implicit

Hi Sandy,

I'm wondering if it is possible to change the scheduler's rpc cast to
rpc call. This way the exceptions should be magically propagated back
to the scheduler, right? Naturally the scheduler can find another node
to retry or decide to give up and report failure. If we need to
provision many instances, we can spawn a few green threads for that.

Yun

On Wed, Dec 7, 2011 at 10:26 AM, Sandy Walsh sandy.wa...@rackspace.com wrote:
 For orchestration (and now the scheduler improvements) we need to know when 
 an operation fails ... and specifically, which resource was involved. In the 
 majority of the cases it's an instance_uuid we're looking for, but it could 
 be a security group id or a reservation id.

 With most of the compute.manager calls the resource id is the third parameter 
 in the call (after self  context), but there are some oddities. And 
 sometimes we need to know the additional parameters (like a migration id 
 related to an instance uuid). So simply enforcing parameter orders may be 
 insufficient and impossible to enforce programmatically.

 A little background:

 In nova, exceptions are generally handled in the RPC or middleware layers as 
 a logged event and life goes on. In an attempt to tie this into the 
 notification system, a while ago I added stuff to the wrap_exception 
 decorator. I'm sure you've seen this nightmare scattered around the code:
 @exception.wrap_exception(notifier=notifier, publisher_id=publisher_id())

 What started as a simple decorator now takes parameters and the code has 
 become nasty.

 But it works ... no matter where the exception was generated, the notifier 
 gets:
 *   compute.host_id
 *   method name
 *   and whatever arguments the method takes.

 So, we know what operation failed and the host it failed on, but someone 
 needs to crack the argument nut to get the goodies. It's a fragile coupling 
 from publisher to receiver.

 One, less fragile, alternative is to put a try/except block inside every 
 top-level nova.compute.manager method and send meaningful exceptions right 
 from the source. More fidelity, but messier code. Although explicit is 
 better than implicit keeps ringing in my head.

 Or, we make a general event parser that anyone can use ... but again, the 
 link between the actual method and the parser is fragile. The developers have 
 to remember to update both.

 Opinions?

 -S








 ___
 Mailing list: https://launchpad.net/~openstack
 Post to : openstack@lists.launchpad.net
 Unsubscribe : https://launchpad.net/~openstack
 More help   : https://help.launchpad.net/ListHelp

___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp


Re: [Openstack] [Orchestration] Handling error events ... explicit vs. implicit

2011-12-07 Thread Sandy Walsh
*removing our Asynchronous nature.

(heh, such a key point to typo on)



From: openstack-bounces+sandy.walsh=rackspace@lists.launchpad.net 
[openstack-bounces+sandy.walsh=rackspace@lists.launchpad.net] on behalf of 
Sandy Walsh [sandy.wa...@rackspace.com]
Sent: Wednesday, December 07, 2011 1:55 PM
To: Yun Mao
Cc: openstack@lists.launchpad.net
Subject: Re: [Openstack] [Orchestration] Handling error events ... explicit vs. 
implicit

True ... this idea has come up before (and is still being kicked around). My 
biggest concern is what happens if that scheduler dies? We need a mechanism 
that can live outside of a single scheduler service.

The more of these long-running processes we leave in a service the greater the 
impact when something fails. Shouldn't we let the queue provide the resiliency 
and not depend on the worker staying alive? Personally I'm not a fan of 
removing our synchronous nature.


From: Yun Mao [yun...@gmail.com]
Sent: Wednesday, December 07, 2011 1:03 PM
To: Sandy Walsh
Cc: openstack@lists.launchpad.net
Subject: Re: [Openstack] [Orchestration] Handling error events ... explicit vs. 
implicit

Hi Sandy,

I'm wondering if it is possible to change the scheduler's rpc cast to
rpc call. This way the exceptions should be magically propagated back
to the scheduler, right? Naturally the scheduler can find another node
to retry or decide to give up and report failure. If we need to
provision many instances, we can spawn a few green threads for that.

Yun

On Wed, Dec 7, 2011 at 10:26 AM, Sandy Walsh sandy.wa...@rackspace.com wrote:
 For orchestration (and now the scheduler improvements) we need to know when 
 an operation fails ... and specifically, which resource was involved. In the 
 majority of the cases it's an instance_uuid we're looking for, but it could 
 be a security group id or a reservation id.

 With most of the compute.manager calls the resource id is the third parameter 
 in the call (after self  context), but there are some oddities. And 
 sometimes we need to know the additional parameters (like a migration id 
 related to an instance uuid). So simply enforcing parameter orders may be 
 insufficient and impossible to enforce programmatically.

 A little background:

 In nova, exceptions are generally handled in the RPC or middleware layers as 
 a logged event and life goes on. In an attempt to tie this into the 
 notification system, a while ago I added stuff to the wrap_exception 
 decorator. I'm sure you've seen this nightmare scattered around the code:
 @exception.wrap_exception(notifier=notifier, publisher_id=publisher_id())

 What started as a simple decorator now takes parameters and the code has 
 become nasty.

 But it works ... no matter where the exception was generated, the notifier 
 gets:
 *   compute.host_id
 *   method name
 *   and whatever arguments the method takes.

 So, we know what operation failed and the host it failed on, but someone 
 needs to crack the argument nut to get the goodies. It's a fragile coupling 
 from publisher to receiver.

 One, less fragile, alternative is to put a try/except block inside every 
 top-level nova.compute.manager method and send meaningful exceptions right 
 from the source. More fidelity, but messier code. Although explicit is 
 better than implicit keeps ringing in my head.

 Or, we make a general event parser that anyone can use ... but again, the 
 link between the actual method and the parser is fragile. The developers have 
 to remember to update both.

 Opinions?

 -S








 ___
 Mailing list: https://launchpad.net/~openstack
 Post to : openstack@lists.launchpad.net
 Unsubscribe : https://launchpad.net/~openstack
 More help   : https://help.launchpad.net/ListHelp

___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp


Re: [Openstack] Openstack with XenServer

2011-12-07 Thread Ewan Mellor
Hi Jeff,

Can you be more specific about what doesn't work?  There are lots of people 
using OpenStack with XenServer, including Citrix and Rackspace, so I can 
guarantee that it works!  The docs are lacking though, that's for certain.  
Where did you get stuck?

Thanks,

Ewan.

From: openstack-bounces+ewan.mellor=citrix@lists.launchpad.net 
[mailto:openstack-bounces+ewan.mellor=citrix@lists.launchpad.net] On Behalf 
Of jeffrey coho
Sent: Wednesday, December 07, 2011 4:32 AM
To: openstack@lists.launchpad.net
Subject: [Openstack] Openstack with XenServer

Hi,all,
   I am trying to install OpenStack with XenServer.I followed what this page 
says,but with no luck.
(http://wiki.openstack.org/XenServerDevelopment)  Is there any one who 
successfully installed this?
Is there any guides available beyond the page above?
  What's more, I tried this way: 
https://github.com/cloudbuilders/devstack/tree/master/tools/xen
and it doesn't seem  to work either.Any thoughts? Thanks very much.

Yours,
Jeff
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp


Re: [Openstack] Providing packages for stable releases of OpenStack

2011-12-07 Thread Stefano Maffulli
On Tue, 2011-12-06 at 23:56 +0100, Loic Dachary wrote:
 I think there is an opportunity to leverage the momentum that is
 growing in each distribution by creating an openstack team for them to
 meet. Maybe Stefano Maffulli has an idea about how to go in this
 direction. The IRC channel was a great idea and it could become more.
 
It seems that there is general consensus that OpenStack should be the
good upstream provider and ease the job of downstream distributions.
There is an informal team that is coordinating the packaging effort.
Would it help if this team became more formal? What would it mean for
the openstack-packaging team to be 'formally' recognized?

/stef


___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp


Re: [Openstack] [Orchestration] Handling error events ... explicit vs. implicit

2011-12-07 Thread Michael Pittaro
On Wed, Dec 7, 2011 at 7:26 AM, Sandy Walsh sandy.wa...@rackspace.com wrote:
 For orchestration (and now the scheduler improvements) we need to know when 
 an operation fails ... and specifically, which resource was involved. In the 
 majority of the cases it's an instance_uuid we're looking for, but it could 
 be a security group id or a reservation id.

 With most of the compute.manager calls the resource id is the third parameter 
 in the call (after self  context), but there are some oddities. And 
 sometimes we need to know the additional parameters (like a migration id 
 related to an instance uuid). So simply enforcing parameter orders may be 
 insufficient and impossible to enforce programmatically.

 A little background:

 In nova, exceptions are generally handled in the RPC or middleware layers as 
 a logged event and life goes on. In an attempt to tie this into the 
 notification system, a while ago I added stuff to the wrap_exception 
 decorator. I'm sure you've seen this nightmare scattered around the code:
 @exception.wrap_exception(notifier=notifier, publisher_id=publisher_id())

 What started as a simple decorator now takes parameters and the code has 
 become nasty.

 But it works ... no matter where the exception was generated, the notifier 
 gets:
 *   compute.host_id
 *   method name
 *   and whatever arguments the method takes.

 So, we know what operation failed and the host it failed on, but someone 
 needs to crack the argument nut to get the goodies. It's a fragile coupling 
 from publisher to receiver.

I'm just wondering if we can get the notification message down to
something more standardized, and avoid including the full argument
list.
That is one way to reduce the coupling.

What is the minimum information we need to know when a failure occurs ?
I think we have
operation
host it failed on,
instance_id,
migration_id (maybe)
reservation_id, (maybe)
security group id (maybe)

If we can avoid cracking open the remaining arguments, a list this
long might be manageable.


 One, less fragile, alternative is to put a try/except block inside every 
 top-level nova.compute.manager method and send meaningful exceptions right 
 from the source. More fidelity, but messier code. Although explicit is 
 better than implicit keeps ringing in my head.

I like explicit better than implicit, but I think we need to trigger
off any and all exceptions to make all of this reliable.

 Or, we make a general event parser that anyone can use ... but again, the 
 link between the actual method and the parser is fragile. The developers have 
 to remember to update both.

 Opinions?

 -S



___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp


Re: [Openstack] Providing packages for stable releases of OpenStack

2011-12-07 Thread Duncan McGreggor
On 07 Dec 2011 - 08:22, Mark McLoughlin wrote:
 On Tue, 2011-12-06 at 10:11 -0800, Duncan McGreggor wrote:
  On 06 Dec 2011 - 14:28, Thierry Carrez wrote:
   So the general consensus so far on this discussion seems to be:
  
   (0) The 2011.3 release PPA bears false expectations and should be
   removed now. In the future, we should not provide such PPAs: 0-day
   packages for the release should be available from the last milestone
   PPA anyway.
  
   (1) OpenStack, as an upstream project, should focus on development
   rather than on providing a production-ready distribution.
  
   (2) We could provide daily builds from the stable/diablo branch for a
   variety of releases (much like what we do for the master branch), but
   those should be clearly marked not for production use and be
   best-effort only (like our master branch builds).
  
   (3) This should not prevent a group in the community from working on a
   project providing an openstack on Lucid production-ready distribution
   if they so wishes. This project would just be another distribution of
   OpenStack.
 
  This doesn't seem like enough to me. OpenStack isn't just a library;
  it's a fairly substantial collection of software and services, intended
  to be used as a product. If it can't be used as a product, what's the
  use?
 
  Someone made the suggestion that a new OpenStack group be started, one
  whose focus is on producing a production-ready, distribution-ready,
  release of the software. So can we add one more (need some help with
  wording, here...):

 This paragraph makes it sound like you think OpenStack upstream should
 produce production-ready binary packages?

Nope, just production-ready code. Doing the many things necessary so
that:
 1) when someone wants to build a binary package, they have everything
that they need to do so;
 2) when the OpenStack components are installed from these packages,
they run well;
 3) when someone wants to read the documentation for that release, it's
available, up-to-date, and accurate;
 4) when the unit tests are run for a given release, they all pass on a
properly configured system (which is documented, supporting #3
above);
 5) when the next release is out, upgrading is a clear and
straight-forward process.

The idea is not that these are *all* missing; rather, it appears to me
(and others on this list) that the activities outlined above are not
well-coordinated and/or prioritized.

  (4) OpenStack will accept and foster a new project, one that is not
  focused on development, but rather the distribution and it's general
  stability. This distro project will be responsible for advocating on
  behalf of various operating systems/distros/sponsoring vendors for bugs
  that affect performance and stability of OpenStack, or prevent an
  operating system from running OpenStack.

 This sounds like you want want OpenStack to focus more on the quality of
 its releases.

Agreed, but it's not just about QA -- there is a lot involved here. With
lots of different teams. My thought is that if there was an actual point
of contact (person or team) that was responsible for coordinating on the
specific set of areas that were agreed to have the most impact on
creating a production-ready product, operators/users, vendors, etc.,
would have much more confidence in OpenStack as a dependable platform
that business-critical systems could put their trust in.

 It also sounds like you want more co-ordination upstream between
 downstream distributions of OpenStack. I think there's already pretty
 good co-ordination. Is there any specific problem you've seen on that
 front?

I think that if we had something in place like the 5 numbered points
above (and other points that have been mentioned in this thread), such
that an operator could quickly and easily get from 0 to production-ready
in a few short steps, OpenStack would be truly unstoppable. And an
obvious choice for both those with no budget and those with big budgets.

d

___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp


Re: [Openstack] Providing packages for stable releases of OpenStack

2011-12-07 Thread Loic Dachary
On 12/07/2011 10:32 PM, Stefano Maffulli wrote:
 On Tue, 2011-12-06 at 23:56 +0100, Loic Dachary wrote:
 I think there is an opportunity to leverage the momentum that is
 growing in each distribution by creating an openstack team for them to
 meet. Maybe Stefano Maffulli has an idea about how to go in this
 direction. The IRC channel was a great idea and it could become more.

 It seems that there is general consensus that OpenStack should be the
 good upstream provider and ease the job of downstream distributions.
 There is an informal team that is coordinating the packaging effort.
 Would it help if this team became more formal? What would it mean for
 the openstack-packaging team to be 'formally' recognized?
Thierry Carrez and Mark McLoughlin essentially asked the same question on IRC a 
few hours ago. You are on the same line ;-) I honestly don't know how to 
express myself more clearly than I did in my previous mails.

The Debian GNU/Linux packaging team is doing fine. Since you are all 
comfortable with the fact that it operates independently, there probably is no 
need to create tighter relationships. If you ever feel that this needs to be 
discussed again, let me know.

Cheers

attachment: loic.vcf

signature.asc
Description: OpenPGP digital signature
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp


[Openstack] Delete server spec

2011-12-07 Thread Nachi Ueno
Hi folks

I wanna make Delete server spec clear.

The API doc says,
When a server is deleted, all images created from that server are also removed

http://docs.openstack.org/api/openstack-compute/1.1/content/Delete_Server-d1e2883.html

IMO, all images is vm images which stored on compute node instances_path.
Is this correct?

Cheers
Nati

___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp


Re: [Openstack] Delete server spec

2011-12-07 Thread Jesse Andrews
I would interpret that to include the snapshots - but I'm not sure
that is what I'd expect as a user.

On Wed, Dec 7, 2011 at 5:05 PM, Nachi Ueno
ueno.na...@nttdata-agilenet.com wrote:
 Hi folks

 I wanna make Delete server spec clear.

 The API doc says,
 When a server is deleted, all images created from that server are also 
 removed

 http://docs.openstack.org/api/openstack-compute/1.1/content/Delete_Server-d1e2883.html

 IMO, all images is vm images which stored on compute node instances_path.
 Is this correct?

 Cheers
 Nati

 ___
 Mailing list: https://launchpad.net/~openstack
 Post to     : openstack@lists.launchpad.net
 Unsubscribe : https://launchpad.net/~openstack
 More help   : https://help.launchpad.net/ListHelp

___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp


Re: [Openstack] Delete server spec

2011-12-07 Thread Nachi Ueno
Hi Jessy

Thanks.
Hmm, there are no implementation of cleanup snapshot images.
IMO, Snapshot image should not deleted, in case of API request mistake.

https://github.com/openstack/nova/blob/master/nova/compute/manager.py#L540

2011/12/7 Jesse Andrews anotherje...@gmail.com:
 I would interpret that to include the snapshots - but I'm not sure
 that is what I'd expect as a user.

 On Wed, Dec 7, 2011 at 5:05 PM, Nachi Ueno
 ueno.na...@nttdata-agilenet.com wrote:
 Hi folks

 I wanna make Delete server spec clear.

 The API doc says,
 When a server is deleted, all images created from that server are also 
 removed

 http://docs.openstack.org/api/openstack-compute/1.1/content/Delete_Server-d1e2883.html

 IMO, all images is vm images which stored on compute node instances_path.
 Is this correct?

 Cheers
 Nati

 ___
 Mailing list: https://launchpad.net/~openstack
 Post to     : openstack@lists.launchpad.net
 Unsubscribe : https://launchpad.net/~openstack
 More help   : https://help.launchpad.net/ListHelp

___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp


Re: [Openstack] Delete server spec

2011-12-07 Thread Jay Pipes
I would agree with that. If I delete a server instance, I don't want
to destroy snapshot images I took of that server...

-jay

On Wed, Dec 7, 2011 at 8:50 PM, Nachi Ueno
ueno.na...@nttdata-agilenet.com wrote:
 Hi Jessy

 Thanks.
 Hmm, there are no implementation of cleanup snapshot images.
 IMO, Snapshot image should not deleted, in case of API request mistake.

 https://github.com/openstack/nova/blob/master/nova/compute/manager.py#L540

 2011/12/7 Jesse Andrews anotherje...@gmail.com:
 I would interpret that to include the snapshots - but I'm not sure
 that is what I'd expect as a user.

 On Wed, Dec 7, 2011 at 5:05 PM, Nachi Ueno
 ueno.na...@nttdata-agilenet.com wrote:
 Hi folks

 I wanna make Delete server spec clear.

 The API doc says,
 When a server is deleted, all images created from that server are also 
 removed

 http://docs.openstack.org/api/openstack-compute/1.1/content/Delete_Server-d1e2883.html

 IMO, all images is vm images which stored on compute node instances_path.
 Is this correct?

 Cheers
 Nati

 ___
 Mailing list: https://launchpad.net/~openstack
 Post to     : openstack@lists.launchpad.net
 Unsubscribe : https://launchpad.net/~openstack
 More help   : https://help.launchpad.net/ListHelp

 ___
 Mailing list: https://launchpad.net/~openstack
 Post to     : openstack@lists.launchpad.net
 Unsubscribe : https://launchpad.net/~openstack
 More help   : https://help.launchpad.net/ListHelp

___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp


Re: [Openstack] Nova Subteam Changes

2011-12-07 Thread Trey Morris
excellent ideas. I especially like the standardized list of headers.

just to be sure, mondays at 2100utc? if so, no conflicts on my end

On Wed, Dec 7, 2011 at 4:51 AM, Thierry Carrez thie...@openstack.orgwrote:

 Vishvananda Ishaya wrote:

  2) *Closing down the team mailinglists.*  Some of the lists have been a
  bit active, but I think the approach that soren has been using of
  sending messages to the regular list with a team [header] is a better
  approach. Examples:
  [db] Should we use zookeeper?
  [scaling] Plans for bursting
  I realize that this will lead to a little more noise on some of the
  channels, but I think it makes sure that we don't isolate knowledge too
 much

 Could we have a standardized list of headers ? Those would double as
 tags for Nova bugs, so the scaling team could see a list of bugs in
 their area by searching for the scaling tag.

  3) *Closing teams.* Some of the teams haven't really started having much
  activity.  I'm going to move these teams into a WANTED section on the
  teams page here:
  http://wiki.openstack.org/Teams
  For now, people can just discuss stuff on the main mailing list, but we
  won't target blueprints to those teams specifically. We can make the
  teams official again once they have some activity and a clear person to
  be responsible for running them.  Please keep in mind that I'm not
  saying that this stuff isn't important, just that there is no need to
  separate out if there isn't a lot of activity.
  Specifically the teams which haven't really been very active are:
 
   1. Nova Upgrades Team 
 http://wiki.openstack.org/Teams#Nova_Upgrades_Team
   2. Nova Auth Team http://wiki.openstack.org/Teams#Nova_Auth_Team
   3. Nova Security Improvements Team
  http://wiki.openstack.org/Teams#Nova_Security_Improvements_Team

 Agreed, so far it's mostly been individual efforts, nothing coordinated.

   4. Nova Operational Support Team
  http://wiki.openstack.org/Teams#Nova_Operational_Support_Team
   5. Nova EC2 API Team http://wiki.openstack.org/Teams#Nova_EC2_API_Team
 
 
  Im going to leave the ec2 team for now, because it is relatively new.
   If anyone feels that the other teams above should not be folded back
  in, please let me know.

 The EC2 team has been meeting a few times already, and started triaging
 and working on EC2 bugs. I think it belongs to the active category :)

 Regards,

 --
 Thierry Carrez (ttx)
 Release Manager, OpenStack

 ___
 Mailing list: https://launchpad.net/~openstack
 Post to : openstack@lists.launchpad.net
 Unsubscribe : https://launchpad.net/~openstack
 More help   : https://help.launchpad.net/ListHelp

___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp