Re: [openstack-dev] [taskflow] Taskflow Video Tutorial

2013-08-01 Thread Sergey Lukjanov
Great, thank you!

Very interesting overview/demo of the current state of TaskFlow project.

Sincerely yours,
Sergey Lukjanov
Savanna Technical Lead
Mirantis Inc.

On Aug 2, 2013, at 4:51, Jessica Lucci  wrote:

> Hello all,
> 
> Provided here is a link to a quick overview and tutorial of the task flow 
> library. The video focuses on a distributed backend approach, but still 
> includes useful insight into the project as a whole. If you'd rather just 
> read the wiki and see examples, go ahead and skip to around 10:45. ;p
> 
> https://www.dropbox.com/s/kmpypmi95pk2taw/taskflow.mov
> 
> Useful links for the project:
> https://wiki.openstack.org/wiki/TaskFlow
> https://launchpad.net/taskflow
> 
> Thanks all!
> ___
> OpenStack-dev mailing list
> OpenStack-dev@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


[openstack-dev] [qa] How to apply submit Nova v3 API tempest tests

2013-08-01 Thread Christopher Yeoh
Hi,

Matthew Trenish brought up an issue on one of the proposed Nova V3 API
tempest tests:

> So I get why you do things this way. But, unlike nova we aren't going to
be able to do part1
> being a straight copy and paste. Doing so will double the amount of times
we run these tests in
> the gate once this gets merged. I think it would be best to go about this
as smaller patches in a
> longer series, just adding the v3 tests, instead of copying and pasting.
But, that has it's own
> tradeoffs too. I think you should probably bring this up on the mailing
list to get other opinions
> on how to go about adding the v3 tests. I feel that this needs a wider
discussion.

I think that the part 1/part 2 approach is the best way to go for the
tempest tests. For the Nova changes we had a policy of reviewing, but not
approving the part 1 patch until the part 2 patch had been approved. So
there is no extra gate load until its ready for v3.

I think its much easier to review when you only need to look at the v2 vs
v3 changes rather than see it as brand new code. Looking at it as brand new
code does have an advantage in that sometimes bugs are picked up in the old
code, but its a *lot* more effort for the reviewers even if the patch is
split up into many parts.

As for the extra gate load, the extra v3 tests are an issue longer term but
I think we're hoping the parallelisation work is going to save us :-)

Regards,

Chris
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


[openstack-dev] Fwd: code review

2013-08-01 Thread Yijing Zhang
Hello there,

I'd like you to do a code review.  Please visit

   https://review.openstack.org/#/c/38429/


Regards and best wishes,

Yijing Zhang
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Nova] Proposal to add Nikola Đipanov to nova-core

2013-08-01 Thread Michael Still
+1

On Thu, Aug 1, 2013 at 11:52 PM, Andrew Laski
 wrote:
> +1
>
>
> On 07/31/13 at 03:10pm, Russell Bryant wrote:
>>
>> Greetings,
>>
>> I propose that we add Nikola Đipanov to the nova-core team [1].
>>
>> Nikola has been actively contributing to nova for a while now, both in
>> code and reviews.  He provides high quality reviews. so I think he would
>> make a good addition to the review team.
>>
>>https://review.openstack.org/#/q/reviewer:ndipa...@redhat.com,n,z
>>
>>https://review.openstack.org/#/q/owner:ndipa...@redhat.com,n,z
>>
>> Please respond with +1/-1.
>>
>> Thanks,
>>
>> [1] https://wiki.openstack.org/wiki/Nova/CoreTeam
>>
>> --
>> Russell Bryant
>>
>> ___
>> OpenStack-dev mailing list
>> OpenStack-dev@lists.openstack.org
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
> ___
> OpenStack-dev mailing list
> OpenStack-dev@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



-- 
Rackspace Australia

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] Oslo.config raises AttributeError instead of NoSuchOptError

2013-08-01 Thread Nachi Ueno
Thanks! This code works

2013/8/1 Davanum Srinivas :
> Nachi,
>
> I've submitted a review for oslo.config -
> https://review.openstack.org/#/c/39850/ - please see if that helps in
> your environment as well.
>
> -- dims
>
> On Thu, Aug 1, 2013 at 8:22 PM, Nachi Ueno  wrote:
>> Hi Mark
>>
>> When I use latest oslo,
>> Oslo.config raises AttributeError instead of NoSuchOptError.
>> (now devstack is using latest oslo, and this is also breaking neutron gating)
>>
>> Neutron agents handles NoSuchOptError, so this change brokes
>> most of neutron agents.
>>
>> (see
>> https://bugs.launchpad.net/oslo/+bug/1207541 )
>>
>> I prefer NoSuchOptError than AtrtributeError, because
>> NoSuchOptError has more info and it is subclass of AtrtributeError.
>> At least, since this is API change, this change should be agreed on 
>> community.
>>
>> I would like to suggest to revert the change which makes this change.
>> https://github.com/openstack/oslo.config/commit/305ecd817836a90a8f5e495cbf6a770dcea1f7e8
>>
>> Best
>> Nachi
>>
>> ___
>> OpenStack-dev mailing list
>> OpenStack-dev@lists.openstack.org
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
>
> --
> Davanum Srinivas :: http://davanum.wordpress.com
>
> ___
> OpenStack-dev mailing list
> OpenStack-dev@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Grizzly] Retrieving a flavors extra-specs in horizon

2013-08-01 Thread Tom Fifield
Can you update 
https://ask.openstack.org/question/3674/get-a-flavors-extra-specs-in-horizon/ 
too :)


Regards,

Tom

On 02/08/13 11:50, Dale, StewartX T wrote:

Switching to your mentioned function fixed my issues.  Thanks!
-Original Message-
From: Kieran Spear [mailto:kisp...@gmail.com]
Sent: Thursday, August 01, 2013 5:30 PM
To: OpenStack Development Mailing List
Subject: Re: [openstack-dev] [Grizzly] Retrieving a flavors extra-specs in
horizon

On 2 August 2013 09:32, Dale, StewartX T  wrote:

Hi List,



I'm trying to port some UI(horizon) customization I did in Folsom so
that it works in Grizzly and I could use everyone's help. Must of the
process has gone smoothly, just one piece that isn't working. It's the
code that gets the extra-specs of a flavor. My old Folsom based code looks

like this:




flavor = api.flavor_get(self.request, flavor_id)

extras = flavor.get_keys()

inst.flavor_id = flavor_id

if extras:

  inst.myvalue = extras['mySpecs']

else:

  inst.myvalue = "-"



But using that code in Grizzly causes a crash ( and no log entries so
I don't even know why, just the). I did some research and saw in the
release notes of Grizzly that they had added support for getting a
flavors extra specs in horizon
(https://wiki.openstack.org/wiki/ReleaseNotes/Grizzly#Flavor_Extra_Spe
cs_Support) but I can't find any actual examples of how to do it.


The flavor part of code above should work fine (assuming 'mySpecs' is
actually in the returned specs). But there's a flavor_get_extras function
you can use too:


extras = api.nova.flavor_get_extras(self.request, flavor_id,
raw=True) extras

{u'test': u'value'}

extras.get('test', '-')

u'value'

extras.get('mySpecs', '-')

'-'

Check out the admin flavors panel for other examples.

Kieran





I am hoping someone on the list might know.





-Stewart Dale




___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev




___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] Oslo.config raises AttributeError instead of NoSuchOptError

2013-08-01 Thread Davanum Srinivas
Nachi,

I've submitted a review for oslo.config -
https://review.openstack.org/#/c/39850/ - please see if that helps in
your environment as well.

-- dims

On Thu, Aug 1, 2013 at 8:22 PM, Nachi Ueno  wrote:
> Hi Mark
>
> When I use latest oslo,
> Oslo.config raises AttributeError instead of NoSuchOptError.
> (now devstack is using latest oslo, and this is also breaking neutron gating)
>
> Neutron agents handles NoSuchOptError, so this change brokes
> most of neutron agents.
>
> (see
> https://bugs.launchpad.net/oslo/+bug/1207541 )
>
> I prefer NoSuchOptError than AtrtributeError, because
> NoSuchOptError has more info and it is subclass of AtrtributeError.
> At least, since this is API change, this change should be agreed on community.
>
> I would like to suggest to revert the change which makes this change.
> https://github.com/openstack/oslo.config/commit/305ecd817836a90a8f5e495cbf6a770dcea1f7e8
>
> Best
> Nachi
>
> ___
> OpenStack-dev mailing list
> OpenStack-dev@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



-- 
Davanum Srinivas :: http://davanum.wordpress.com

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Grizzly] Retrieving a flavors extra-specs in horizon

2013-08-01 Thread Dale, StewartX T
Switching to your mentioned function fixed my issues.  Thanks!
-Original Message-
From: Kieran Spear [mailto:kisp...@gmail.com] 
Sent: Thursday, August 01, 2013 5:30 PM
To: OpenStack Development Mailing List
Subject: Re: [openstack-dev] [Grizzly] Retrieving a flavors extra-specs in
horizon

On 2 August 2013 09:32, Dale, StewartX T  wrote:
> Hi List,
>
>
>
> I'm trying to port some UI(horizon) customization I did in Folsom so 
> that it works in Grizzly and I could use everyone's help. Must of the 
> process has gone smoothly, just one piece that isn't working. It's the 
> code that gets the extra-specs of a flavor. My old Folsom based code looks
like this:
>
>
>
> flavor = api.flavor_get(self.request, flavor_id)
>
> extras = flavor.get_keys()
>
> inst.flavor_id = flavor_id
>
> if extras:
>
>  inst.myvalue = extras['mySpecs']
>
> else:
>
>  inst.myvalue = "-"
>
>
>
> But using that code in Grizzly causes a crash ( and no log entries so 
> I don't even know why, just the). I did some research and saw in the 
> release notes of Grizzly that they had added support for getting a 
> flavors extra specs in horizon
> (https://wiki.openstack.org/wiki/ReleaseNotes/Grizzly#Flavor_Extra_Spe
> cs_Support) but I can't find any actual examples of how to do it.

The flavor part of code above should work fine (assuming 'mySpecs' is
actually in the returned specs). But there's a flavor_get_extras function
you can use too:

>>> extras = api.nova.flavor_get_extras(self.request, flavor_id, 
>>> raw=True) extras
{u'test': u'value'}
>>> extras.get('test', '-')
u'value'
>>> extras.get('mySpecs', '-')
'-'

Check out the admin flavors panel for other examples.

Kieran

>
>
>
> I am hoping someone on the list might know.
>
>
>
>
>
> -Stewart Dale
>
>
>
>
> ___
> OpenStack-dev mailing list
> OpenStack-dev@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


smime.p7s
Description: S/MIME cryptographic signature
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Nova] Review request: Blurprint of API validation

2013-08-01 Thread Ken'ichi Ohmichi

Hi Russell,

I have assigned you as the approver of bp "nova-api-validation-fw",
please approve it or give some comments.
  https://blueprints.launchpad.net/nova/+spec/nova-api-validation-fw

The implementation of core part is almost complete.
now waiting for blueprint approval.
  https://review.openstack.org/#/c/25358/


Thanks,
Ken'ichi Ohmichi

---
On Tue, 16 Jul 2013 13:37:15 +0900
"Ken'ichi Ohmichi"  wrote:
> 
> Hi Russell,
> 
> I assigned you as the approver of bp "nova-api-validation-fw",
> so could you take a look at this bp?
> 
> You have changed the priority from Medium to Low these days.
> Could you show your concerns about this bp if you have.
> 
> 
> Thanks
> Ken'ichi Ohmichi
> 
> ---
> On Tue, 9 Jul 2013 20:45:57 +0900
> "Ken'ichi Ohmichi"  wrote:
> > 
> > Hi,
> > 
> > The blueprint "nova-api-validation-fw" has not been approved yet.
> > I hope the core patch of this blueprint is merged to Havana-2,
> > because of completing comprehensive API validation of Nova v3 API
> > for Havana release. What should we do for it?
> > 
> > 
> > The summary of "nova-api-validation-fw":
> >   * The patch review is in good progress.
> >   * "nova-api-validation-fw" is API input validation framework.
> >   * Simplify the code in the extensions of Nova v3 API.
> >   * Define API schema with JSON Schema.
> >   * Define common data formats to standardize whole of Nova v3 API.
> >   * Possible to choose APIs which are applied with this framework.
> >   * Possible to migrate other web framework.
> > 
> > 
> > The details of "nova-api-validation-fw" are the following:
> > 
> > * The patch review is in good progress.
> >   7 reviewers have marked +1 now.
> >   We are waiting for the approval of this blueprint before merging the
> >   patch. (https://review.openstack.org/#/c/25358/)
> > 
> > * "nova-api-validation-fw" is API input validation framework.
> >   This framework validates API parameters in a request body before the
> >   execution of API method. If the parameter is invalid, nova-api returns
> >   a BadRequest response including its reason.  
> > 
> > * Simplify the code in the extensions of Nova v3 API.
> >   There are a lot of validation code in each API method.
> >   After applying this framework, we will be able to separate them from
> >   API methods by defining API schema. That makes the code more readable.
> >   Also through trying to define API schema of each API, we will find the
> >   lacks of validation code and complement them for better v3 API.
> >   Necessary API schemas are for API which contains a request body. They
> >   are 37 APIs on Nova v3 API now, and they will increase later because
> >   the tasks, which port v2 API to v3 API, is in progress.
> > 
> > * Define API schema with JSON Schema.
> >   JSON Schema contains many features for validation, the details are
> >   written on http://json-schema.org/.
> >   Here is the schema of v3 keypairs API as a sample:
> >   == Request body sample of keypairs API ===
> >   {
> > "keypair": {
> > "name": "keypair-dab428fe-6186-4a14-b3de-92131f76cd39",
> > "public_key": "ssh-rsa B3NzaC1yc2EA[..]== Generated by Nova"
> >   }
> >   == API schema of keypairs API 
> >   {
> >   'type': 'object',
> >   'properties': {
> >   'keypair': {
> >   'type': 'object',
> >   'properties': {
> >   'name': {'type': 'string', 'minLength': 1, 'maxLength': 
> > 255},
> >   'public_key': {'type': 'string'},
> >   },
> >   'required': ['name'],
> >   },
> >   },
> >   'required': ['keypair'],
> >   }
> > 
> > * Define common data formats to standardize whole of Nova v3 API.
> >   We can define common data formats with FormatChecker of JSON Schema
> >   library.
> > 
> >   e.g. Data format of 'boolean':
> > @jsonschema.FormatChecker.cls_checks('boolean')
> > def validate_boolean_format(instance):
> > return instance.upper() == 'TRUE' or instance.upper() == 'FALSE'
> > 
> >   The formats can be used in API schema:
> > 'onSharedStorage': {
> > 'type': 'string', 'format': 'boolean',
> > },
> > 
> >   By re-using these common formats in many API schemas, that will be
> >   able to standardize whole of Nova v3 API.
> > 
> > * Possible to choose APIs which are applied with this framework.
> >   We can apply this framework to Nova v3 API only, the existing API (v2) can
> >   be out of scope of this framework.
> > 
> > * Possible to migrate other web framework.
> >   The API validation of this framework is executed with the decorator of API
> >   method, that means this framework does not depend on web framework.
> >   Also when migrating other web framework(e.g. Pecan/WSME) in the future, we
> >   will be able to use this framework.
> > 
> > 
> > Thanks
> > Ken'ichi Ohmichi
> > 
> > ---
> > On Fri, 5 Jul 2013 16:27:39 +0900
> > "Ken'ichi Ohmichi"  wrote:
> > > 
> > > Hi,
> > 

Re: [openstack-dev] [taskflow] Taskflow Video Tutorial

2013-08-01 Thread Joshua Harlow
Awesome work jessica!

From: Jessica Lucci 
mailto:jessica.lu...@rackspace.com>>
Reply-To: OpenStack Development Mailing List 
mailto:openstack-dev@lists.openstack.org>>
Date: Thursday, August 1, 2013 5:51 PM
To: 
"openstack-dev@lists.openstack.org" 
mailto:openstack-dev@lists.openstack.org>>
Subject: [openstack-dev] [taskflow] Taskflow Video Tutorial

Hello all,

Provided here is a link to a quick overview and tutorial of the task flow 
library. The video focuses on a distributed backend approach, but still 
includes useful insight into the project as a whole. If you'd rather just read 
the wiki and see examples, go ahead and skip to around 10:45. ;p

https://www.dropbox.com/s/kmpypmi95pk2taw/taskflow.mov

Useful links for the project:
https://wiki.openstack.org/wiki/TaskFlow
https://launchpad.net/taskflow

Thanks all!
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Ceilometer] Alarming should be outside of Ceilometer as a separate package.

2013-08-01 Thread Sandy Walsh


On 08/01/2013 07:22 PM, Doug Hellmann wrote:
> 
> 
> 
> On Thu, Aug 1, 2013 at 10:31 AM, Sandy Walsh  > wrote:
> 
> Hey y'all,
> 
> I've had a little thorn in my claw on this topic for a while and thought
> I'd ask the larger group.
> 
> I applaud the efforts of the people working on the alarming additions to
> Ceilometer, but I've got concerns that we're packaging things the
> wrong way.
> 
> I fear we're making another Zenoss/Nagios with Ceilometer. It's trying
> to do too much.
> 
> The current trend in the monitoring work (#monitoringlove) is to build
> your own stack from a series of components. These components take in
> inputs, process them and spit out outputs.
> Collectors/Transformers/Publishers. This is the model CM is built on.
> 
> Making an all-singing-all-dancing monolithic monitoring package is the
> old way of building these tools. People want to use best-of-breed
> components for their monitoring stack. I'd like to be able to use
> reimann.io  for my stream manager, diamond for my
> collector, logstash for
> my parser, etc. Alarming should just be another consumer.
> 
> CM should do one thing well. Collect data from openstack, store and
> process them, and make them available to other systems via the API or
> publishers. That's all. It should understand these events better than
> any other product out there. It should be able to produce meaningful
> metrics/meters from these events.
> 
> Alarming should be yet another consumer of the data CM produces. Done
> right, the If-This-Then-That nature of the alarming tool could be
> re-used by the orchestration team or perhaps even scheduling.
> Intertwining it with CM is making the whole thing too complex and rigid
> (imho).
> 
> CM should be focused on extending our publishers and input plug-ins.
> 
> I'd like to propose that alarming becomes its own project outside of
> Ceilometer. Or, at the very least, its own package, external of the
> Ceilometer code base. Perhaps it still lives under the CM moniker, but
> packaging-wise, I think it should live as a separate code base.
> 
> 
> It is currently implemented as a pair of daemons (one to monitor the
> alarm state, another to send the notifications). Both daemons use a
> ceilometer client to talk to the REST API to consume the sample data or
> get the alarm details, as required. It looks like alarms are triggered
> by sending RPC cast message, and that those in turn trigger the webhook
> invocation. That seems pretty loosely coupled, as far as the runtime
> goes. Granted, it's still in the main ceilometer code tree, but that
> doesn't say anything about how the distros will package it.
> 
> I'll admit I haven't been closely involved in the development of this
> feature, so maybe my quick review of the code missed something that is
> bringing on this sentiment?

No, you hit the nail on the head. It's nothing with the implementation,
it's purely with the packaging and having it co-exist within ceilometer.
Since it has its own services, uses Oslo, the CM client and operates via
the public API, it should be able to live outside the main CM codebase.
My concern is that it has a different mandate than CM (or the CM mandate
is too broad).

What really brought it on for me was doing code reviews for CM and
hitting all this alarm stuff and thinking "this is mental context switch
from what CM does, it really doesn't belong here." (though I'm happy to
help out with the reviews)

-S


> 
> Doug
>  
> 
> 
> Please, change my view. :)
> 
> -S
> 
> Side note: I might even argue that the statistical features of CM are
> going a little too far. My only counter-argument is that statistics are
> the only way to prevent us from sending large amounts of data over the
> wire for post-processing. But that's a separate discussion.
> 
> 
> ___
> OpenStack-dev mailing list
> OpenStack-dev@lists.openstack.org
> 
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> 
> 
> 
> 
> ___
> OpenStack-dev mailing list
> OpenStack-dev@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> 

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


[openstack-dev] [taskflow] Taskflow Video Tutorial

2013-08-01 Thread Jessica Lucci
Hello all,

Provided here is a link to a quick overview and tutorial of the task flow 
library. The video focuses on a distributed backend approach, but still 
includes useful insight into the project as a whole. If you'd rather just read 
the wiki and see examples, go ahead and skip to around 10:45. ;p

https://www.dropbox.com/s/kmpypmi95pk2taw/taskflow.mov

Useful links for the project:
https://wiki.openstack.org/wiki/TaskFlow
https://launchpad.net/taskflow

Thanks all!
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Openstack] Ceilometer and notifications

2013-08-01 Thread Sandy Walsh


On 08/01/2013 07:25 PM, Doug Hellmann wrote:
> 
> 
> 
> On Thu, Aug 1, 2013 at 9:25 AM, Julien Danjou  > wrote:
> 
> On Thu, Aug 01 2013, Sandy Walsh wrote:
> 
> > Right, that is a concern. Within RAX we have two downstream services
> > that consume notifications (StackTach and Yagi) and we've configured
> > nova to write to two queues. --notifications_topics can take a list.
> 
> Fair enough. So IIUC that means we should change Ceilometer to consume
> CONF.notification_topics using the default queue associated with this
> topic, and let the deployer add more notification_topics in nova if they
> have several consumer (that should have their own
> CONF.notification_topics).
> 
> This would be the correct way, even if not really deployer-friendly.
> 
> 
> The other solution would be to have the sender of the message NOT
> declare a queue at all. The queue is a client-side feature, and the
> sender shouldn't care about the queue.

But that's the problem. Once openstack is running (and making money) we
don't want to lose any events. So we have to create the queue on the
producer side. Normally, yes, the consumer should create the queue.

> 
> Doug
>  
> 
> 
> > I don't think so, but if that was possible it would solve our problem.
> >
> > AFAIK, amqp only uses the exchange as a dispatcher and all storage is
> > done in the queue ... but I could be wrong. I vaguely recall there
> being
> > a durable exchange setting as well as durable queue.
> >
> > I'll do some investigating.
> 
> Yeah I've little hope too, but might worth a look. Thanks Sandy!
> 
> --
> Julien Danjou
> # Free Software hacker # freelance consultant
> # http://julien.danjou.info
> 
> ___
> OpenStack-dev mailing list
> OpenStack-dev@lists.openstack.org
> 
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> 
> 

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Grizzly] Retrieving a flavors extra-specs in horizon

2013-08-01 Thread Kieran Spear
On 2 August 2013 09:32, Dale, StewartX T  wrote:
> Hi List,
>
>
>
> I'm trying to port some UI(horizon) customization I did in Folsom so that it
> works in Grizzly and I could use everyone’s help. Must of the process has
> gone smoothly, just one piece that isn't working. It’s the code that gets
> the extra-specs of a flavor. My old Folsom based code looks like this:
>
>
>
> flavor = api.flavor_get(self.request, flavor_id)
>
> extras = flavor.get_keys()
>
> inst.flavor_id = flavor_id
>
> if extras:
>
>  inst.myvalue = extras['mySpecs']
>
> else:
>
>  inst.myvalue = "-"
>
>
>
> But using that code in Grizzly causes a crash ( and no log entries so I
> don't even know why, just the). I did some research and saw in the release
> notes of Grizzly that they had added support for getting a flavors extra
> specs in horizon
> (https://wiki.openstack.org/wiki/ReleaseNotes/Grizzly#Flavor_Extra_Specs_Support)
> but I can't find any actual examples of how to do it.

The flavor part of code above should work fine (assuming 'mySpecs' is
actually in the returned specs). But there's a flavor_get_extras
function you can use too:

>>> extras = api.nova.flavor_get_extras(self.request, flavor_id, raw=True)
>>> extras
{u'test': u'value'}
>>> extras.get('test', '-')
u'value'
>>> extras.get('mySpecs', '-')
'-'

Check out the admin flavors panel for other examples.

Kieran

>
>
>
> I am hoping someone on the list might know.
>
>
>
>
>
> -Stewart Dale
>
>
>
>
> ___
> OpenStack-dev mailing list
> OpenStack-dev@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


[openstack-dev] Oslo.config raises AttributeError instead of NoSuchOptError

2013-08-01 Thread Nachi Ueno
Hi Mark

When I use latest oslo,
Oslo.config raises AttributeError instead of NoSuchOptError.
(now devstack is using latest oslo, and this is also breaking neutron gating)

Neutron agents handles NoSuchOptError, so this change brokes
most of neutron agents.

(see
https://bugs.launchpad.net/oslo/+bug/1207541 )

I prefer NoSuchOptError than AtrtributeError, because
NoSuchOptError has more info and it is subclass of AtrtributeError.
At least, since this is API change, this change should be agreed on community.

I would like to suggest to revert the change which makes this change.
https://github.com/openstack/oslo.config/commit/305ecd817836a90a8f5e495cbf6a770dcea1f7e8

Best
Nachi

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Grizzly] Retrieving a flavors extra-specs in horizon

2013-08-01 Thread Dolph Mathews
On Thu, Aug 1, 2013 at 6:32 PM, Dale, StewartX T
wrote:

> Hi List,
>
> ** **
>
> I'm trying to port some UI(horizon) customization I did in Folsom so that
> it works in Grizzly and I could use everyone’s help. Must of the process
> has gone smoothly, just one piece that isn't working. It’s the code that
> gets the extra-specs of a flavor. My old Folsom based code looks like this:
> 
>
> ** **
>
> flavor = api.flavor_get(self.request, flavor_id)
>
> extras = flavor.get_keys()
>
> inst.flavor_id = flavor_id
>
> if extras:
>
>  inst.myvalue = extras['mySpecs']
>
> else:
>
>  inst.myvalue = "-"
>
> ** **
>
> But using that code in Grizzly causes a crash ( and no log entries so I
> don't even know why, just the).
>

It looks like you were going to link to a backtrace or something at the end
there :)


> I did some research and saw in the release notes of Grizzly that they had
> added support for getting a flavors extra specs in horizon (
> https://wiki.openstack.org/wiki/ReleaseNotes/Grizzly#Flavor_Extra_Specs_Support)
> but I can't find any actual examples of how to do it.  
>
> ** **
>
> I am hoping someone on the list might know.
>
> ** **
>
> ** **
>
> -Stewart Dale
>
> ** **
>
> ___
> OpenStack-dev mailing list
> OpenStack-dev@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>


-- 

-Dolph
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


[openstack-dev] Savanna deployement / fixed

2013-08-01 Thread Linus Nova
Hi everyone,

I come to close my last request about savanna deployment. All is work now.

The trouble come from bad configuration file.

Best regrds.

Linus Nova
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


[openstack-dev] Savanna deployement

2013-08-01 Thread Linus Nova
Hi,

I have some troubles when I want connect savanna UI (august version) in
Horizon. This what I have in apache file error :

[Fri Aug 02 01:26:33 2013] [notice] caught SIGTERM, shutting down
[Fri Aug 02 01:26:34 2013] [warn] mod_wsgi: Compiled for Python/2.7.2+.
[Fri Aug 02 01:26:34 2013] [warn] mod_wsgi: Runtime using Python/2.7.3.
[Fri Aug 02 01:26:34 2013] [notice] Apache/2.2.22 (Ubuntu) mod_wsgi/3.3
Python/2.7.3 configured -- resuming normal operations
[Fri Aug 02 01:27:00 2013] [notice] caught SIGTERM, shutting down
[Fri Aug 02 01:27:00 2013] [warn] mod_wsgi: Compiled for Python/2.7.2+.
[Fri Aug 02 01:27:00 2013] [warn] mod_wsgi: Runtime using Python/2.7.3.
[Fri Aug 02 01:27:00 2013] [notice] Apache/2.2.22 (Ubuntu) mod_wsgi/3.3
Python/2.7.3 configured -- resuming normal operations
[Thu Aug 01 23:27:07 2013] [error] INFO:savannadashboard.dashboard:Savanna
recognizes Dashboard release as "grizzly"
[Thu Aug 01 23:27:07 2013] [error] INFO:root:Parameter SAVANNA_SERVICE is
not found in local_settings.py, using default "mapreduce"
[Thu Aug 01 23:27:07 2013] [error] ERROR:django.request:Internal Server
Error: /horizon/
[Thu Aug 01 23:27:07 2013] [error] Traceback (most recent call last):
[Thu Aug 01 23:27:07 2013] [error]   File
"/usr/lib/python2.7/dist-packages/django/core/handlers/base.py", line 111,
in get_response
[Thu Aug 01 23:27:07 2013] [error] response = callback(request,
*callback_args, **callback_kwargs)
[Thu Aug 01 23:27:07 2013] [error]   File
"/usr/lib/python2.7/dist-packages/django/views/decorators/vary.py", line
36, in inner_func
[Thu Aug 01 23:27:07 2013] [error] response = func(*args, **kwargs)
[Thu Aug 01 23:27:07 2013] [error]   File
"/usr/share/openstack-dashboard/openstack_dashboard/wsgi/../../openstack_dashboard/views.py",
line 38, in splash
[Thu Aug 01 23:27:07 2013] [error] return shortcuts.render(request,
'splash.html', {'form': form})
[Thu Aug 01 23:27:07 2013] [error]   File
"/usr/lib/python2.7/dist-packages/django/shortcuts/__init__.py", line 44,
in render
[Thu Aug 01 23:27:07 2013] [error] return
HttpResponse(loader.render_to_string(*args, **kwargs),
[Thu Aug 01 23:27:07 2013] [error]   File
"/usr/lib/python2.7/dist-packages/django/template/loader.py", line 169, in
render_to_string
[Thu Aug 01 23:27:07 2013] [error] t = get_template(template_name)
[Thu Aug 01 23:27:07 2013] [error]   File
"/usr/lib/python2.7/dist-packages/django/template/loader.py", line 145, in
get_template
[Thu Aug 01 23:27:07 2013] [error] template, origin =
find_template(template_name)
[Thu Aug 01 23:27:07 2013] [error]   File
"/usr/lib/python2.7/dist-packages/django/template/loader.py", line 138, in
find_template
[Thu Aug 01 23:27:07 2013] [error] raise TemplateDoesNotExist(name)
[Thu Aug 01 23:27:07 2013] [error] TemplateDoesNotExist: splash.html

Any idea?

Thanks in advance for your help.

Best regards.
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Ceilometer] Ceilometer and nova compute cells

2013-08-01 Thread Sam Morrison

On 31/07/2013, at 6:45 PM, Julien Danjou  wrote:

> On Wed, Jul 31 2013, Sam Morrison wrote:
> 
> Hi Sam,
> 
>> Does everything that gets stored in the datastore go through the
>> ceilometer.collector.metering queue?
> 
> If you only use the RPC publisher (which is the default), yes.
> 
>> If so maybe the collector could instead of storing the message forward these
>> messages to another rabbit server where another collector is running who in
>> turn either forwards it up or stores it in the datastore (or both maybe)
>> I think the confusing thing to me is how notification messages get into
>> ceilometer. The collector does 2 things I think? And it seems to leave the
>> messages on the notification.info queue to build up?
> 
> Yes, the collector has 2 purposes right now:
> 1. It listens for notifications on the RPC bus and converts them into
>   Samples;
> 2. It publishes these Samples according to your pipeline.yaml file to
>   different conduct, the default being a RPC call on a collector for
>   storing these samples.

Thanks for the info, looking at the code I think the way to do this would be to 
create a new dispatcher that instead of recording the metering data just 
forwards it on to another RPC server for another collector to consume.
Do you think that's the way to go?

Cheers,
Sam



> 
> -- 
> Julien Danjou
> ;; Free Software hacker ; freelance consultant
> ;; http://julien.danjou.info


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


[openstack-dev] [Grizzly] Retrieving a flavors extra-specs in horizon

2013-08-01 Thread Dale, StewartX T
Hi List,

 

I'm trying to port some UI(horizon) customization I did in Folsom so that it
works in Grizzly and I could use everyone's help. Must of the process has
gone smoothly, just one piece that isn't working. It's the code that gets
the extra-specs of a flavor. My old Folsom based code looks like this:

 

flavor = api.flavor_get(self.request, flavor_id)

extras = flavor.get_keys()

inst.flavor_id = flavor_id

if extras:

 inst.myvalue = extras['mySpecs']

else:

 inst.myvalue = "-"

 

But using that code in Grizzly causes a crash ( and no log entries so I
don't even know why, just the). I did some research and saw in the release
notes of Grizzly that they had added support for getting a flavors extra
specs in horizon
(https://wiki.openstack.org/wiki/ReleaseNotes/Grizzly#Flavor_Extra_Specs_Sup
port) but I can't find any actual examples of how to do it.  

 

I am hoping someone on the list might know.

 

 

-Stewart Dale

 



smime.p7s
Description: S/MIME cryptographic signature
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [CI] Now gating on Python 3.3

2013-08-01 Thread Doug Hellmann
Excellent, thank you all!


On Wed, Jul 31, 2013 at 11:18 PM, Jeremy Stanley  wrote:

> A quick announcement... thanks to the invaluable assistance of Chuck
> Short, Dan Prince, Julien Danjou and others, some projects are now
> being successfully tested against and gated on Python 3.3 (with even
> more projects very close to ready for the same).
>
> As discussed previously in summit sessions and on this list,
> libraries and clients whose various dependencies already fully
> support Python 3.3 are great candidates for addition in the near
> term. Feel free to reach out to the infrastructure team on the
> Freenode IRC network in #openstack-infra or through the
> openstack-in...@lists.openstack.org mailing list, should you require
> assistance setting up tests for Python 3.3 compatibility in a
> project.
> --
> Jeremy Stanley
>
> ___
> OpenStack-dev mailing list
> OpenStack-dev@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Neutron] devstack + neutron fails on firewall_driver

2013-08-01 Thread James Kyle
Sorry, I cleaned them out after resolving. 

Kyle helped me with the bug:

> This fixed it for me:
> 
> cd /usr/local/lib/python2.7/dist-packages
> sudo rm -rf oslo*
> sudo pip install --upgrade 
> http://tarballs.openstack.org/oslo.config/oslo.config-1.2.0a3.tar.gz#egg=oslo.config-1.2.0a3

It's tracked in the launchpad issue 
https://bugs.launchpad.net/neutron/+bug/1206013

-james

On Aug 1, 2013, at 2:48 PM, Rajesh Mohan  wrote:

> The links for the error messages are failing for me.
> 
> Can you send me the actual error message?
> 
> Thanks.
> 
> 
> 
> On Thu, Aug 1, 2013 at 9:51 AM, James Kyle  wrote:
> Morning,
> 
> I'm having some issues getting devstack + neutron going.
> 
> If I don't use Q_USE_DEBUG_COMMAND, it completes and notes that the l3 agent 
> has failed to start. Here's a paste of the stack.sh error and the stack trace 
> from q-l3 => https://gist.github.com/jameskyle/6133049
> 
> If I do use the debug command, this is the error/logs I get: 
> https://gist.github.com/jameskyle/6133125
> 
> And this is my localrc https://gist.github.com/jameskyle/6133134
> 
> Currently running on an ubuntu 12.04 vm.
> 
> The behavior is somewhat recent, last week or so. But I can't find any 
> references to the issue in bugs.
> 
> Thanks for any input!
> 
> Cheers,
> 
> -james
> ___
> OpenStack-dev mailing list
> OpenStack-dev@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> 
> ___
> OpenStack-dev mailing list
> OpenStack-dev@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Openstack] Ceilometer and notifications

2013-08-01 Thread Doug Hellmann
On Thu, Aug 1, 2013 at 9:25 AM, Julien Danjou  wrote:

> On Thu, Aug 01 2013, Sandy Walsh wrote:
>
> > Right, that is a concern. Within RAX we have two downstream services
> > that consume notifications (StackTach and Yagi) and we've configured
> > nova to write to two queues. --notifications_topics can take a list.
>
> Fair enough. So IIUC that means we should change Ceilometer to consume
> CONF.notification_topics using the default queue associated with this
> topic, and let the deployer add more notification_topics in nova if they
> have several consumer (that should have their own
> CONF.notification_topics).
>
> This would be the correct way, even if not really deployer-friendly.
>

The other solution would be to have the sender of the message NOT declare a
queue at all. The queue is a client-side feature, and the sender shouldn't
care about the queue.

Doug


>
> > I don't think so, but if that was possible it would solve our problem.
> >
> > AFAIK, amqp only uses the exchange as a dispatcher and all storage is
> > done in the queue ... but I could be wrong. I vaguely recall there being
> > a durable exchange setting as well as durable queue.
> >
> > I'll do some investigating.
>
> Yeah I've little hope too, but might worth a look. Thanks Sandy!
>
> --
> Julien Danjou
> # Free Software hacker # freelance consultant
> # http://julien.danjou.info
>
> ___
> OpenStack-dev mailing list
> OpenStack-dev@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Ceilometer] Alarming should be outside of Ceilometer as a separate package.

2013-08-01 Thread Doug Hellmann
On Thu, Aug 1, 2013 at 10:31 AM, Sandy Walsh wrote:

> Hey y'all,
>
> I've had a little thorn in my claw on this topic for a while and thought
> I'd ask the larger group.
>
> I applaud the efforts of the people working on the alarming additions to
> Ceilometer, but I've got concerns that we're packaging things the wrong
> way.
>
> I fear we're making another Zenoss/Nagios with Ceilometer. It's trying
> to do too much.
>
> The current trend in the monitoring work (#monitoringlove) is to build
> your own stack from a series of components. These components take in
> inputs, process them and spit out outputs.
> Collectors/Transformers/Publishers. This is the model CM is built on.
>
> Making an all-singing-all-dancing monolithic monitoring package is the
> old way of building these tools. People want to use best-of-breed
> components for their monitoring stack. I'd like to be able to use
> reimann.io for my stream manager, diamond for my collector, logstash for
> my parser, etc. Alarming should just be another consumer.
>
> CM should do one thing well. Collect data from openstack, store and
> process them, and make them available to other systems via the API or
> publishers. That's all. It should understand these events better than
> any other product out there. It should be able to produce meaningful
> metrics/meters from these events.
>
> Alarming should be yet another consumer of the data CM produces. Done
> right, the If-This-Then-That nature of the alarming tool could be
> re-used by the orchestration team or perhaps even scheduling.
> Intertwining it with CM is making the whole thing too complex and rigid
> (imho).
>
> CM should be focused on extending our publishers and input plug-ins.
>
> I'd like to propose that alarming becomes its own project outside of
> Ceilometer. Or, at the very least, its own package, external of the
> Ceilometer code base. Perhaps it still lives under the CM moniker, but
> packaging-wise, I think it should live as a separate code base.
>

It is currently implemented as a pair of daemons (one to monitor the alarm
state, another to send the notifications). Both daemons use a ceilometer
client to talk to the REST API to consume the sample data or get the alarm
details, as required. It looks like alarms are triggered by sending RPC
cast message, and that those in turn trigger the webhook invocation. That
seems pretty loosely coupled, as far as the runtime goes. Granted, it's
still in the main ceilometer code tree, but that doesn't say anything about
how the distros will package it.

I'll admit I haven't been closely involved in the development of this
feature, so maybe my quick review of the code missed something that is
bringing on this sentiment?

Doug


>
> Please, change my view. :)
>
> -S
>
> Side note: I might even argue that the statistical features of CM are
> going a little too far. My only counter-argument is that statistics are
> the only way to prevent us from sending large amounts of data over the
> wire for post-processing. But that's a separate discussion.
>
>
> ___
> OpenStack-dev mailing list
> OpenStack-dev@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Ceilometer] Nomination for Mehdi Abaakouk

2013-08-01 Thread Doug Hellmann
+1


On Wed, Jul 31, 2013 at 8:55 PM, Lu, Lianhao  wrote:

> +1
>
> -Lianhao
>
> Angus Salkeld wrote on 2013-07-31:
> > On 31/07/13 10:56 +0200, Julien Danjou wrote:
> >> Hi,
> >>
> >> I'd like to propose to add Mehdi Abaakouk (sileht) to ceilometer-core.
> >> He has been a valuable contributor for the last months, doing a lot of
> >> work in the alarming blueprints, and useful code reviews.
> >
> > +1
> >
> >>
> >> --
> >> Julien Danjou
> >> -- Free Software hacker - freelance consultant
> >> -- http://julien.danjou.info
>
>
> ___
> OpenStack-dev mailing list
> OpenStack-dev@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Ceilometer] Wanted to Introduce Myself

2013-08-01 Thread Doug Hellmann
Hi, Thomas,

It's great to have another contributor!

Doug


On Wed, Jul 31, 2013 at 6:04 PM, Thomas Maddox
wrote:

>  Hey everyone!
>
>  I recently got freed up to work on Ceilometer, so I thought I'd send out
> a quick note to introduce myself.
>
>  I'm a developer at RAX and I'll be working with Andrew Melton and Sandy
> Walsh on Ceilometer going forward. I started at RAX 3 months ago to work on
> StackTach and other monitoring/analytics efforts for our cloud. I just
> finished setting up my dev environment for CM and will be snagging a bug or
> two to wet my feet in the codebase here in a next day or so. I'm very
> excited to move on to working with you all. :)
>
>  Cheers!
>
>  -Thomas
>
> ___
> OpenStack-dev mailing list
> OpenStack-dev@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Neutron] devstack + neutron fails on firewall_driver

2013-08-01 Thread Rajesh Mohan
The links for the error messages are failing for me.

Can you send me the actual error message?

Thanks.



On Thu, Aug 1, 2013 at 9:51 AM, James Kyle  wrote:

> Morning,
>
> I'm having some issues getting devstack + neutron going.
>
> If I don't use Q_USE_DEBUG_COMMAND, it completes and notes that the l3
> agent has failed to start. Here's a paste of the stack.sh error and the
> stack trace from q-l3 => https://gist.github.com/jameskyle/6133049
>
> If I do use the debug command, this is the error/logs I get:
> https://gist.github.com/jameskyle/6133125
>
> And this is my localrc https://gist.github.com/jameskyle/6133134
>
> Currently running on an ubuntu 12.04 vm.
>
> The behavior is somewhat recent, last week or so. But I can't find any
> references to the issue in bugs.
>
> Thanks for any input!
>
> Cheers,
>
> -james
> ___
> OpenStack-dev mailing list
> OpenStack-dev@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] Keystone auth_token middleware and the auth_host/auth_port configuration options

2013-08-01 Thread Dolph Mathews
On Thu, Aug 1, 2013 at 12:43 AM, Jay Pipes  wrote:

> I have a question for the auth_token middleware developers.
>
> The default auth_port is set to 35357, which is the admin API port for
> Keystone:
>
> https://github.com/openstack/**python-keystoneclient/blob/**
> master/keystoneclient/**middleware/auth_token.py#L201


I'm going to chalk this one up to an existing lack of documentation, a
situation which happens to be vastly improved by (but not at all rectified
by): https://review.openstack.org/#/c/34430/ ).

There's two use cases for the 'auth_*' options in auth_token.

1) Direct unauthenticated clients to the public/service API in order to
authenticate, so that they can return with a valid token (this *requires*
the port 5000 public/service API endpoint)

2) Allow auth_token to communicate with keystone for token validation, etc,
at an administrative level (this *requires* the port 35357 admin API
endpoint)

The first use case explicitly requires the public API endpoint due to the
"long" authentication flow on v2.0 (where the client has no default tenancy
and must retrieve a list of tenants) as the GET :5000/v2.0/tenants
(intended for users to list tenants they have explicit access to) will
return a completely different response to GET :35357/v2.0/tenants (intended
for admins to list all tenants in the system).

The second use case explicitly requires the admin API because v2.0
public/service API (GET :5000/v2.0/tokens) is not supported.


>
>
> However, the token validation API call (POST /v2.0/tokens or POST
> /v3/auth/tokens) uses the *service* API port, not the admin API port... in
> fact, in the v3 API, there is no longer any distinction (thankfully)
> between the service API and the admin API ... it's all just one API.
>

> So, my question is this: we've been setting all of the auth_token config
> options to the 5000 port -- the service API port for v2.0 Keystone. Is this
> problematic?


Yes! We need to immediately start logging a warning if this conditional is
met:


https://github.com/openstack/python-keystoneclient/blob/master/keystoneclient/middleware/auth_token.py#L287-L290

This appears to be providing some lazy support for a broken configuration?
Regardless, this behavior should not be supported by auth_token (and as
explained above, keystone doesn't support it). I opened a bug to track a
fix:

  https://bugs.launchpad.net/python-keystoneclient/+bug/1207517

auth_uri => point at public API endpoint in support of use case #1 above

auth_host, auth_port, auth, auth_protocol, auth_admin_prefix => point at
admin API in support of use case #2 above

Documentation fixes for the above, plus a warning log if auth_uri is not
set:

  https://review.openstack.org/#/c/39796/


> I'm having a heck of a time tracking down the source of this bug:
>
> https://bugs.launchpad.net/**nova/+bug/1206330
>
> And am trying to cross any possible configuration issues with the
> auth_token middleware off of my list of possible causes.
>
> Any and all insight would be most appreciated.
>
> Best,
> -jay
>
> __**_
> OpenStack-dev mailing list
> OpenStack-dev@lists.openstack.**org 
> http://lists.openstack.org/**cgi-bin/mailman/listinfo/**openstack-dev
>



-- 

-Dolph
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


[openstack-dev] [Cinder] Volume Encryption flow between Nova and Cinder

2013-08-01 Thread Benjamin, Bruce P.
In support of some questions we've received on the Volume Encryption code 
review 'Add encryption metadata to volume table' 
https://review.openstack.org/#/c/30974/ , the following etherpad contains flow 
details on Cinder and Nova interactions: 
https://etherpad.openstack.org/f1Fr9QqjHt .  Comments and questions are 
welcomed.  You can respond on the code review link, on this list, or directly 
on the etherpad.  Thanks.

Bruce Benjamin
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


[openstack-dev] OpenStack Cinder List Snapshots Support

2013-08-01 Thread Mikhail Khodos
The Cinder Support Matrix
https://wiki.openstack.org/wiki/CinderSupportMatrix, states that snapshot
listing is implemented in drivers by: LVM, EMC, NetApp, IBM, etc. However,
I could not find any methods or interfaces in the cinder volume api that
implement snapshot listing. Could anybody clarify if snapshot listing is
supported only via DB?
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


[openstack-dev] [Marconi] Agenda for tomorrow's meeting at 16:00 UTC

2013-08-01 Thread Flavio Percoco

The Marconi project team holds a weekly meeting in #openstack-meeting-alt
on Thursdays, 16:00 UTC.

The next meeting is tomorrow, August 02.

Everyone is welcome. However, please take a minute to review the wiki
before attending for the first time:

  http://wiki.openstack.org/marconi

## Agenda (60 mins): ##
* Review actions from last time
* Blueprint triage H-3
* Tests re-factor and improvements. 
* Placement Service[0]

* Open Discussion

[0] https://wiki.openstack.org/wiki/Marconi/bp/placement-service

See also:

  http://wiki.openstack.org/Meetings/Marconi


Cheers,
FF

--
@flaper87
Flavio Percoco

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Openstack] devstack with neutron/quantum

2013-08-01 Thread James Kyle
Did the trick, thanks!

I was in the process of catching the unhanded exception in legacy.py to see 
what was going. Definitely saved me some time.

cheers,

-james

On Aug 1, 2013, at 10:03 AM, Kyle Mestery (kmestery)  wrote:

> On Aug 1, 2013, at 11:56 AM, Eleouet Francois 
> wrote:
>> Hi,
>> 
>> The issue is tracked here: https://bugs.launchpad.net/neutron/+bug/1206013
>> 
>> s/cfg.NoSuchOptError/AttributeError/ in neutron/agent/linux/ip_lib.py
>> worked well for me.
>> 
> I just replied on another thread for this, but this worked for me too:
> 
> This fixed it for me:
> 
> cd /usr/local/lib/python2.7/dist-packages
> sudo rm -rf oslo*
> sudo pip install --upgrade 
> http://tarballs.openstack.org/oslo.config/oslo.config-1.2.0a3.tar.gz#egg=oslo.config-1.2.0a3
> 
> Thanks,
> Kyle
> 
> 
>> Francois.
>> 
>> 2013/8/1 Kyle Mestery (kmestery) :
>>> 
>>> On Jul 31, 2013, at 2:28 PM, Remo Mattei  wrote:
>>> 
 Hi everyone, I wonder if anyone has seen this error where neutron does
 not start. here is the error I get with devstack. Any suggestions are
 welcomed.
 
 
 Remo
 
 remo@openstack-ub:~/devstack$ cd /opt/stack/neutron && python =
 /usr/local/bin/neutron-l3-agent --config-file /etc/neutron/neutron.conf =
 --config-file=3D/etc/neutron/l3_agent.ini || touch =
 "/opt/stack/status/stack/q-l3.failure"
 2013-07-31 13:05:07.698 18378 INFO neutron.common.config [-] Logging =
 enabled!
 2013-07-31 13:05:07.703 18378 ERROR neutron.common.legacy [-] Skipping =
 unknown group key: firewall_driver
 2013-07-31 13:05:07.703 18378 DEBUG neutron.common.utils [-] Reloading =
 cached file /etc/neutron/policy.json read_cached_file =
 /opt/stack/neutron/neutron/common/utils.py:55
 2013-07-31 13:05:07.704 18378 DEBUG neutron.policy [-] loading policies =
 from file: /etc/neutron/policy.json _set_rules =
 /opt/stack/neutron/neutron/policy.py:88
 2013-07-31 13:05:07.705 18378 CRITICAL neutron [-]
 2013-07-31 13:05:07.705 18378 TRACE neutron Traceback (most recent call =
 last):
 2013-07-31 13:05:07.705 18378 TRACE neutron   File =
 "/usr/local/bin/neutron-l3-agent", line 10, in 
 2013-07-31 13:05:07.705 18378 TRACE neutron sys.exit(main())
 2013-07-31 13:05:07.705 18378 TRACE neutron   File =
 "/opt/stack/neutron/neutron/agent/l3_agent.py", line 833, in main
 2013-07-31 13:05:07.705 18378 TRACE neutron =
 manager=3D'neutron.agent.l3_agent.L3NATAgentWithStateReport')
 2013-07-31 13:05:07.705 18378 TRACE neutron   File =
 "/opt/stack/neutron/neutron/service.py", line 204, in create
 2013-07-31 13:05:07.705 18378 TRACE neutron =
 periodic_fuzzy_delay=3Dperiodic_fuzzy_delay)
 2013-07-31 13:05:07.705 18378 TRACE neutron   File =
 "/opt/stack/neutron/neutron/service.py", line 137, in __init__
 2013-07-31 13:05:07.705 18378 TRACE neutron self.manager =3D =
 manager_class(host=3Dhost, *args, **kwargs)
 2013-07-31 13:05:07.705 18378 TRACE neutron   File =
 "/opt/stack/neutron/neutron/agent/l3_agent.py", line 756, in __init__
 2013-07-31 13:05:07.705 18378 TRACE neutron =
 super(L3NATAgentWithStateReport, self).__init__(host=3Dhost, conf=3Dconf)
 2013-07-31 13:05:07.705 18378 TRACE neutron   File =
 "/opt/stack/neutron/neutron/agent/l3_agent.py", line 213, in __init__
 2013-07-31 13:05:07.705 18378 TRACE neutron =
 self._destroy_router_namespaces(self.conf.router_id)
 2013-07-31 13:05:07.705 18378 TRACE neutron   File =
 "/opt/stack/neutron/neutron/agent/l3_agent.py", line 228, in =
 _destroy_router_namespaces
 2013-07-31 13:05:07.705 18378 TRACE neutron root_ip =3D =
 ip_lib.IPWrapper(self.root_helper)
 2013-07-31 13:05:07.705 18378 TRACE neutron   File =
 "/opt/stack/neutron/neutron/agent/linux/ip_lib.py", line 82, in __init__
 2013-07-31 13:05:07.705 18378 TRACE neutron namespace=3Dnamespace)
 2013-07-31 13:05:07.705 18378 TRACE neutron   File =
 "/opt/stack/neutron/neutron/agent/linux/ip_lib.py", line 39, in __init__
 2013-07-31 13:05:07.705 18378 TRACE neutron self.force_root =3D =
 cfg.CONF.ip_lib_force_root
 2013-07-31 13:05:07.705 18378 TRACE neutron   File =
 "/opt/stack/oslo.config/oslo/config/cfg.py", line 1626, in __getattr__
 2013-07-31 13:05:07.705 18378 TRACE neutron raise AttributeError
 2013-07-31 13:05:07.705 18378 TRACE neutron AttributeError
 2013-07-31 13:05:07.705 18378 TRACE neutron=
 
>>> I'm seeing this error with the latest neutron and devstack as well, and I
>>> haven't figured out a way around it now. Moving this to the openstack-dev
>>> list to get some additional eyes on this as well.
>>> 
>>> Kyle
>>> 
 ___
 Mailing list: http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
 Post to : openst...@lists.openstack.org
 Unsubscribe : http://lists.open

[openstack-dev] [savanna] team meeting minutes August 1

2013-08-01 Thread Sergey Lukjanov
Thanks everyone who have joined today's Savanna meeting.

Here are the logs from the meeting:

Minutes: 
http://eavesdrop.openstack.org/meetings/savanna/2013/savanna.2013-08-01-18.06.html
Minutes (text): 
http://eavesdrop.openstack.org/meetings/savanna/2013/savanna.2013-08-01-18.06.txt
Log: 
http://eavesdrop.openstack.org/meetings/savanna/2013/savanna.2013-08-01-18.06.log.html

Sincerely yours,
Sergey Lukjanov
Savanna Technical Lead
Mirantis Inc.


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] Separate Migration Repos

2013-08-01 Thread Dolph Mathews
On Thu, Aug 1, 2013 at 12:28 PM, Adam Young  wrote:

>  On 08/01/2013 11:40 AM, Dolph Mathews wrote:
>
>
>  On Wed, Jul 31, 2013 at 5:00 AM, Henry Nash wrote:
>
>> Hi Adam,
>>
>> Wanted to just give you more detail on the issue I keep pressing on for
>> your change (https://review.openstack.org/#/c/36731/).
>>
>> For extensions which create their own "private" tables, I totally get it.
>>  I'd like, however, to understand what happens for a more complex
>> extension.  Let's imagine an (only-partially) hypothetical example of an
>> extension that does (all of) the following:
>>
>> 1) It adds or changes the use of some columns in existing core tables,
>> and has migrations and code that goes along with that.
>> 2) It adds a new "private" table, and has all the code to handle that
>>
>
>  I see the need for quotations here as a big red flag: what exactly is
> the use case we're solving for? The quotes imply to me that we're simply
> moving code around within the git repo as a refactor with no real gain.
>
>
> The code in question was a table that linked endpoints to projects.
>

This won't flay, as both endpoints nad projects are in separate backends.
>

You lost me at "flay."

Theoretically, both could come from a backing store other than SQL.  The
> solution to this particular problem is:
>
> As an extension, put it in a table with values to that point to the public
> IDs for endspoints and projects.
> xor
> Make the table part of the assignments backend, and give it a fk
> constraint to proejcts
> xor
> Make the table part of the catalog backend and give it a fk constraint to
> endpoints.
>
> As it is going in as an extension, it has to be the first option.
>

+1


>
>

>
>
>
>
>  If the goal is to allow an extension to back to some sql store that is
> not shared by the rest of keystone (which is I thought how this
> conversation started?), then I'm not clear on how separate migration repos
> would be the first step in that direction. If the extension owns it's own
> sql store and therefore it's sqlalchemy engine, only then does it makes
> sense (to me) to bundle the extension with it's own migration repo. That
> would make "private" tables *actually *private, given that they'd be
> configurable with unique sqlalchemy connection strings, etc.
>
>
> What it does it splits the migration version for the extension from the
> rest of Keystone.
>

That's the refactor I'm referring to above.


> We could drop the Endpoint table all together, but the endpoint-filtering
> plugin would still work.  Core tables don't care about extensions.
>

That's how it is today with a single migration repo.


>
> It will also greatly reduce the rebase ovehead of database migrations
> patches.
>

Sure, but it's not very common to require much more than changing the
version number of the newest migration. Or in the land of alembic, merge
back to a linear series of changes. Either way, the overhead is typically
minimal.


>
> My patch does run the migrations for all extension, but it does not have
> to do so.  It might make sense, either as part of this patch or as a follow
> on, to provide command line switches to show the set of extensions that
> have repos, and to show the versions of those extensions in the database
> pointed to by the connection string.  There is another blueprint for
> supporting multiple SQL sources, and, with that, we could potentially map
> extension to different databases than the core repo.  We can also add a
> command line parameter that explicitly specifies the extension for which to
> run the migrations.
>
>
>
>
>
>> 3) New APIs etc. to create new REST calls to drive the extension
>>
>> It is part 1) in the above that I am trying to understand how we would
>> implement in this new model.  What I am imagining is that the best way to
>> do 1) is that you would break (at least part of it) out of the extension
>> and it would be a core patch.  This would cover modifications to core
>> columns and changing any core code to make sure that such changes were
>> benign to the rest of core (and indeed any other extensions).  Migrations
>> for this part of the schema change would be in the core repo.  Our new
>> extension would then build on this, have its private new table in its own
>> repo and any unique code in contrib. Is that how you imagined this working?
>>
>> This hypothetical example is, of course, not too far from reality - the
>> recent change I did for  inherited roles (
>> https://review.openstack.org/#/c/35986/) is an example that comes close
>> to the above - and it would seem to me that it would be much safer (from a
>> code dependency point of view) to have the DB changes done separately and
>> integrated into core - and the extensions just, in this case, use the
>> advantages of the new schema to provide its functionality.
>>
>> Henry
>>
>>
>> ___
>> OpenStack-dev mailing list
>> OpenStack-dev@lists.openstack.org
>> http://lists.openstack.org/cgi-bin/

Re: [openstack-dev] [Horizon] how to add unmerged dependency to test-requirements

2013-08-01 Thread Gabriel Hurley
The short answer is: "you can test and develop it locally but you cannot push 
it upstream until you get the dependencies released." As much as that may be 
frustrating, it prevents an enormous amount of pain which OpenStack has gone 
through in the past when things fail to be released as expected.

 Please work with the Neutron team to get their end of things done first.

Thanks!

- Gabriel

> -Original Message-
> From: Monty Taylor [mailto:mord...@inaugust.com]
> Sent: Wednesday, July 31, 2013 10:29 PM
> To: openstack-dev@lists.openstack.org
> Subject: Re: [openstack-dev] [Horizon] how to add unmerged dependency
> to test-requirements
> 
> Hi!
> 
> Not really. We don't support OpenStack projects having dependencies on
> unreleased software. What does fwaas come from? Is this part of neutron?
> 
> On 08/01/2013 01:24 AM, Kuang-Ching Wang wrote:
> > Hi, I am working on fwaas Horizon support, which requires fwaas CLI to
> work.  Since fwaas CLI has not been merged (though it is functional), is there
> anyway I can specify the dependency in horizon/test-requirements file, such
> that Jenkins would be able to run for my horizon patch.
> >
> > Thanks!
> > KC
> > ___
> > OpenStack-dev mailing list
> > OpenStack-dev@lists.openstack.org
> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> >
> 
> ___
> OpenStack-dev mailing list
> OpenStack-dev@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Openstack] devstack with neutron/quantum

2013-08-01 Thread Sean Dague

On 08/01/2013 01:03 PM, Kyle Mestery (kmestery) wrote:

On Aug 1, 2013, at 11:56 AM, Eleouet Francois 
  wrote:

Hi,

The issue is tracked here: https://bugs.launchpad.net/neutron/+bug/1206013

s/cfg.NoSuchOptError/AttributeError/ in neutron/agent/linux/ip_lib.py
worked well for me.


I just replied on another thread for this, but this worked for me too:

This fixed it for me:

cd /usr/local/lib/python2.7/dist-packages
sudo rm -rf oslo*
sudo pip install --upgrade 
http://tarballs.openstack.org/oslo.config/oslo.config-1.2.0a3.tar.gz#egg=oslo.config-1.2.0a3


So the right thing to do is get a patch into neutron requirements.txt to 
match the way it is in nova.


That being said, in devstack, this is fixed as of yesterday, as we're 
using oslo.config from git now.


That won't help people not using devstack though.

-Sean

--
Sean Dague
http://dague.net

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] Separate Migration Repos

2013-08-01 Thread Adam Young

On 08/01/2013 11:40 AM, Dolph Mathews wrote:


On Wed, Jul 31, 2013 at 5:00 AM, Henry Nash > wrote:


Hi Adam,

Wanted to just give you more detail on the issue I keep pressing
on for your change (https://review.openstack.org/#/c/36731/).

For extensions which create their own "private" tables, I totally
get it.  I'd like, however, to understand what happens for a more
complex extension.  Let's imagine an (only-partially) hypothetical
example of an extension that does (all of) the following:

1) It adds or changes the use of some columns in existing core
tables, and has migrations and code that goes along with that.
2) It adds a new "private" table, and has all the code to handle that


I see the need for quotations here as a big red flag: what exactly is 
the use case we're solving for? The quotes imply to me that we're 
simply moving code around within the git repo as a refactor with no 
real gain.


The code in question was a table that linked endpoints to projects. This 
won't flay, as both endpoints nad projects are in separate backends.  
Theoretically, both could come from a backing store other than SQL.  The 
solution to this particular problem is:


As an extension, put it in a table with values to that point to the 
public IDs for endspoints and projects.

xor
Make the table part of the assignments backend, and give it a fk 
constraint to proejcts

xor
Make the table part of the catalog backend and give it a fk constraint 
to endpoints.


As it is going in as an extension, it has to be the first option.





If the goal is to allow an extension to back to some sql store that is 
not shared by the rest of keystone (which is I thought how this 
conversation started?), then I'm not clear on how separate migration 
repos would be the first step in that direction. If the extension owns 
it's own sql store and therefore it's sqlalchemy engine, only then 
does it makes sense (to me) to bundle the extension with it's own 
migration repo. That would make "private" tables /actually /private, 
given that they'd be configurable with unique sqlalchemy connection 
strings, etc.


What it does it splits the migration version for the extension from the 
rest of Keystone.  We could drop the Endpoint table all together, but 
the endpoint-filtering plugin would still work.  Core tables don't care 
about extensions.


It will also greatly reduce the rebase ovehead of database migrations 
patches.


My patch does run the migrations for all extension, but it does not have 
to do so.  It might make sense, either as part of this patch or as a 
follow on, to provide command line switches to show the set of 
extensions that have repos, and to show the versions of those extensions 
in the database pointed to by the connection string. There is another 
blueprint for supporting multiple SQL sources, and, with that, we could 
potentially map extension to different databases than the core repo.  We 
can also add a command line parameter that explicitly specifies the 
extension for which to run the migrations.




3) New APIs etc. to create new REST calls to drive the extension

It is part 1) in the above that I am trying to understand how we
would implement in this new model.  What I am imagining is that
the best way to do 1) is that you would break (at least part of
it) out of the extension and it would be a core patch.  This would
cover modifications to core columns and changing any core code to
make sure that such changes were benign to the rest of core (and
indeed any other extensions).  Migrations for this part of the
schema change would be in the core repo.  Our new extension would
then build on this, have its private new table in its own repo and
any unique code in contrib. Is that how you imagined this working?

This hypothetical example is, of course, not too far from reality
- the recent change I did for  inherited roles
(https://review.openstack.org/#/c/35986/) is an example that comes
close to the above - and it would seem to me that it would be much
safer (from a code dependency point of view) to have the DB
changes done separately and integrated into core - and the
extensions just, in this case, use the advantages of the new
schema to provide its functionality.

Henry


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org

http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev




--

-Dolph


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack

Re: [openstack-dev] [CI] Now gating on Python 3.3

2013-08-01 Thread Jeremy Stanley
On 2013-08-01 14:43:48 +0300 (+0300), Roman Prykhodchenko wrote:
[...]
> It would be very helpful to mention those projects.

Sure! For the past day or so, openstack/oslo.config and
openstack-dev/pbr have been gating their unit tests on Python 3.3
(I've been told this even caught a potential py3k incompatibility in
a proposed change to pbr yesterday).

The goal in targeting the most common lower-level dependencies like
these first is, of course, so that developers interested in py3k
support can climb their way up the tree in a slightly more efficient
manner without branches falling out from under them (at least where
the dependencies under our direct control are concerned). This also
means we'll probably have to start pinning third-party dependencies
in the future if they temporarily break py3k compatibility.
-- 
Jeremy Stanley

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Openstack] devstack with neutron/quantum

2013-08-01 Thread Kyle Mestery (kmestery)
On Aug 1, 2013, at 11:56 AM, Eleouet Francois 
 wrote:
> Hi,
> 
> The issue is tracked here: https://bugs.launchpad.net/neutron/+bug/1206013
> 
> s/cfg.NoSuchOptError/AttributeError/ in neutron/agent/linux/ip_lib.py
> worked well for me.
> 
I just replied on another thread for this, but this worked for me too:

This fixed it for me:

cd /usr/local/lib/python2.7/dist-packages
sudo rm -rf oslo*
sudo pip install --upgrade 
http://tarballs.openstack.org/oslo.config/oslo.config-1.2.0a3.tar.gz#egg=oslo.config-1.2.0a3

Thanks,
Kyle


> Francois.
> 
> 2013/8/1 Kyle Mestery (kmestery) :
>> 
>> On Jul 31, 2013, at 2:28 PM, Remo Mattei  wrote:
>> 
>>> Hi everyone, I wonder if anyone has seen this error where neutron does
>>> not start. here is the error I get with devstack. Any suggestions are
>>> welcomed.
>>> 
>>> 
>>> Remo
>>> 
>>> remo@openstack-ub:~/devstack$ cd /opt/stack/neutron && python =
>>> /usr/local/bin/neutron-l3-agent --config-file /etc/neutron/neutron.conf =
>>> --config-file=3D/etc/neutron/l3_agent.ini || touch =
>>> "/opt/stack/status/stack/q-l3.failure"
>>> 2013-07-31 13:05:07.698 18378 INFO neutron.common.config [-] Logging =
>>> enabled!
>>> 2013-07-31 13:05:07.703 18378 ERROR neutron.common.legacy [-] Skipping =
>>> unknown group key: firewall_driver
>>> 2013-07-31 13:05:07.703 18378 DEBUG neutron.common.utils [-] Reloading =
>>> cached file /etc/neutron/policy.json read_cached_file =
>>> /opt/stack/neutron/neutron/common/utils.py:55
>>> 2013-07-31 13:05:07.704 18378 DEBUG neutron.policy [-] loading policies =
>>> from file: /etc/neutron/policy.json _set_rules =
>>> /opt/stack/neutron/neutron/policy.py:88
>>> 2013-07-31 13:05:07.705 18378 CRITICAL neutron [-]
>>> 2013-07-31 13:05:07.705 18378 TRACE neutron Traceback (most recent call =
>>> last):
>>> 2013-07-31 13:05:07.705 18378 TRACE neutron   File =
>>> "/usr/local/bin/neutron-l3-agent", line 10, in 
>>> 2013-07-31 13:05:07.705 18378 TRACE neutron sys.exit(main())
>>> 2013-07-31 13:05:07.705 18378 TRACE neutron   File =
>>> "/opt/stack/neutron/neutron/agent/l3_agent.py", line 833, in main
>>> 2013-07-31 13:05:07.705 18378 TRACE neutron =
>>> manager=3D'neutron.agent.l3_agent.L3NATAgentWithStateReport')
>>> 2013-07-31 13:05:07.705 18378 TRACE neutron   File =
>>> "/opt/stack/neutron/neutron/service.py", line 204, in create
>>> 2013-07-31 13:05:07.705 18378 TRACE neutron =
>>> periodic_fuzzy_delay=3Dperiodic_fuzzy_delay)
>>> 2013-07-31 13:05:07.705 18378 TRACE neutron   File =
>>> "/opt/stack/neutron/neutron/service.py", line 137, in __init__
>>> 2013-07-31 13:05:07.705 18378 TRACE neutron self.manager =3D =
>>> manager_class(host=3Dhost, *args, **kwargs)
>>> 2013-07-31 13:05:07.705 18378 TRACE neutron   File =
>>> "/opt/stack/neutron/neutron/agent/l3_agent.py", line 756, in __init__
>>> 2013-07-31 13:05:07.705 18378 TRACE neutron =
>>> super(L3NATAgentWithStateReport, self).__init__(host=3Dhost, conf=3Dconf)
>>> 2013-07-31 13:05:07.705 18378 TRACE neutron   File =
>>> "/opt/stack/neutron/neutron/agent/l3_agent.py", line 213, in __init__
>>> 2013-07-31 13:05:07.705 18378 TRACE neutron =
>>> self._destroy_router_namespaces(self.conf.router_id)
>>> 2013-07-31 13:05:07.705 18378 TRACE neutron   File =
>>> "/opt/stack/neutron/neutron/agent/l3_agent.py", line 228, in =
>>> _destroy_router_namespaces
>>> 2013-07-31 13:05:07.705 18378 TRACE neutron root_ip =3D =
>>> ip_lib.IPWrapper(self.root_helper)
>>> 2013-07-31 13:05:07.705 18378 TRACE neutron   File =
>>> "/opt/stack/neutron/neutron/agent/linux/ip_lib.py", line 82, in __init__
>>> 2013-07-31 13:05:07.705 18378 TRACE neutron namespace=3Dnamespace)
>>> 2013-07-31 13:05:07.705 18378 TRACE neutron   File =
>>> "/opt/stack/neutron/neutron/agent/linux/ip_lib.py", line 39, in __init__
>>> 2013-07-31 13:05:07.705 18378 TRACE neutron self.force_root =3D =
>>> cfg.CONF.ip_lib_force_root
>>> 2013-07-31 13:05:07.705 18378 TRACE neutron   File =
>>> "/opt/stack/oslo.config/oslo/config/cfg.py", line 1626, in __getattr__
>>> 2013-07-31 13:05:07.705 18378 TRACE neutron raise AttributeError
>>> 2013-07-31 13:05:07.705 18378 TRACE neutron AttributeError
>>> 2013-07-31 13:05:07.705 18378 TRACE neutron=
>>> 
>> I'm seeing this error with the latest neutron and devstack as well, and I
>> haven't figured out a way around it now. Moving this to the openstack-dev
>> list to get some additional eyes on this as well.
>> 
>> Kyle
>> 
>>> ___
>>> Mailing list: http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
>>> Post to : openst...@lists.openstack.org
>>> Unsubscribe : http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
>> 
>> 
>> ___
>> OpenStack-dev mailing list
>> OpenStack-dev@lists.openstack.org
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> 
> ___
> OpenStack-dev mailing list
> OpenStack-dev@lists.o

Re: [openstack-dev] [Neutron] devstack + neutron fails on firewall_driver

2013-08-01 Thread Kyle Mestery (kmestery)
On Aug 1, 2013, at 11:51 AM, James Kyle  wrote:

> Morning,
> 
> I'm having some issues getting devstack + neutron going. 
> 
> If I don't use Q_USE_DEBUG_COMMAND, it completes and notes that the l3 agent 
> has failed to start. Here's a paste of the stack.sh error and the stack trace 
> from q-l3 => https://gist.github.com/jameskyle/6133049
> 
> If I do use the debug command, this is the error/logs I get: 
> https://gist.github.com/jameskyle/6133125
> 
> And this is my localrc https://gist.github.com/jameskyle/6133134
> 
> Currently running on an ubuntu 12.04 vm. 
> 
> The behavior is somewhat recent, last week or so. But I can't find any 
> references to the issue in bugs.
> 
> Thanks for any input!
> 
This fixed it for me:

cd /usr/local/lib/python2.7/dist-packages
sudo rm -rf oslo*
sudo pip install --upgrade 
http://tarballs.openstack.org/oslo.config/oslo.config-1.2.0a3.tar.gz#egg=oslo.config-1.2.0a3

Thanks,
Kyle

> Cheers,
> 
> -james
> ___
> OpenStack-dev mailing list
> OpenStack-dev@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Openstack] devstack with neutron/quantum

2013-08-01 Thread Eleouet Francois
Hi,

The issue is tracked here: https://bugs.launchpad.net/neutron/+bug/1206013

s/cfg.NoSuchOptError/AttributeError/ in neutron/agent/linux/ip_lib.py
worked well for me.

Francois.

2013/8/1 Kyle Mestery (kmestery) :
>
> On Jul 31, 2013, at 2:28 PM, Remo Mattei  wrote:
>
>> Hi everyone, I wonder if anyone has seen this error where neutron does
>> not start. here is the error I get with devstack. Any suggestions are
>> welcomed.
>>
>>
>> Remo
>>
>> remo@openstack-ub:~/devstack$ cd /opt/stack/neutron && python =
>> /usr/local/bin/neutron-l3-agent --config-file /etc/neutron/neutron.conf =
>> --config-file=3D/etc/neutron/l3_agent.ini || touch =
>> "/opt/stack/status/stack/q-l3.failure"
>> 2013-07-31 13:05:07.698 18378 INFO neutron.common.config [-] Logging =
>> enabled!
>> 2013-07-31 13:05:07.703 18378 ERROR neutron.common.legacy [-] Skipping =
>> unknown group key: firewall_driver
>> 2013-07-31 13:05:07.703 18378 DEBUG neutron.common.utils [-] Reloading =
>> cached file /etc/neutron/policy.json read_cached_file =
>> /opt/stack/neutron/neutron/common/utils.py:55
>> 2013-07-31 13:05:07.704 18378 DEBUG neutron.policy [-] loading policies =
>> from file: /etc/neutron/policy.json _set_rules =
>> /opt/stack/neutron/neutron/policy.py:88
>> 2013-07-31 13:05:07.705 18378 CRITICAL neutron [-]
>> 2013-07-31 13:05:07.705 18378 TRACE neutron Traceback (most recent call =
>> last):
>> 2013-07-31 13:05:07.705 18378 TRACE neutron   File =
>> "/usr/local/bin/neutron-l3-agent", line 10, in 
>> 2013-07-31 13:05:07.705 18378 TRACE neutron sys.exit(main())
>> 2013-07-31 13:05:07.705 18378 TRACE neutron   File =
>> "/opt/stack/neutron/neutron/agent/l3_agent.py", line 833, in main
>> 2013-07-31 13:05:07.705 18378 TRACE neutron =
>> manager=3D'neutron.agent.l3_agent.L3NATAgentWithStateReport')
>> 2013-07-31 13:05:07.705 18378 TRACE neutron   File =
>> "/opt/stack/neutron/neutron/service.py", line 204, in create
>> 2013-07-31 13:05:07.705 18378 TRACE neutron =
>> periodic_fuzzy_delay=3Dperiodic_fuzzy_delay)
>> 2013-07-31 13:05:07.705 18378 TRACE neutron   File =
>> "/opt/stack/neutron/neutron/service.py", line 137, in __init__
>> 2013-07-31 13:05:07.705 18378 TRACE neutron self.manager =3D =
>> manager_class(host=3Dhost, *args, **kwargs)
>> 2013-07-31 13:05:07.705 18378 TRACE neutron   File =
>> "/opt/stack/neutron/neutron/agent/l3_agent.py", line 756, in __init__
>> 2013-07-31 13:05:07.705 18378 TRACE neutron =
>> super(L3NATAgentWithStateReport, self).__init__(host=3Dhost, conf=3Dconf)
>> 2013-07-31 13:05:07.705 18378 TRACE neutron   File =
>> "/opt/stack/neutron/neutron/agent/l3_agent.py", line 213, in __init__
>> 2013-07-31 13:05:07.705 18378 TRACE neutron =
>> self._destroy_router_namespaces(self.conf.router_id)
>> 2013-07-31 13:05:07.705 18378 TRACE neutron   File =
>> "/opt/stack/neutron/neutron/agent/l3_agent.py", line 228, in =
>> _destroy_router_namespaces
>> 2013-07-31 13:05:07.705 18378 TRACE neutron root_ip =3D =
>> ip_lib.IPWrapper(self.root_helper)
>> 2013-07-31 13:05:07.705 18378 TRACE neutron   File =
>> "/opt/stack/neutron/neutron/agent/linux/ip_lib.py", line 82, in __init__
>> 2013-07-31 13:05:07.705 18378 TRACE neutron namespace=3Dnamespace)
>> 2013-07-31 13:05:07.705 18378 TRACE neutron   File =
>> "/opt/stack/neutron/neutron/agent/linux/ip_lib.py", line 39, in __init__
>> 2013-07-31 13:05:07.705 18378 TRACE neutron self.force_root =3D =
>> cfg.CONF.ip_lib_force_root
>> 2013-07-31 13:05:07.705 18378 TRACE neutron   File =
>> "/opt/stack/oslo.config/oslo/config/cfg.py", line 1626, in __getattr__
>> 2013-07-31 13:05:07.705 18378 TRACE neutron raise AttributeError
>> 2013-07-31 13:05:07.705 18378 TRACE neutron AttributeError
>> 2013-07-31 13:05:07.705 18378 TRACE neutron=
>>
> I'm seeing this error with the latest neutron and devstack as well, and I
> haven't figured out a way around it now. Moving this to the openstack-dev
> list to get some additional eyes on this as well.
>
> Kyle
>
>> ___
>> Mailing list: http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
>> Post to : openst...@lists.openstack.org
>> Unsubscribe : http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
>
>
> ___
> OpenStack-dev mailing list
> OpenStack-dev@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


[openstack-dev] [Neutron] devstack + neutron fails on firewall_driver

2013-08-01 Thread James Kyle
Morning,

I'm having some issues getting devstack + neutron going. 

If I don't use Q_USE_DEBUG_COMMAND, it completes and notes that the l3 agent 
has failed to start. Here's a paste of the stack.sh error and the stack trace 
from q-l3 => https://gist.github.com/jameskyle/6133049

If I do use the debug command, this is the error/logs I get: 
https://gist.github.com/jameskyle/6133125

And this is my localrc https://gist.github.com/jameskyle/6133134

Currently running on an ubuntu 12.04 vm. 

The behavior is somewhat recent, last week or so. But I can't find any 
references to the issue in bugs.

Thanks for any input!

Cheers,

-james
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] Python overhead for rootwrap

2013-08-01 Thread Joe Gordon
On Aug 1, 2013 2:06 AM, "Thierry Carrez"  wrote:
>
> Joe Gordon wrote:
> > I tried
> > swapping out rootwrap for sudo and that made the issue go away.So I
> > think we should go back to supporting just using sudo instead of
> > rootwrap, and make sure any future solutions support a sudo only option
> > as well.  But I am open to other ideas, I just think we need to
> > implement something for Havana.
>
> How did you make the swap ? Patched utils.execute ? I guess we could
> introduce some use_sudo_as_root_helper parameter (defaulting to False,
> with security warnings) and preserve rootwrap_config as it is...
>

Yes, or something like that.  Hopefully it would be something in Oslo so
other projects have this option. If they use rootwrap as well.

Having rootwrap on by default makes nova-network scale very poorly by
default.  Which doesn't sound like a good default, but not sure if no
rootwrap is a better default. So we may need a better story for deployers
...

I haven't tried neutron perf yet but that may be better?  But it won't be
default until icehouse it looks like.

> It will require a passwordless blanket sudo access for the nova user.

Can't we go back to having a sudoers file white listing which binaries it
can call, like before?

> The trick is that this is equivalent to running Nova as the root user,
> from a privilege escalation / security domain standpoint. And you can
> already do that, and it will also "make the issue go away" (rootwrap is
> bypassed if called from euid 0).
>
> It's cleaner to implement it as an option though, as it will not screw
> up permissions everywhere in case you want to go back to using rootwrap
> at some point in the future.
>
> Another important take-away from your comment is that we should preserve
> the ability to run those commands under direct sudo... so if we want to
> allow executing python snippets within rootwrap, we'd need to find a way
> to do it with a call that is sudo-compatible, which might prove a bit
> tricky.
>
> --
> Thierry Carrez (ttx)
>
> ___
> OpenStack-dev mailing list
> OpenStack-dev@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Openstack] devstack with neutron/quantum

2013-08-01 Thread Kyle Mestery (kmestery)

On Jul 31, 2013, at 2:28 PM, Remo Mattei  wrote:

> Hi everyone, I wonder if anyone has seen this error where neutron does 
> not start. here is the error I get with devstack. Any suggestions are 
> welcomed.
> 
> 
> Remo
> 
> remo@openstack-ub:~/devstack$ cd /opt/stack/neutron && python =
> /usr/local/bin/neutron-l3-agent --config-file /etc/neutron/neutron.conf =
> --config-file=3D/etc/neutron/l3_agent.ini || touch =
> "/opt/stack/status/stack/q-l3.failure"
> 2013-07-31 13:05:07.698 18378 INFO neutron.common.config [-] Logging =
> enabled!
> 2013-07-31 13:05:07.703 18378 ERROR neutron.common.legacy [-] Skipping =
> unknown group key: firewall_driver
> 2013-07-31 13:05:07.703 18378 DEBUG neutron.common.utils [-] Reloading =
> cached file /etc/neutron/policy.json read_cached_file =
> /opt/stack/neutron/neutron/common/utils.py:55
> 2013-07-31 13:05:07.704 18378 DEBUG neutron.policy [-] loading policies =
> from file: /etc/neutron/policy.json _set_rules =
> /opt/stack/neutron/neutron/policy.py:88
> 2013-07-31 13:05:07.705 18378 CRITICAL neutron [-]
> 2013-07-31 13:05:07.705 18378 TRACE neutron Traceback (most recent call =
> last):
> 2013-07-31 13:05:07.705 18378 TRACE neutron   File =
> "/usr/local/bin/neutron-l3-agent", line 10, in 
> 2013-07-31 13:05:07.705 18378 TRACE neutron sys.exit(main())
> 2013-07-31 13:05:07.705 18378 TRACE neutron   File =
> "/opt/stack/neutron/neutron/agent/l3_agent.py", line 833, in main
> 2013-07-31 13:05:07.705 18378 TRACE neutron =
> manager=3D'neutron.agent.l3_agent.L3NATAgentWithStateReport')
> 2013-07-31 13:05:07.705 18378 TRACE neutron   File =
> "/opt/stack/neutron/neutron/service.py", line 204, in create
> 2013-07-31 13:05:07.705 18378 TRACE neutron =
> periodic_fuzzy_delay=3Dperiodic_fuzzy_delay)
> 2013-07-31 13:05:07.705 18378 TRACE neutron   File =
> "/opt/stack/neutron/neutron/service.py", line 137, in __init__
> 2013-07-31 13:05:07.705 18378 TRACE neutron self.manager =3D =
> manager_class(host=3Dhost, *args, **kwargs)
> 2013-07-31 13:05:07.705 18378 TRACE neutron   File =
> "/opt/stack/neutron/neutron/agent/l3_agent.py", line 756, in __init__
> 2013-07-31 13:05:07.705 18378 TRACE neutron =
> super(L3NATAgentWithStateReport, self).__init__(host=3Dhost, conf=3Dconf)
> 2013-07-31 13:05:07.705 18378 TRACE neutron   File =
> "/opt/stack/neutron/neutron/agent/l3_agent.py", line 213, in __init__
> 2013-07-31 13:05:07.705 18378 TRACE neutron =
> self._destroy_router_namespaces(self.conf.router_id)
> 2013-07-31 13:05:07.705 18378 TRACE neutron   File =
> "/opt/stack/neutron/neutron/agent/l3_agent.py", line 228, in =
> _destroy_router_namespaces
> 2013-07-31 13:05:07.705 18378 TRACE neutron root_ip =3D =
> ip_lib.IPWrapper(self.root_helper)
> 2013-07-31 13:05:07.705 18378 TRACE neutron   File =
> "/opt/stack/neutron/neutron/agent/linux/ip_lib.py", line 82, in __init__
> 2013-07-31 13:05:07.705 18378 TRACE neutron namespace=3Dnamespace)
> 2013-07-31 13:05:07.705 18378 TRACE neutron   File =
> "/opt/stack/neutron/neutron/agent/linux/ip_lib.py", line 39, in __init__
> 2013-07-31 13:05:07.705 18378 TRACE neutron self.force_root =3D =
> cfg.CONF.ip_lib_force_root
> 2013-07-31 13:05:07.705 18378 TRACE neutron   File =
> "/opt/stack/oslo.config/oslo/config/cfg.py", line 1626, in __getattr__
> 2013-07-31 13:05:07.705 18378 TRACE neutron raise AttributeError
> 2013-07-31 13:05:07.705 18378 TRACE neutron AttributeError
> 2013-07-31 13:05:07.705 18378 TRACE neutron=
> 
I'm seeing this error with the latest neutron and devstack as well, and I
haven't figured out a way around it now. Moving this to the openstack-dev
list to get some additional eyes on this as well.

Kyle

> ___
> Mailing list: http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
> Post to : openst...@lists.openstack.org
> Unsubscribe : http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [CI] Now gating on Python 3.3

2013-08-01 Thread Russell Bryant
On 08/01/2013 08:02 AM, Sean Dague wrote:
> On 07/31/2013 11:18 PM, Jeremy Stanley wrote:
>> A quick announcement... thanks to the invaluable assistance of Chuck
>> Short, Dan Prince, Julien Danjou and others, some projects are now
>> being successfully tested against and gated on Python 3.3 (with even
>> more projects very close to ready for the same).
>>
>> As discussed previously in summit sessions and on this list,
>> libraries and clients whose various dependencies already fully
>> support Python 3.3 are great candidates for addition in the near
>> term. Feel free to reach out to the infrastructure team on the
>> Freenode IRC network in #openstack-infra or through the
>> openstack-in...@lists.openstack.org mailing list, should you require
>> assistance setting up tests for Python 3.3 compatibility in a
>> project.
>>
> 
> Nicely done folks!

Indeed!  This is fantastic progress.

-- 
Russell Bryant

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] Separate Migration Repos

2013-08-01 Thread Dolph Mathews
On Wed, Jul 31, 2013 at 5:00 AM, Henry Nash wrote:

> Hi Adam,
>
> Wanted to just give you more detail on the issue I keep pressing on for
> your change (https://review.openstack.org/#/c/36731/).
>
> For extensions which create their own "private" tables, I totally get it.
>  I'd like, however, to understand what happens for a more complex
> extension.  Let's imagine an (only-partially) hypothetical example of an
> extension that does (all of) the following:
>
> 1) It adds or changes the use of some columns in existing core tables, and
> has migrations and code that goes along with that.
> 2) It adds a new "private" table, and has all the code to handle that
>

I see the need for quotations here as a big red flag: what exactly is the
use case we're solving for? The quotes imply to me that we're simply moving
code around within the git repo as a refactor with no real gain.

If the goal is to allow an extension to back to some sql store that is not
shared by the rest of keystone (which is I thought how this conversation
started?), then I'm not clear on how separate migration repos would be the
first step in that direction. If the extension owns it's own sql store and
therefore it's sqlalchemy engine, only then does it makes sense (to me) to
bundle the extension with it's own migration repo. That would make
"private" tables *actually *private, given that they'd be configurable with
unique sqlalchemy connection strings, etc.


> 3) New APIs etc. to create new REST calls to drive the extension
>
> It is part 1) in the above that I am trying to understand how we would
> implement in this new model.  What I am imagining is that the best way to
> do 1) is that you would break (at least part of it) out of the extension
> and it would be a core patch.  This would cover modifications to core
> columns and changing any core code to make sure that such changes were
> benign to the rest of core (and indeed any other extensions).  Migrations
> for this part of the schema change would be in the core repo.  Our new
> extension would then build on this, have its private new table in its own
> repo and any unique code in contrib. Is that how you imagined this working?
>
> This hypothetical example is, of course, not too far from reality - the
> recent change I did for  inherited roles (
> https://review.openstack.org/#/c/35986/) is an example that comes close
> to the above - and it would seem to me that it would be much safer (from a
> code dependency point of view) to have the DB changes done separately and
> integrated into core - and the extensions just, in this case, use the
> advantages of the new schema to provide its functionality.
>
> Henry
>
>
> ___
> OpenStack-dev mailing list
> OpenStack-dev@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>



-- 

-Dolph
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] Openstack Service requirement

2013-08-01 Thread Ravi Chunduru
Monty,
 I had a quick look on ceilometer, it looks one place to get all
notifications.
I will deploy and try it out.

Thanks,
-Ravi.

On Wed, Jul 31, 2013 at 10:34 PM, Monty Taylor  wrote:

> Also, if you're wanting to buld services on top of OpenStack that want
> to respond to events - you probably want to look in to ceilometer, which
> has an interface to export such events to you.
>
> On 08/01/2013 01:02 AM, Addepalli Srini-B22160 wrote:
> > RPC  will send the notifications to the queues that have joined the
> > exchanges.
> >
> >
> >
> > Any notifications that were published before your service registers are
> > not seen by your service.
> >
> >
> >
> > Your service needs to get hold of the data using Nova and Neutron API.
> >
> >
> >
> > In summary,  when your service comes up,  in addition to registering for
> > notifications,  it is needed to get the existing data using
> > Nova/Neutrion API.
> >
> >
> >
> > Thanks
> >
> > SRini
> >
> >
> >
> > *From:*Ravi Chunduru [mailto:ravi...@gmail.com]
> > *Sent:* Wednesday, July 31, 2013 10:57 AM
> > *To:* OpenStack Development Mailing List
> > *Subject:* Re: [openstack-dev] Openstack Service requirement
> >
> >
> >
> > Hi,
> >
> >  Could any one respond on this? This helps us in designing services
> > written on top of openstack.
> >
> >
> >
> > Thanks,
> >
> > -Ravi.
> >
> > On Tue, Jul 30, 2013 at 7:38 PM, Ravi Chunduru  > > wrote:
> >
> > Hi,
> >
> >  I was designing a Openstack service that listens on notifications
> > generated from Nova, Neutron etc., for our custom needs to monitor and
> > act up on events.
> >
> > I have an interesting observation from openstack perspective and I would
> > like to know community thinking about it.
> >
> >
> >
> > By the time an openstack service starts, it would have missed some
> > notifications.  How do we deal with it?
> >
> >
> >
> > For example, Designate(DNSaaS) starts and by that time some VMs got
> > created, it would loose to update the DNS servers for those VMs.
> >
> > Should any openstack service such as Designate  call openstack APIs and
> > get existing configuration?
> >
> >
> >
> > Thanks,
> >
> > --
> > Ravi
> >
> >
> >
> >
> >
> > --
> > Ravi
> >
> >
> >
> > ___
> > OpenStack-dev mailing list
> > OpenStack-dev@lists.openstack.org
> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> >
>
> ___
> OpenStack-dev mailing list
> OpenStack-dev@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>



-- 
Ravi
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Glance] images tasks API -- final call for comments

2013-08-01 Thread Paul Bourke
Hi Brian,

I had a read over the summary wiki page and the individual docs.

My main thought was that although the asynchronous nature seems attractive,
the problems the new API is setting out to solve could be adapted to the
existing images API.  This view seems to be shared and well highlighted by
Jay in this thread:
http://lists.openstack.org/pipermail/openstack-dev/2013-May/009527.html

Was there any follow up to that mail? (sorry if I missed it!)

Thanks,
-Paul


On 30 July 2013 19:36, Brian Rosmaita  wrote:

> After lots of discussion, here are the image import/export/cloning
> blueprints unified as a tasks API.  Please reply with comments!
>
> Take a look at this doc first:
> https://wiki.openstack.org/wiki/Glance-tasks-api
> It's got a "related documents" section with links to the previous
> proposals and also to the earlier email list discussion.  It's also got
> links to the details for the import, export, and cloning tasks, which I
> guess I might as well paste here so you have 'em handy:
> https://wiki.openstack.org/wiki/Glance-tasks-import
> https://wiki.openstack.org/wiki/Glance-tasks-export
> https://wiki.openstack.org/wiki/Glance-tasks-clone
>
> Finally, here's a link to an experimental "product" page for this feature:
> https://wiki.openstack.org/wiki/Glance-tasks-api-product
>
> cheers,
> brian
>
> ___
> OpenStack-dev mailing list
> OpenStack-dev@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] Openstack Service requirement

2013-08-01 Thread Ravi Chunduru
Thanks. This will be expected behavior then.

-Ravi.

On Wed, Jul 31, 2013 at 10:02 PM, Addepalli Srini-B22160 <
b22...@freescale.com> wrote:

>  RPC  will send the notifications to the queues that have joined the
> exchanges.
>
> ** **
>
> Any notifications that were published before your service registers are
> not seen by your service.
>
> ** **
>
> Your service needs to get hold of the data using Nova and Neutron API.
>
> ** **
>
> In summary,  when your service comes up,  in addition to registering for
> notifications,  it is needed to get the existing data using Nova/Neutrion
> API.  
>
> ** **
>
> Thanks
>
> SRini
>
> ** **
>
> *From:* Ravi Chunduru [mailto:ravi...@gmail.com]
> *Sent:* Wednesday, July 31, 2013 10:57 AM
> *To:* OpenStack Development Mailing List
> *Subject:* Re: [openstack-dev] Openstack Service requirement
>
> ** **
>
> Hi,
>
>  Could any one respond on this? This helps us in designing services
> written on top of openstack.
>
> ** **
>
> Thanks,
>
> -Ravi.
>
> On Tue, Jul 30, 2013 at 7:38 PM, Ravi Chunduru  wrote:*
> ***
>
> Hi,
>
>  I was designing a Openstack service that listens on notifications
> generated from Nova, Neutron etc., for our custom needs to monitor and act
> up on events.
>
> I have an interesting observation from openstack perspective and I would
> like to know community thinking about it.
>
> ** **
>
> By the time an openstack service starts, it would have missed some
> notifications.  How do we deal with it?
>
> ** **
>
> For example, Designate(DNSaaS) starts and by that time some VMs got
> created, it would loose to update the DNS servers for those VMs.
>
> Should any openstack service such as Designate  call openstack APIs and
> get existing configuration?
>
> ** **
>
> Thanks,
>
> --
> Ravi
>
>
>
> 
>
> ** **
>
> --
> Ravi
>
> ___
> OpenStack-dev mailing list
> OpenStack-dev@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>


-- 
Ravi
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Openstack] Ceilometer and notifications

2013-08-01 Thread Rosa, Andrea (HP Cloud Services)
HI 

>AFAIK, amqp only uses the exchange as a dispatcher and all storage is done in
>the queue ... but I could be wrong. I vaguely recall there being a durable
>exchange setting as well as durable queue.

AFAIK if a message can't be routed there are three options:
 - the message is discarded (I think this is the default option)
-  the message is sent back to the publisher (some params need to be specified 
when a message is published)
- some broker can have specific queues for those messages, for example Rabbitmq 
has a "dead letter queue" where not routable messages are stored.

Durable exchanges are exchanges that survive to a server restart, they don't 
need to be re-declared.
HTH
--
Andrea Rosa

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


[openstack-dev] [Glance] async worker design meeting

2013-08-01 Thread Brian Rosmaita
asynchronous worker meeting 2 Aug 2013 14:00 UTC in #openstack-glance

please look at https://etherpad.openstack.org/havana-glance-requirements before 
the meeting

cheers,
brian

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


[openstack-dev] [Ceilometer] Alarming should be outside of Ceilometer as a separate package.

2013-08-01 Thread Sandy Walsh
Hey y'all,

I've had a little thorn in my claw on this topic for a while and thought
I'd ask the larger group.

I applaud the efforts of the people working on the alarming additions to
Ceilometer, but I've got concerns that we're packaging things the wrong way.

I fear we're making another Zenoss/Nagios with Ceilometer. It's trying
to do too much.

The current trend in the monitoring work (#monitoringlove) is to build
your own stack from a series of components. These components take in
inputs, process them and spit out outputs.
Collectors/Transformers/Publishers. This is the model CM is built on.

Making an all-singing-all-dancing monolithic monitoring package is the
old way of building these tools. People want to use best-of-breed
components for their monitoring stack. I'd like to be able to use
reimann.io for my stream manager, diamond for my collector, logstash for
my parser, etc. Alarming should just be another consumer.

CM should do one thing well. Collect data from openstack, store and
process them, and make them available to other systems via the API or
publishers. That's all. It should understand these events better than
any other product out there. It should be able to produce meaningful
metrics/meters from these events.

Alarming should be yet another consumer of the data CM produces. Done
right, the If-This-Then-That nature of the alarming tool could be
re-used by the orchestration team or perhaps even scheduling.
Intertwining it with CM is making the whole thing too complex and rigid
(imho).

CM should be focused on extending our publishers and input plug-ins.

I'd like to propose that alarming becomes its own project outside of
Ceilometer. Or, at the very least, its own package, external of the
Ceilometer code base. Perhaps it still lives under the CM moniker, but
packaging-wise, I think it should live as a separate code base.

Please, change my view. :)

-S

Side note: I might even argue that the statistical features of CM are
going a little too far. My only counter-argument is that statistics are
the only way to prevent us from sending large amounts of data over the
wire for post-processing. But that's a separate discussion.


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Nova] Proposal to add Nikola Đipanov to nova-core

2013-08-01 Thread Andrew Laski

+1

On 07/31/13 at 03:10pm, Russell Bryant wrote:

Greetings,

I propose that we add Nikola Đipanov to the nova-core team [1].

Nikola has been actively contributing to nova for a while now, both in
code and reviews.  He provides high quality reviews. so I think he would
make a good addition to the review team.

   https://review.openstack.org/#/q/reviewer:ndipa...@redhat.com,n,z

   https://review.openstack.org/#/q/owner:ndipa...@redhat.com,n,z

Please respond with +1/-1.

Thanks,

[1] https://wiki.openstack.org/wiki/Nova/CoreTeam

--
Russell Bryant

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Openstack] Ceilometer and notifications

2013-08-01 Thread Sandy Walsh


On 08/01/2013 10:25 AM, Julien Danjou wrote:
> On Thu, Aug 01 2013, Sandy Walsh wrote:
> 
>> Right, that is a concern. Within RAX we have two downstream services
>> that consume notifications (StackTach and Yagi) and we've configured
>> nova to write to two queues. --notifications_topics can take a list.
> 
> Fair enough. So IIUC that means we should change Ceilometer to consume
> CONF.notification_topics using the default queue associated with this
> topic, and let the deployer add more notification_topics in nova if they
> have several consumer (that should have their own
> CONF.notification_topics).
> 
> This would be the correct way, even if not really deployer-friendly.

There are risks, but I think it'll offer a better "out-of-the-box"
experience.

> 
>> I don't think so, but if that was possible it would solve our problem.
>>
>> AFAIK, amqp only uses the exchange as a dispatcher and all storage is
>> done in the queue ... but I could be wrong. I vaguely recall there being
>> a durable exchange setting as well as durable queue.
>>
>> I'll do some investigating.
> 
> Yeah I've little hope too, but might worth a look. Thanks Sandy!
> 

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Openstack] Ceilometer and notifications

2013-08-01 Thread Julien Danjou
On Thu, Aug 01 2013, Sandy Walsh wrote:

> Right, that is a concern. Within RAX we have two downstream services
> that consume notifications (StackTach and Yagi) and we've configured
> nova to write to two queues. --notifications_topics can take a list.

Fair enough. So IIUC that means we should change Ceilometer to consume
CONF.notification_topics using the default queue associated with this
topic, and let the deployer add more notification_topics in nova if they
have several consumer (that should have their own
CONF.notification_topics).

This would be the correct way, even if not really deployer-friendly.

> I don't think so, but if that was possible it would solve our problem.
>
> AFAIK, amqp only uses the exchange as a dispatcher and all storage is
> done in the queue ... but I could be wrong. I vaguely recall there being
> a durable exchange setting as well as durable queue.
>
> I'll do some investigating.

Yeah I've little hope too, but might worth a look. Thanks Sandy!

-- 
Julien Danjou
# Free Software hacker # freelance consultant
# http://julien.danjou.info


signature.asc
Description: PGP signature
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Nova] Proposal to add Nikola Đipanov to nova-core

2013-08-01 Thread Gary Kotton
+1

-Original Message-
From: Sean Dague [mailto:s...@dague.net] 
Sent: Thursday, August 01, 2013 1:51 PM
To: OpenStack Development Mailing List
Subject: Re: [openstack-dev] [Nova] Proposal to add Nikola Đipanov to nova-core

On 07/31/2013 03:10 PM, Russell Bryant wrote:
> Greetings,
>
> I propose that we add Nikola Đipanov to the nova-core team [1].
>
> Nikola has been actively contributing to nova for a while now, both in 
> code and reviews.  He provides high quality reviews. so I think he 
> would make a good addition to the review team.
>
>  https://review.openstack.org/#/q/reviewer:ndipa...@redhat.com,n,z
>
>  https://review.openstack.org/#/q/owner:ndipa...@redhat.com,n,z
>
> Please respond with +1/-1.
>
> Thanks,
>
> [1] https://wiki.openstack.org/wiki/Nova/CoreTeam
>

+1

--
Sean Dague
http://dague.net

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] Weight normalization in scheduler

2013-08-01 Thread Sandy Walsh


On 08/01/2013 09:51 AM, Álvaro López García wrote:
> On Thu 01 Aug 2013 (09:07), Sandy Walsh wrote:
>>
>>
>> On 08/01/2013 04:24 AM, Álvaro López García wrote:
>>> Hi all.
>>>
>>> TL;DR: I've created a blueprint [1] regarding weight normalization.
>>> I would be very glad if somebody could examine and comment it.
>>
>> Something must have changed. It's been a while since I've done anything
>> with the scheduler, but normalized weights is the way it was designed
>> and implemented.
> 
> It seems reasonable, but it is not there anymore:
> 
> class RAMWeigher(weights.BaseHostWeigher):
> (...)
> def _weigh_object(self, host_state, weight_properties):
> """Higher weights win.  We want spreading to be the default."""
> return host_state.free_ram_mb

Hmm, that's unfortunate. We use our own weighing functions internally,
so perhaps we were unaffected by this change.

> 
> 
>> The separate Weighing plug-ins are responsible for taking the specific
>> units (cpu load, disk, ram, etc) and converting them into normalized
>> 0.0-1.0 weights. Internally the plug-ins can work however they like, but
>> their output should be 0-1.
> 
> With the current code, this is not true. Anyway, I think this responsability
> should be implemented in the BaseWeightHandler rather than each weigher.
> This way each weigher can return whatever they want, but we will be
> always using a correct value.

I think the problem with moving it to the base handler is that the base
doesn't know the max range of the value ... of course, this could be
passed down. But yeah, we wouldn't want to duplicate the normalization
code itself in every function.

>> The multiplier, however, could scale this outside that range (if disk is
>> more important than cpu, for example).
> 
> Yes, of course, since the multiplier is applied *after* normalizing the
> weights.
> 
>> Actually, I remember it being offset + scale * weight, so you could put
>> certain factors in bands: cpu: 1000+, disk: 1+, etc. Hopefully
>> offset is still there too?
> 
> No, it is not.

poop. You may want to consider adding it back. Offset + Scale * Weight
allows for many more weighing constructs.

> 
> Thanks,
> 

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] Weight normalization in scheduler

2013-08-01 Thread Álvaro López García
On Thu 01 Aug 2013 (09:07), Sandy Walsh wrote:
> 
> 
> On 08/01/2013 04:24 AM, Álvaro López García wrote:
> > Hi all.
> > 
> > TL;DR: I've created a blueprint [1] regarding weight normalization.
> > I would be very glad if somebody could examine and comment it.
> 
> Something must have changed. It's been a while since I've done anything
> with the scheduler, but normalized weights is the way it was designed
> and implemented.

It seems reasonable, but it is not there anymore:

class RAMWeigher(weights.BaseHostWeigher):
(...)
def _weigh_object(self, host_state, weight_properties):
"""Higher weights win.  We want spreading to be the default."""
return host_state.free_ram_mb


> The separate Weighing plug-ins are responsible for taking the specific
> units (cpu load, disk, ram, etc) and converting them into normalized
> 0.0-1.0 weights. Internally the plug-ins can work however they like, but
> their output should be 0-1.

With the current code, this is not true. Anyway, I think this responsability
should be implemented in the BaseWeightHandler rather than each weigher.
This way each weigher can return whatever they want, but we will be
always using a correct value.

> The multiplier, however, could scale this outside that range (if disk is
> more important than cpu, for example).

Yes, of course, since the multiplier is applied *after* normalizing the
weights.

> Actually, I remember it being offset + scale * weight, so you could put
> certain factors in bands: cpu: 1000+, disk: 1+, etc. Hopefully
> offset is still there too?

No, it is not.

Thanks,
-- 
Álvaro López García  al...@ifca.unican.es

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


[openstack-dev] Global Requirements enforcement in the devstack gate ... coming soon

2013-08-01 Thread Sean Dague
Over the past week Monty and I have been running at how to solve the 
problem that because of the way we install devstack nodes, our loose 
coordination of projects causes some "interesting" issues.


===
The basic problem
===

We install projects in the gate in this order (based on linear order in 
devstack):


install_infra
install_oslo
install_keystoneclient
install_glanceclient
install_cinderclient
install_novaclient

(then conditionally if certain services are enabled)
install_swiftclient
install_neutronclient
install_heatclient
install_keystone
install_swift
install_glance
install_cinder
install_neutron
install_neutron_third_party
install_nova
install_horizon
install_ceilometerclient
install_ceilometer
install_heat
install_tempest

At every point we "python setup.py develop" to pull in requirements. 
That means, that if in install_glance we require "jsonschema" we'll 
install the latest, i.e. 2.0. And if at install_tempest we require 
"jsonschema>=1.3.0,!=1.4.0,<2" (which is the global requirements line), 
we will uninstall jsonschema 2.0. And if something, like cinder, dragged 
in something that required jsonschema, and uses entry points, which 
really care about python package versions, it will be built against 2.0, 
which will have been removed, and we'll have 1.3 instead.


Note: that's not a theoretical scenario, that's why no one could merge 
code last weekend.


We had an equivalent wedge with horizon a month ago, which was the 
python-*client uncap push.


Basically we're spinning 20 plates all at the same time, which all 
have different review times, +2 teams, and entropy means they tend to 
drift away from each other over time. It took us a month to get 
python-*clients uncapped across all the projects because of the 
intricate ways in which our projects require our other projects in 
interesting ways.



Proposed Solution - part #1 - use global requirements in devstack gate


The basic idea is simple, in the devstack gate, we're going to forcibly 
update all the project's requirements.txt files to global requirements 
versions, before running setup.py. That means that any project which 
requires a thing will require it at exactly the same version range as 
everyone else. Which ensures that we don't have this uninstall / 
reinstall issue.


We are really close to having this working. Monty and I have been 
running at it all week, and I actually think this requirements review - 
https://review.openstack.org/#/c/39461/ might be the last bit we need 
before we can land the devstack changes for this.


Things this solves:
 * makes global requirements actually mean something, right now the 
sqlalchemy line is known wrong, which prevents nova from working. There 
are probably other issues as well.

 * ensures we really have a set of requirements that work for all projects.

Things this will allow:
 * devstack gating on proposed changes to global-requirements, so 
you'll know if all the projects can work with global-requirements before 
you land them.


Things that we had to change along the way:
 * oslo.config / oslo.messaging now in devstack - in order to let 
projects use prerelease features of those projects, in this model, we 
needed their git trees to be in our environment, like the python* clients.
 * oslo.config in the devstack-gate (landing soon) - we'll now know on 
every oslo.config proposed commit that it works with all the projects in 
the devstack gate. Once oslo.messaging is proposed for use, we'll 
devstack-gate that as well.


Assuming sqlalchemy is the only requirements issue (it's likely that is 
the case), the rest of the gating around this should land today. This 
should just slide in quietly, and not cause any problems, however, there 
is an interesting caveat


=
Interesting Caveat
=

Global requirements will be used by everyone in the devstack-gate and 
the grenade-gate, *however* in unit tests, projects will still be using 
their local requirements. These might not be the same (actually, for 
almost all projects I can tell you these aren't the same today).


We need to think about the resolution here. There are a number of ones I 
can think of, but this is going to clearly require more thought to get 
right. We need to be able to support things like prerelease Oslo, for 
instance. So lets just say this is tabled until phase #1 is fixed.


-Sean

--
Sean Dague
http://dague.net

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Openstack] Ceilometer and notifications

2013-08-01 Thread Sandy Walsh


On 08/01/2013 09:19 AM, Julien Danjou wrote:
> On Thu, Aug 01 2013, Sandy Walsh wrote:
> 
>> Hmm, if notifications are turned on, it should fill up. For billing
>> purposes we don't want to lose events simply because there is no
>> consumer. Operations would alert on it and someone would need to put out
>> the fire.
> 
> So currently, are we're possibly losing events because we don't use the
> standard queue but one defined by Ceilometer upon connection?
> We can't consume events from the default notifications queue or we would
> break any tool possibly using it. Each tool consuming needs a copy of
> them.

Right, that is a concern. Within RAX we have two downstream services
that consume notifications (StackTach and Yagi) and we've configured
nova to write to two queues. --notifications_topics can take a list.

> Isn't there a way to queue the message in exchanges if there's no queue
> at all?

I don't think so, but if that was possible it would solve our problem.

AFAIK, amqp only uses the exchange as a dispatcher and all storage is
done in the queue ... but I could be wrong. I vaguely recall there being
a durable exchange setting as well as durable queue.

I'll do some investigating.

> 

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Neutron][API Doc] Easy way to generate JSON output for doc?

2013-08-01 Thread Paul Michali
Yes that will! Copying and pasting the public key is difficult.

Will try this out.

Regards.

PCM (Paul Michali)

MAIL p...@cisco.com
IRC   pcm_  (irc.freenode.net)
TW   @pmichali

On Aug 1, 2013, at 2:43 AM, Morgan Fainberg  wrote:

> Paul,
> 
> Depending on what version of keystone you are utilizing there are a couple 
> options to use the UUID token format (instead of PKI). 
> 
> Very recent (current master) of keystone uses a pluggable provider system. To 
> set the provider to uuid, in the [token] section set the option:
> provider=keystone.token.providers.uuid.Provider
> 
> In older versions I believe the option (still in the [token] section) is:
> 
> token_format=UUID
> 
> I hope this info helps you out some.
> 
> Cheers,
> Morgan Fainberg
> 
> Sent from my iPhone (please excuse the brevity) 
> 
>  31/07/2013, Paul Michali :
> Yeah I was playing with that, however I'm having an issue with authentication…
> 
> If I do a request to keystone to get the auth ID, I get a huge key, which I 
> have to try to paste into a subsequent request to neutron.
> 
> Is there a way to force this to use the old style (small) auth ID, no auth, 
> or username/password for auth on the requests?
> 
> I've been playing with using the CLI and --verbose, and then trying to 
> extract the RESP:… output and run through JSON. A bit jacky, but it sorta 
> works.
> 
> 
> PCM (Paul Michali)
> 
> Contact info for Cisco users http://twiki.cisco.com/Main/pcm
> 
> 
> On Jul 31, 2013, at 7:00 PM, "Mellquist, Peter"  
> wrote:
> 
>> Paul,
>> There are a few ways of doing this but I have used curl then pipe the 
>> results through a python JSON pretty print tool. This formats the JSON for 
>> easily dropping into API docs.
>>  
>> curl ( some Openstack request with JSON output )| python –mjson.tool
>>  
>> Hope this helps,
>> Peter.
>>  
>>  
>>  
>> From: Paul Michali [mailto:p...@cisco.com] 
>> Sent: Wednesday, July 31, 2013 3:41 PM
>> To: OpenStack Development Mailing List
>> Subject: [openstack-dev] [Neutron][API Doc] Easy way to generate JSON output 
>> for doc?
>>  
>> Hi!
>>  
>> I'm writing API doc pages for VPNaaS and was wondering if there are any 
>> tools/scripts to make it easy to generate the needed JSON result output for 
>> various operations?
>>  
>> It looks like I can do the neutron command with --verbose to get unformatted 
>> JSON output. Should I do that and then reformat the output (or is there way 
>> to do that easier)?
>>  
>> I did try to use json.loads() on the RESP: output, but it threw an 
>> ValueError "Expecting property name: line 1 column 1 (char 1)"
>>  
>> Ideas?
>>  
>> PCM (Paul Mi
>> ___
>> OpenStack-dev mailing list
>> OpenStack-dev@lists.openstack.org
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> 
> 
> 
> -- 
> Sent from my iPhone (please excuse the brevity and any typos)
> ___
> OpenStack-dev mailing list
> OpenStack-dev@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Openstack] Ceilometer and notifications

2013-08-01 Thread Julien Danjou
On Thu, Aug 01 2013, Sandy Walsh wrote:

> Hmm, if notifications are turned on, it should fill up. For billing
> purposes we don't want to lose events simply because there is no
> consumer. Operations would alert on it and someone would need to put out
> the fire.

So currently, are we're possibly losing events because we don't use the
standard queue but one defined by Ceilometer upon connection?
We can't consume events from the default notifications queue or we would
break any tool possibly using it. Each tool consuming needs a copy of
them.

Isn't there a way to queue the message in exchanges if there's no queue
at all?

-- 
Julien Danjou
-- Free Software hacker - freelance consultant
-- http://julien.danjou.info


signature.asc
Description: PGP signature
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] Weight normalization in scheduler

2013-08-01 Thread Sandy Walsh


On 08/01/2013 04:24 AM, Álvaro López García wrote:
> Hi all.
> 
> TL;DR: I've created a blueprint [1] regarding weight normalization.
> I would be very glad if somebody could examine and comment it.

Something must have changed. It's been a while since I've done anything
with the scheduler, but normalized weights is the way it was designed
and implemented.

The separate Weighing plug-ins are responsible for taking the specific
units (cpu load, disk, ram, etc) and converting them into normalized
0.0-1.0 weights. Internally the plug-ins can work however they like, but
their output should be 0-1.

The multiplier, however, could scale this outside that range (if disk is
more important than cpu, for example).

Actually, I remember it being offset + scale * weight, so you could put
certain factors in bands: cpu: 1000+, disk: 1+, etc. Hopefully
offset is still there too?

-S



> Recently I've been developing some weighers to be used within nova and I
> found that the weight system was using raw values. This makes difficult
> for an operator to stablish the importance of a weigher against the rest
> of them, since the values can range freely and one big magnitude
> returned by a weigher could shade another one.
> 
> One solution is to inflate either the multiplier or the weight that is
> returned by the weigher, but this is an ugly hack (for example, if you
> increase the RAM on your systems, you will need to adjust the
> multipliers again). A much better approach is to use weight
> normalization before actually using the weights
> 
> With weight normalization a weigher will still return a list of RAW
> values, but the BaseWeightHandler will normalize all of them into a
> range of values (0.0 and 1.0) before adding them up. This way, a weight
> for a given object will be:
> 
>   weight = w1_multiplier * norm(w1) + w2_multiplier * norm(w2) + ...
> 
> This makes easier to stablish the importance of a weigher regarding the
> rest, by just adjusting the multiplier. This is explained in [1], and
> implemented in [2] (with some suggestions by the reviewers).
> 
> [1] 
> https://blueprints.launchpad.net/openstack/?searchtext=normalize-scheduler-weights
> [2] https://review.openstack.org/#/c/27160/
> 
> Thanks for your feedback,
> 

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Openstack] Ceilometer and notifications

2013-08-01 Thread Sandy Walsh


On 08/01/2013 07:02 AM, Mark McLoughlin wrote:
> On Thu, 2013-08-01 at 10:36 +0200, Julien Danjou wrote:
>> On Thu, Aug 01 2013, Sam Morrison wrote:
>>
>>> OK so is it that ceilometer just leaves the message on the queue or
>>> only consumes certain messages?
>>
>> Ceilometer uses its own queue. There might be other processes consuming
>> this notifications, so removing them may be not a good idea.
>>
>> The problem may be that the notification sender create a queue by
>> default even if there's no consumer on that. Maybe that's something we
>> should avoid doing in Oslo (Cc'ing -dev to get advice on that).
> 
> I'm missing the context here, but it sounds like the default
> notifications queue created isn't the one consumed by ceilometer so it
> fills up and we just shouldn't be creating that queue.
> 
> Sounds reasonable to me. Definitely file a bug for it.

Hmm, if notifications are turned on, it should fill up. For billing
purposes we don't want to lose events simply because there is no
consumer. Operations would alert on it and someone would need to put out
the fire.

That's the reason we create the queue up front in the first place.
Ideally, we could only write to the exchange, but we need the queue to
ensure we don't lose any events.

The CM Collector consumes from two queues: it's internal queue and the
Nova queue (if configured). If CM is looking at the wrong nova queue by
default, the bug would be over there.


> 
> Cheers,
> Mark.
> 
> 
> ___
> OpenStack-dev mailing list
> OpenStack-dev@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> 

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [CI] Now gating on Python 3.3

2013-08-01 Thread Sean Dague

On 07/31/2013 11:18 PM, Jeremy Stanley wrote:

A quick announcement... thanks to the invaluable assistance of Chuck
Short, Dan Prince, Julien Danjou and others, some projects are now
being successfully tested against and gated on Python 3.3 (with even
more projects very close to ready for the same).

As discussed previously in summit sessions and on this list,
libraries and clients whose various dependencies already fully
support Python 3.3 are great candidates for addition in the near
term. Feel free to reach out to the infrastructure team on the
Freenode IRC network in #openstack-infra or through the
openstack-in...@lists.openstack.org mailing list, should you require
assistance setting up tests for Python 3.3 compatibility in a
project.



Nicely done folks!

-Sean

--
Sean Dague
http://dague.net

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [CI] Now gating on Python 3.3

2013-08-01 Thread Roman Prykhodchenko
Jeremy,

thank you for this announce. It would be very helpful to mention those projects.


- Roman Prykhodchenko

On Aug 1, 2013, at 06:18 , Jeremy Stanley  wrote:

> A quick announcement... thanks to the invaluable assistance of Chuck
> Short, Dan Prince, Julien Danjou and others, some projects are now
> being successfully tested against and gated on Python 3.3 (with even
> more projects very close to ready for the same).
> 
> As discussed previously in summit sessions and on this list,
> libraries and clients whose various dependencies already fully
> support Python 3.3 are great candidates for addition in the near
> term. Feel free to reach out to the infrastructure team on the
> Freenode IRC network in #openstack-infra or through the
> openstack-in...@lists.openstack.org mailing list, should you require
> assistance setting up tests for Python 3.3 compatibility in a
> project.
> -- 
> Jeremy Stanley
> 
> ___
> OpenStack-dev mailing list
> OpenStack-dev@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Nova] Proposal to add Nikola Đipanov to nova-core

2013-08-01 Thread Boris Pavlovic
+1


On Thu, Aug 1, 2013 at 2:51 PM, Sean Dague  wrote:

> On 07/31/2013 03:10 PM, Russell Bryant wrote:
>
>> Greetings,
>>
>> I propose that we add Nikola Đipanov to the nova-core team [1].
>>
>> Nikola has been actively contributing to nova for a while now, both in
>> code and reviews.  He provides high quality reviews. so I think he would
>> make a good addition to the review team.
>>
>>  https://review.openstack.org/#**/q/reviewer:ndipanov@redhat.**
>> com,n,z
>>
>>  
>> https://review.openstack.org/#**/q/owner:ndipa...@redhat.com,**n,z
>>
>> Please respond with +1/-1.
>>
>> Thanks,
>>
>> [1] 
>> https://wiki.openstack.org/**wiki/Nova/CoreTeam
>>
>>
> +1
>
> --
> Sean Dague
> http://dague.net
>
>
> __**_
> OpenStack-dev mailing list
> OpenStack-dev@lists.openstack.**org 
> http://lists.openstack.org/**cgi-bin/mailman/listinfo/**openstack-dev
>
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Nova] Proposal to add Nikola Đipanov to nova-core

2013-08-01 Thread Sean Dague

On 07/31/2013 03:10 PM, Russell Bryant wrote:

Greetings,

I propose that we add Nikola Đipanov to the nova-core team [1].

Nikola has been actively contributing to nova for a while now, both in
code and reviews.  He provides high quality reviews. so I think he would
make a good addition to the review team.

 https://review.openstack.org/#/q/reviewer:ndipa...@redhat.com,n,z

 https://review.openstack.org/#/q/owner:ndipa...@redhat.com,n,z

Please respond with +1/-1.

Thanks,

[1] https://wiki.openstack.org/wiki/Nova/CoreTeam



+1

--
Sean Dague
http://dague.net

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Openstack] Ceilometer and notifications

2013-08-01 Thread Mark McLoughlin
On Thu, 2013-08-01 at 10:36 +0200, Julien Danjou wrote:
> On Thu, Aug 01 2013, Sam Morrison wrote:
> 
> > OK so is it that ceilometer just leaves the message on the queue or
> > only consumes certain messages?
> 
> Ceilometer uses its own queue. There might be other processes consuming
> this notifications, so removing them may be not a good idea.
> 
> The problem may be that the notification sender create a queue by
> default even if there's no consumer on that. Maybe that's something we
> should avoid doing in Oslo (Cc'ing -dev to get advice on that).

I'm missing the context here, but it sounds like the default
notifications queue created isn't the one consumed by ceilometer so it
fills up and we just shouldn't be creating that queue.

Sounds reasonable to me. Definitely file a bug for it.

Cheers,
Mark.


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


[openstack-dev] [metrics] releases study ready plus new look and feel

2013-08-01 Thread Daniel Izquierdo

Hi!

You can find at [1] the new look and feel of the activity dashboard. 
This is providing the same results as the old one, but with some 
improvements regarding to unique identities and affiliations.


In addition in [2], you'll see a releases study: Essex, Folsom, Grizzly 
and daily updated Havana, plus the whole history of the project. (Still 
takes a bit long to download everything u_u).


As usual, you can download the databases in [3] and json files in [4]. 
Both views, the activity dashboard and the releases study are using this 
dataset.


Finally, I'd like to stress the point over the use of the actionable 
dashboard, where you can produce your own charts and compare if needed 
activity in projects or companies [5]. Given that it's a bit complex to 
have in a dashboard all of the necessities from the community, this 
place was built to create any chart you may require. I'd say that this 
is a really powerful tool to learn about the OpenStack community. And 
although this is only focused on companies and projects, any other 
metric or type of analysis could be potentially added.


As you can see, this is work in progress, so any feedback is more than 
welcome.


Please, feel free to comment. This is a tool built for the community. So 
your requirements should be part of further developments.
An example of such requirements are to provide Gerrit metrics [6] per 
project as identified in the mailing lists some days ago (work in progress).


Cheers!,
Daniel

[1] http://activity.openstack.org/dash/browser/
[2] http://activity.openstack.org/dash/releases/
[3] http://activity.openstack.org/dash/browser/data/db/
[4] http://activity.openstack.org/dash/browser/data/json/
[5] http://activity.openstack.org/dash/dashboard/
[6] http://activity.openstack.org/dash/browser/scr.html

Links to the toolset and dash:
https://github.com/Bitergia/openstack-dashboard
https://github.com/VizGrimoire/VizGrimoireJS/tree/openstack-bootstrap
https://github.com/VizGrimoire/VizGrimoireR/tree/openstack

Features request/bugs/...:
https://bugs.launchpad.net/openstack-community

--
Daniel Izquierdo Cortazar, PhD
Chief Data Officer
-
"Software Analytics for your peace of mind"
www.bitergia.com
@bitergia


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] Program Proposal: Release cycle management

2013-08-01 Thread Mark McLoughlin
On Thu, 2013-08-01 at 11:39 +0200, Thierry Carrez wrote:
> Following past discussions[1] on the TC, here is my proposal to cover
> for release management, stable branch management and VMT efforts within
> OpenStack.
> 
> Feel free to suggest title or wording changes :)
> 
> """
> Official Title: Release cycle management
> PTL: Thierry Carrez
> 
> Mission Statement:
> To organize the release cycle and the work necessary to produce
> time-based, coordinated releases of the integrated components of
> OpenStack. To collect bugfix backports and produce stable point releases
> for the previously-released branch. To coordinate the publication of
> security patches and advisories (OSSA) for security-supported branches.
> 
> Expected deliverables:
> Signed release tarballs, security advisories, release tooling and automation
> 
> Team members:
> Members of the Release Managers team, the stable-maint team and the
> Vulnerability Management team. (Note: All members must become ATCs
> through direct contribution to other supported OpenStack repositories)
> """

Sounds pretty spot on to me.

You could go into more details and mention producing the release
schedule, running project meetings, milestone releases, stable branch
releases, etc. etc. but it's just all details implicit in the mission
above.

Cheers,
Mark.


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


[openstack-dev] Program Proposal: Release cycle management

2013-08-01 Thread Thierry Carrez
Following past discussions[1] on the TC, here is my proposal to cover
for release management, stable branch management and VMT efforts within
OpenStack.

Feel free to suggest title or wording changes :)

"""
Official Title: Release cycle management
PTL: Thierry Carrez

Mission Statement:
To organize the release cycle and the work necessary to produce
time-based, coordinated releases of the integrated components of
OpenStack. To collect bugfix backports and produce stable point releases
for the previously-released branch. To coordinate the publication of
security patches and advisories (OSSA) for security-supported branches.

Expected deliverables:
Signed release tarballs, security advisories, release tooling and automation

Team members:
Members of the Release Managers team, the stable-maint team and the
Vulnerability Management team. (Note: All members must become ATCs
through direct contribution to other supported OpenStack repositories)
"""

[1]
http://eavesdrop.openstack.org/meetings/tc/2013/tc.2013-07-16-20.02.log.html

-- 
Thierry Carrez (ttx)

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] Python overhead for rootwrap

2013-08-01 Thread Thierry Carrez
Joe Gordon wrote:
> I tried
> swapping out rootwrap for sudo and that made the issue go away.So I
> think we should go back to supporting just using sudo instead of
> rootwrap, and make sure any future solutions support a sudo only option
> as well.  But I am open to other ideas, I just think we need to
> implement something for Havana.

How did you make the swap ? Patched utils.execute ? I guess we could
introduce some use_sudo_as_root_helper parameter (defaulting to False,
with security warnings) and preserve rootwrap_config as it is...

It will require a passwordless blanket sudo access for the nova user.
The trick is that this is equivalent to running Nova as the root user,
from a privilege escalation / security domain standpoint. And you can
already do that, and it will also "make the issue go away" (rootwrap is
bypassed if called from euid 0).

It's cleaner to implement it as an option though, as it will not screw
up permissions everywhere in case you want to go back to using rootwrap
at some point in the future.

Another important take-away from your comment is that we should preserve
the ability to run those commands under direct sudo... so if we want to
allow executing python snippets within rootwrap, we'd need to find a way
to do it with a call that is sudo-compatible, which might prove a bit
tricky.

-- 
Thierry Carrez (ttx)

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Openstack] Ceilometer and notifications

2013-08-01 Thread Julien Danjou
On Thu, Aug 01 2013, Sam Morrison wrote:

> OK so is it that ceilometer just leaves the message on the queue or
> only consumes certain messages?

Ceilometer uses its own queue. There might be other processes consuming
this notifications, so removing them may be not a good idea.

The problem may be that the notification sender create a queue by
default even if there's no consumer on that. Maybe that's something we
should avoid doing in Oslo (Cc'ing -dev to get advice on that).

-- 
Julien Danjou
-- Free Software hacker - freelance consultant
-- http://julien.danjou.info


signature.asc
Description: PGP signature
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


[openstack-dev] Is there any document about how to make a lxc image using diskimage-builder?

2013-08-01 Thread hzguanqiang
Hi Steve,
I found it's hard to understand how to make a image from README on github 
address https://github.com/stackforge/diskimage-builder . Is there any other 
document about how to make a lxc image using diskimage-builder step by step?

Thanks

--
Best regards!
GuanQiang
2013-08-01


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Neutron]Connecting a VM from one tenant to a non-shared network in another tenant

2013-08-01 Thread Samuel Bercovici
There was another patch needed:
In: /opt/stack/nova/nova/network/neutronv2/api.py
In the def _get_available_networks function, the developer has added a specific 
line of code filtering networks by the tenant_id.
In general as far as I understand, this might be unneeded as quantum will 
already filter the networks based on the tenant_id in the context while if 
is_admin, will elevate and return all networks which I belive is the behavior 
we want.

Do you think this can somehow be solved only on neutron side or must it also be 
done by rmoving the tenant_id filter in the nova side?

When removing the filter of tenant_id + the pathc bellow, I get the behavior 
that as admin, I can createVMs connected to another tenants private network but 
as non-admin I am not able to do so.

Regards,
-Sam.


From: Samuel Bercovici
Sent: Wednesday, July 31, 2013 7:32 PM
To: OpenStack Development Mailing List; sorla...@nicira.com
Subject: Re: [openstack-dev] [Neutron]Connecting a VM from one tenant to a 
non-shared network in another tenant

Hi Slavatore,

I thought that creating a qport  would be enough but it looks like I still 
missing something else.
I have commented in /opt/stack/quantum/neutron/api/v2/base.py in the create 
function the ._validate_network_tenant_ownership call.
I can now as an Admin user, can create a qport from tenant-a that is mapped to 
a private network in tenant-b.

The following still fails with ERROR: The resource could not be found. (HTTP 
404) ...
nova boot --flavor 1 --image  --nic port-id=
Where  is the one I got from the port-create

Any ideas where I should look next?

Regards,
-Sam.


From: Salvatore Orlando [mailto:sorla...@nicira.com]
Sent: Wednesday, July 31, 2013 5:42 PM
To: OpenStack Development Mailing List
Subject: Re: [openstack-dev] [Neutron]Connecting a VM from one tenant to a 
non-shared network in another tenant

Hi Sam,

is what you're trying to do tantamount to creating a port on a network whose 
tenant_id is different from the network's tenant_id?
We have at the moment a fairly strict ownership check - which does not allow 
even admin users to do this operation.

I do not have a strong opinion against relaxing the check, and allowing admin 
users to create ports on any network - I don't think this would constitute a 
potential vulnerability, as in neutron is someone's manages to impersonate an 
admin user, he/she can make much more damage.

Salvatore

On 31 July 2013 16:11, Samuel Bercovici 
mailto:samu...@radware.com>> wrote:
Hi All,

We are providing load balancing services via virtual machines running under an 
admin tenant that needs to be connected to VMs attached to a non-shared/private 
tenant network.
The virtual machine fails to be provisioned connected to the private tenant 
network event if it is provisioned using the admin user which has admin role on 
both tenants.
Please advise?

Best Regards,
-Sam.


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


[openstack-dev] Weight normalization in scheduler

2013-08-01 Thread Álvaro López García
Hi all.

TL;DR: I've created a blueprint [1] regarding weight normalization.
I would be very glad if somebody could examine and comment it.

Recently I've been developing some weighers to be used within nova and I
found that the weight system was using raw values. This makes difficult
for an operator to stablish the importance of a weigher against the rest
of them, since the values can range freely and one big magnitude
returned by a weigher could shade another one.

One solution is to inflate either the multiplier or the weight that is
returned by the weigher, but this is an ugly hack (for example, if you
increase the RAM on your systems, you will need to adjust the
multipliers again). A much better approach is to use weight
normalization before actually using the weights

With weight normalization a weigher will still return a list of RAW
values, but the BaseWeightHandler will normalize all of them into a
range of values (0.0 and 1.0) before adding them up. This way, a weight
for a given object will be:

  weight = w1_multiplier * norm(w1) + w2_multiplier * norm(w2) + ...

This makes easier to stablish the importance of a weigher regarding the
rest, by just adjusting the multiplier. This is explained in [1], and
implemented in [2] (with some suggestions by the reviewers).

[1] 
https://blueprints.launchpad.net/openstack/?searchtext=normalize-scheduler-weights
[2] https://review.openstack.org/#/c/27160/

Thanks for your feedback,
-- 
Álvaro López García  al...@ifca.unican.es

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev