date:20140211

2014-02-10 18:45 GMT+01:00 Chris Friesen chris.frie...@windriver.com:


 In that model, we would pass a bunch of information about multiple
 resources to the solver scheduler, have it perform scheduling *and reserve
 the resources*, then return some kind of resource reservation tokens back
 to the caller for each resource.  The caller could then allocate each
 resource, pass in the reservation token indicating both that the resources
 had already been reserved as well as what the specific resource that had
 been reserved (the compute-host in the case of an instance, for example).

 Chris


Please be aware that Climate [1] already exists for managing resources
reservations. That doesn't make sense and has been discussed during last
summit that reservations should be managed by Nova, but rather by another
service.

-Sylvain

[1] : https://launchpad.net/climate
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Neutron] Nominate Oleg Bondarev for Core

2014-02-11 Thread Gary Kotton

+1


On 2/11/14 1:28 AM, Mark McClain mmccl...@yahoo-inc.com wrote:

All-

I¹d like to nominate Oleg Bondarev to become a Neutron core reviewer.
Oleg has been valuable contributor to Neutron by actively reviewing,
working on bugs, and contributing code.

Neutron cores please reply back with +1/0/-1 votes.

mark
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
https://urldefense.proofpoint.com/v1/url?u=http://lists.openstack.org/cgi-
bin/mailman/listinfo/openstack-devk=oIvRg1%2BdGAgOoM1BIlLLqw%3D%3D%0Ar=e
H0pxTUZo8NPZyF6hgoMQu%2BfDtysg45MkPhCZFxPEq8%3D%0Am=KKs5yVeNC6c7WnUVDuIoU
h%2BlWzSzcE1BVaTK%2B71PUM0%3D%0As=b38d5081ce0f288403c87ba38281d93cd1796c9
3f5482ded0916ee3b26e54774


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Openstack-dev][All] tox 1.7.0 error while running tests

2014-02-11 Thread Dmitry Tantsur

Hi. This seems to be related:
https://bugs.launchpad.net/openstack-ci/+bug/1274135
We also encountered this.

On Tue, 2014-02-11 at 14:56 +0530, Swapnil Kulkarni wrote:
 Hello,
 
 
 I created a new devstack environment today and installed tox 1.7.0,
 and getting error tox.ConfigError: ConfigError: substitution key
 'posargs' not found.
 
 
 Details in [1].
 
 
 Anybody encountered similar error before? Any workarounds/updates
 needed?
 
 
 [1] http://paste.openstack.org/show/64178/
 
 
 
 
 Best Regards,
 Swapnil Kulkarni
 irc : coolsvap
 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [infra] Proposing Sergey Lukjanov for infra-core

2014-02-11 Thread Thierry Carrez

James E. Blair wrote:
 I'm very pleased to propose that we add Sergey Lukjanov to the
 infra-core team.
 
 He is among the top reviewers of projects in openstack-infra, and is
 very familiar with how jenkins-job-builder and zuul are used and
 configured.  He has done quite a bit of work in helping new projects
 through the process and ensuring that changes to the CI system are
 correct.  In addition to providing very helpful reviews he has also
 contributed significant patches to our Python projects illustrating a
 high degree of familiarity with the code base and project direction.
 And as a bonus, we're all looking forward to once again having an
 infra-core member in a non-US time zone!

+1!

My only fear is that Sergey is going in too many directions at the same
time (remember he is also the Savanna^WSahara/Caravan/Batyr/Slonik PTL)
and might burn out. But so far he has been gracefully handling the
increasing load... so let's see how that goes :)

-- 
Thierry Carrez (ttx)

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Heat] in-instance update hooks

Excerpts from Thomas Spatzier's message of 2014-02-11 00:38:53 -0800:
 Hi Clint,
 
 thanks for writing this down. This is a really interesting use case and
 feature, also in relation to what was recently discussed on rolling
 updates.
 
 I have a couple of thoughts and questions:
 
 1) The overall idea seems clear to me but I have problems understanding the
 detailed flow and relation to template definitions and metadata. E.g. in
 addition to the examples you gave in the linked etherpad, where would the
 script or whatever sit that handles the update etc.
 

At the risk of sounding curt and unfeeling: I really don't care how your
servers do their job or talk to heat.. as long as they use the API. :)

Heat is an orchestration tool, and putting script code or explicit tool
callouts in templates is obscene to me. I understand other people are
stuck with vanilla images, but I am not, so I have very little time to
spend thinking about how that is done.

In TripleO, os-collect-config periodically polls the metadata and runs
os-refresh-config when it changes. os-refresh-config just runs a bunch
of scripts. The scripts use os-apply-config to interpret the metadata
section, either through mustache templates or like this:

ACTION=$(os-apply-config --key action.pending --key-default '' --type raw)
case $ACTION in
rebuild)
delete)
  migrate_to_something_else
  ping_the_handle $(os-apply_config --key action.handle --type url)
  ;;
*)
  ;;
esac
  
We just bake all of this into our images.

 2) I am not a big fan of CFN WaitConditions since they let too much
 programming shine thru in a template. So I wonder whether this could be
 made more transparent to the template writer. The underlying mechanism
 could still be the same, but maybe we could make the template look cleaner.
 For example, what Steve Baker is doing for software orchestration also uses
 the underlying mechanisms but does not expose WaitConditions in templates.


Yeah we could allocate the handle transparently without much difficulty.
However, to be debuggable we'd have to make it available as an attribute
of the server if we don't have a resource directly queryable for it. Not
sure if Steve has done that but it is pretty important to be able to
compare the handle URL you see in the stack to the one you see on the
server.

 3) Has the issue of how to express update policies on the rolling updates
 thread been resolved? I followed that thread but seems like there has not
 been a final decision. The reason I am bringing this up is because I think
 this is related. You are suggesting to establish a new top-level section
 'action_hooks' in a resource. Rendering this top-level in the resource is a
 good thing IMO. However, since this is related to updates in a way (you
 want to react to any kind of update event to the resource's state), I
 wonder if those hooks could be attributes of an update policy. UpdatePolicy
 in CFN is also a top-level section in a resource and they seem to provide a
 default one like the following (I am writing this in snake case as we would
 render it in HOT:
 
 resources:
   autoscaling_group1:
 type: AWS::AutoScaling::AutoScalingGroup
 properties:
   # the properties ...
 update_policy:
   auto_scaling_rolling_update:
 min_instances_in_server: 1
 max_batch_size: 1
 pause_time: PT12M5S
 
 (I took this from the CFN user guide).
 I.e. an update policy already is a complex data structure, and we could
 define additional types that include the resource hooks definitions you
 need. ... I don't fully understand the connection between 'actions' and
 'path' in your etherpad example yet, so cannot define a concrete example,
 but I hope you get what I wanted to express.
 

This also works on stack-delete. Perhaps delete is just a special update
that replaces the template with '', but update_policy seems a bit off
base given this. The two features (rolling-updates and update-hooks)
seem related, but I think only because they'd both be more useful with
the other available.

 4) What kind of additional metadata for the update events are you thinking
 about? For example, in case this is done in an update case with a batch
 size of  1 (i.e. you update multiple members in a cluster at a time) -
 unless I put too much interpretation in here concerning the relation to
 rolling updates - you would probably want to tell the server a black list
 of servers to which it should not migrate workload, because they will be
 taken down as well.
 

Agreed, for rolling updates we'd need some additional clues. For just
doing explicit servers, we can think a little more statically and just
do a bit shift of the workloads (first server sends to last.. second to
first.. etc).

 
 As I said, just a couple of thoughts, and maybe for some I am just
 mis-understanding some details.
 Anyway, I would be interested in your view.
 

Your thoughts are most appreciated!

___
OpenStack-dev

Re: [openstack-dev] [Neutron] Nominate Oleg Bondarev for Core

2014-02-11 Thread Robert Kukura

On 02/10/2014 06:28 PM, Mark McClain wrote:
 All-
 
 I’d like to nominate Oleg Bondarev to become a Neutron core reviewer.  Oleg 
 has been valuable contributor to Neutron by actively reviewing, working on 
 bugs, and contributing code.
 
 Neutron cores please reply back with +1/0/-1 votes.

+1

 
 mark
 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
 


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Neutron] Nominate Oleg Bondarev for Core

2014-02-11 Thread Edgar Magana

+1 of course!

Edgar 

Sent from my iPhone

 On Feb 11, 2014, at 8:28 AM, Mark McClain mmccl...@yahoo-inc.com wrote:
 
 All-
 
 I’d like to nominate Oleg Bondarev to become a Neutron core reviewer.  Oleg 
 has been valuable contributor to Neutron by actively reviewing, working on 
 bugs, and contributing code.
 
 Neutron cores please reply back with +1/0/-1 votes.
 
 mark
 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [Murano] Can we migrate to oslo.messaging?

2014-02-11 Thread Serg Melikyan

oslo.messaging http://github.com/openstack/oslo.messaging is a library
that provides RPC and Notifications API, they are part of the same library
for mostly historical reasons. One of the major goals of *oslo.messaging* is
to provide clean RPC and Notification API without any trace of messaging
queue concepts (but two of most advanced drivers used by oslo.messaging is
actually based on AMQP: RabbitMQ and QPID).

We were designing Murano on messaging queue concepts using some
AMQP/RabbitMQ specific features, like queue TTL. Since we never considered
communications between our components in terms of RPC or Notifications and
always thought about them as message exchange through broker it has
influenced our components architecture. In Murano we use simple
wrapperhttps://github.com/stackforge/murano-common/tree/master/muranocommon/messaging
around Puka https://github.com/majek/puka (RabbitMQ client with most
simple and thoughtful async model) that is used in all our components.
We forked
Puka https://github.com/istalker2/puka since we had specific requirements
to SSL and could not yet merge our
workhttps://github.com/majek/puka/pull/43 back
to master.

Can we abandon our own
wrapperhttps://github.com/stackforge/murano-common/tree/master/muranocommon/messagingaround
our own
fork of Puka https://github.com/istalker2/puka in favor of
*oslo.messaging*? *Yes*, but this migration may be tricky. I believe we can
migrate to *oslo.messaging* in a week or so*.*

I had played with *oslo.messaging* emulating our current communication
patterns with *oslo.messaging*, and I am certain that current
implementation can be migrated to *oslo.messaging**. *But I am not sure
that *oslo.messaging* may be easily suited to all future use-cases that we
plan to cover in a few next releases without major contributions. Please,
try to respond with any questions related to *oslo.messaging* implementation
and how it can be fitted with certain use-case.

Below, I tried to describe our current use-cases and what specific MQ
features we are using, how they may be implemented with *oslo.messaging *and
with what limitations we will face.

Use-Case
Murano has several components with communications between them based on
messaging queue:
*murano-api* - *murano-conductor:*

1. *murano-api* sends deployment tasks to murano-conductor

*murano-conductor* - *murano-api:*

1. *murano-conductor* reports to *murano-api* task progress
during processing
2. after processing, *murano-conductor* sends results to *murano-api*

*murano-conductor *-* murano-agent:*

1. during task processing *murano-conductor* sends execution plans with
commands to *murano-agent.*

Note: each of mentioned components above may have more than one instance.

One of great messaging queue specific that we heavily use is a idea of
queue itself, messages sent to component will be handled any time soon as
at least one instance would be started. For example, in case of
*murano-agent*, message is sent even before *murano-agent* is started.
Another one is queue life-time, we control life-time of
*murano-agent*queues to exclude overflow of MQ server with queues that
is not used
anymore.

One thing is also worse to mention: *murano-conductor* communicates with
several components at the same time: process several tasks at the same
time, during task processing *murano-conductor* sends progress
notifications to *murano-api* and execution plans to *murano-agent*.

Implementation
Please, refer to
Conceptshttps://wiki.openstack.org/wiki/Oslo/Messaging#Concepts
section of *oslo.messaging* Wiki before further reading to grasp key
concepts expressed in *oslo.messaging* library. In short, using RPC API we
can 'call' server synchronously and receive some result, or 'cast'
asynchronously (no result is returned). Using Notification API we can send
Notification to the specified Target about happened event with specified
event_type, importance and payload.

If we move to *oslo.messaging* we can only primarily rely on features
provided by RPC/Notifications model:

1. We should not rely on message delivery without other side is properly
up and running. It is not a message delivery, it is *Remote Procedure
Call;*
2. To control queue life-time as we do now, we may be required to 'hack'
*oslo.messaging* by writing own driver.

*murano-api* - *murano-conductor:*

1. *murano-api* sends deployment tasks to murano-conductor: *May be
replaced with RPC Cast*

*murano-conductor* - *murano-api:*

1. *murano-conductor* reports to *murano-api* task progress
during processing: *May be replaced with Notification* or *RPC Cast*
2. after processing, *murano-conductor* sends results to *murano-api: **May
be replaced with RPC Cast*

*murano-conductor *-* murano-agent:*

1. during task processing *murano-conductor* sends execution plans with
commands to *murano-agent*: *May be replaced with two way RPC Cast*
(murano-agent Cast
to murano-conductor with message like 'I

[openstack-dev] [Neutron][IPv6] NAT64 Discussion

2014-02-11 Thread Xuhan Peng

In previous Neutron IPv6 team meeting, we discussed the requirement of
providing NAT64 in OpenStack to facilitate the communication between IPv6
and IPv4 hosts. For example, from tenant IPv6 network to external
traditional IPv4 network. I was suggested to initiate a discussion in ML to
gather all forks thoughts.

I wonder if this is a fairly common requirement and what is the good way to
add this possibility to OpenStack. Several ways we mentioned in previous
sub-team meeting:

1. Add to current Neutron L3
This was also asked by Martinx in [1].
2. Run as service VM
3. Maybe even a new sub-project/project dedicated for address translation

I also want to mention that NAT64 may not be the only requirement to
communicate between IPv6 and IPv4 hosts. 6in4 tunneling and other methods
are als candidates.

[1]
http://lists.openstack.org/pipermail/openstack-dev/2014-January/023265.html

Your comments are appreciated!

Xuhan Peng (irc: xuhanp)
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] Fwd: [Openstack] [GSoC] Call for Mentors and Participants

2014-02-11 Thread Davanum Srinivas

Hi all,

Apologies if you saw this already. Since we have a really tight
deadline to drum up mentor and student participation, forwarding this
once to the openstack-dev mailing list. The deadline is this Friday
the 14th.

thanks,
dims

On Sun, Feb 9, 2014 at 5:34 PM, Davanum Srinivas dava...@gmail.com wrote:

 Hi everyone,

 Anyone wishing to get involved, please flesh out the wiki with topics,
 ideas or by listing your name as a mentor or someone looking for a
 mentee. This is the most important part of our application, we need to
 show enough mentors and project proposals for OpenStack to be accepted
 as a participating Organization. Please don't forget to add your
 contact details, link to blueprints, reviews, anything that would help
 flesh out topics/projects/ideas.

 https://wiki.openstack.org/wiki/GSoC2014

 Anyone wishing to help fill up the application, Here's our etherpad
 for our Application for GSoC 2014. Many thanks to Anne Gentle for
 digging up our application for 2012.

 https://etherpad.openstack.org/p/gsoc2014orgapp

 Thanks,
 Dims.

 ___
 Mailing list: http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
 Post to : openst...@lists.openstack.org
 Unsubscribe : http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack




--
-Debo~


-- 
Davanum Srinivas :: http://davanum.wordpress.com

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] OpenStack-dev Digest, Vol 22, Issue 27

2014-02-11 Thread Abishek Subramanian (absubram)

Hi Shixiong,

Thanks for the reply and clearing up my question!
The document you've shared - they aren't the only possible combinations
correct?
Each of the two fields can have any one of these four values in any
possible combination yes?

Thanks!

On 2/11/14 7:00 AM, openstack-dev-requ...@lists.openstack.org
openstack-dev-requ...@lists.openstack.org wrote:

Hi, Abishek:

Thank you for taking care of Horizon for IPv6 enhancement. So now we have
coverage on both CLI and dashboard side. Very exciting!

W.r.t your questions, these two parameters work independently. In other
words, Horizon should present both options if the interested subnet is
IPv6. For each parameter, the valid values are:
   off
   slacc
   dhcpv6-stateful
   dhcpv6-stateless

The CLI command may look like, for example, something below:

neutron subnet-create --ip-version 6 --ipv6_ra_mode off
--ipv6_address_mode off NETWORK CIDR
neutron subnet-create --ip-version 6 --ipv6_ra_mode off
--ipv6_address_mode dhcpv6-stateful NETWORK CIDR
neutron subnet-create --ip-version 6 --ipv6_ra_mode slaac
--ipv6_address_mode slaac NETWORK CIDR
neutron subnet-create --ip-version 6 --ipv6_ra_mode dhcpv6-stateful
--ipv6_address_mode off NETWORK CIDR
neutron subnet-create --ip-version 6 --ipv6_ra_mode dhcpv6-stateless
--ipv6_address_mode dhcpv6-stateless NETWORK CIDR


The valid combinations are outlined in the PDF file below.

https://www.dropbox.com/s/9bojvv9vywsz8sd/IPv6%20Two%20Modes%20v3.0.pdf

Please let me know if you have any further questions. Thanks!

Shixiong




___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Murano] Can we migrate to oslo.messaging?

On Tue, Feb 11, 2014 at 8:05 AM, Serg Melikyan smelik...@mirantis.comwrote:

We were designing Murano on messaging queue concepts using some
AMQP/RabbitMQ specific features, like queue TTL. Since we never considered
communications between our components in terms of RPC or Notifications and
always thought about them as message exchange through broker it has
influenced our components architecture. In Murano we use simple
wrapperhttps://github.com/stackforge/murano-common/tree/master/muranocommon/messaging
around Puka https://github.com/majek/puka (RabbitMQ client with most
simple and thoughtful async model) that is used in all our components. We
forked
Puka https://github.com/istalker2/puka since we had specific
requirements to SSL and could not yet merge our
workhttps://github.com/majek/puka/pull/43 back
to master.

Can we abandon our own
wrapperhttps://github.com/stackforge/murano-common/tree/master/muranocommon/messagingaround
our own
fork of Puka https://github.com/istalker2/puka in favor of
*oslo.messaging*? *Yes*, but this migration may be tricky. I believe we
can migrate to *oslo.messaging* in a week or so*.*

I had played with *oslo.messaging* emulating our current communication
patterns with *oslo.messaging*, and I am certain that current
implementation can be migrated to *oslo.messaging**. *But I am not sure
that *oslo.messaging* may be easily suited to all future use-cases that
we plan to cover in a few next releases without major contributions.
Please, try to respond with any questions related to *oslo.messaging*
implementation
and how it can be fitted with certain use-case.

Below, I tried to describe our current use-cases and what specific MQ
features we are using, how they may be implemented with *oslo.messaging *and
with what limitations we will face.

Use-Case
Murano has several components with communications between them based on
messaging queue:
*murano-api* - *murano-conductor:*

1. *murano-api* sends deployment tasks to murano-conductor

*murano-conductor* - *murano-api:*

1. *murano-conductor* reports to *murano-api* task progress
during processing
2. after processing, *murano-conductor* sends results to *murano-api*

*murano-conductor *-* murano-agent:*

1. during task processing *murano-conductor* sends execution plans
with commands to *murano-agent.*

Note: each of mentioned components above may have more than one instance.

One of great messaging queue specific that we heavily use is a idea of
queue itself, messages sent to component will be handled any time soon as
at least one instance would be started. For example, in case of
*murano-agent*, message is sent even before *murano-agent* is started.
Another one is queue life-time, we control life-time of *murano-agent*queues
to exclude overflow of MQ server with queues that is not used
anymore.

Implementation
Please, refer to
Conceptshttps://wiki.openstack.org/wiki/Oslo/Messaging#Concepts
section of *oslo.messaging* Wiki before further reading to grasp key
concepts expressed in *oslo.messaging* library. In short, using RPC API
we can 'call' server synchronously and receive some result, or 'cast'
asynchronously (no result is returned). Using Notification API we can send
Notification to the specified Target about happened event with specified
event_type, importance and payload.

If we move to *oslo.messaging* we can only primarily rely on features
provided by RPC/Notifications model:

1. We should not rely on message delivery without other side is
properly up and running. It is not a message delivery, it is *Remote
Procedure Call;*

The RPC modules expect a response and provide timeout behavior, but the
notifications don't require that. Perhaps you could send those messages as
notifications?

1.
2. To control queue life-time as we do now, we may be required to
'hack' *oslo.messaging* by writing own driver.

The library is meant to be reusable, so if the API does not support your
use case please work with us to extend it rather than hacking around it or
forking it.

*murano-api* - *murano-conductor:*

1. *murano-api* sends deployment tasks to murano-conductor: *May be
replaced with RPC Cast*

[openstack-dev] [Nova][Glance]supporting of v1 and v2 glance APIs in Nova

2014-02-11 Thread Lingxian Kong

Greetings

A few days ago, I met some problems when using 'createimage' feature in
Nova, we found that using V1 of glanceclient has some problem with
processing of metadata, the version number and even the glance URIs are
both hardcoded in Nova.

then, we found the bluepring[1] proposed, and the maillist[2] which talked
about the topic before, mainly focused on version autodiscovery by keystone
catalog and config option for nova. But we still need changes in Nova
because the incompatible behavior between v1 and v2, especially when
creating and uploading an image file. The review request[3] of the bp is
abandoned for now.

So, what I want to confirm is, how could this situation be handled? I
mailed Eddie Sheffield, but got no answer, so bring it up here.

[1]: https://blueprints.launchpad.net/nova/+spec/use-glance-v2-api
[2]: http://markmail.org/message/uqrpufsmh4qp5pgy
[4]: https://review.openstack.org/#/c/38414/

-- 
*---*
*Lingxian Kong*
Huawei Technologies Co.,LTD.
IT Product Line CloudOS PDU
China, Xi'an
Mobile: +86-18602962792
Email: konglingx...@huawei.com; anlin.k...@gmail.com
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Mistral]

2014-02-11 Thread Renat Akhmerov

Dmitry, I think you are right here. I think for simple case we should be able
to use in-place action definition without having to define the action
separately. Like you said it’s only valuable if we need to reuse it.

The only difference I see between std:send-email and something like REST_API is
that a set of parameters for the latter is dynamic (versus std:send-email where
it’s always “recipients”, “subject”, “body”). Even though it’s still the same
protocol (HTTP) but a particular request representation may be different (i.e.
query string, headers, the structure of body in case POST etc.). But I think
that doesn’t cancel the idea of being able to define the action along with the
task itself.

So good point. As for the syntax itself, we need to think it over. In the
snippet you provided “action: std:REST_API”, so we need to make sure not to
have ambiguities in the ways how we can refer actions. A convention could be
“if we don’t use a namespace we assume that there’s a separate action
definition included into the same workbook, otherwise it should be considered
in-place action definition and task property “action” refers to an action type
rather than the action itself”. Does that make sense?

Renat Akhmerov
@ Mirantis Inc.

On 11 Feb 2014, at 16:23, Dmitri Zimine d...@stackstorm.com wrote:

Do we have (or think about) a shorthand to calling REST_API action, without
defining a service?

FULL DSL:

Services:
TimeService:
type: REST_API
parameters:
baseUrl:http://api.timezonedb.com
key:my_api_key
actions:
get-time:
task-parameters:
zone:
Workflow:
tasks:
timeInToronto:
action: TimeService:get-time
parameters:
zone: America/Toronto

SHORTCUT - may look something like this:

Workflow:
tasks:
timeInToronto:
action:std:REST_API
parameters:
baseUrl: http://api.timezonedb.com;
method: GET
parameters: zone=/America/Torontokey=my_api_key

Why asking:

1) analogy with std:send-email action. I wonder do we have to make user
define Service for std:send-email? and I think that for standard tasks we
shouldn't have to. If there is any thinking on REST_API, it may apply here.

2) For a one-off web service calls the complete syntax is may be overkill
(but yes, it comes handy for reuse). See examples below.

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Murano] Can we migrate to oslo.messaging?

2014-02-11 Thread Serg Melikyan

Doug, thank you for your response!

With moving to *oslo.messaging *I think we need to carefully
re-evaluate our communication patterns and try to either fit them to
RPC/Notification, or work on extending *oslo.messaging *with some new ideas.
Adopting a new library is always a challenge.

The RPC modules expect a response and provide timeout behavior, but the
notifications don't require that. Perhaps you could send those messages as
notifications?
Yes, notifications can be used as replacement to sending messages, but I
think this may be even broken design than try to replace messaging with RPC
without actual communications re-design.

The library is meant to be reusable, so if the API does not support your
use case please work with us to extend it rather than hacking around it or
forking it
Sure, I have no intents to hack/change/extend/fork anything related to
oslo.messaging without consulting with oslo team first. I understand that
making tools used by OpenStack better is very important to all of us.

When talked about 'hacking' I had in mind a quick way to move to
*oslo.messaging* without losing any of existing functionality. Since
oslo.messaging meant to hide away underlying implementation and message
queue related things I am not sure that adding such specific thing as
queue-ttl for selected queues without extending some classes from driver
layer is possible.

On Tue, Feb 11, 2014 at 6:06 PM, Doug Hellmann
doug.hellm...@dreamhost.comwrote:

On Tue, Feb 11, 2014 at 8:05 AM, Serg Melikyan smelik...@mirantis.comwrote:

We were designing Murano on messaging queue concepts using some
AMQP/RabbitMQ specific features, like queue TTL. Since we never considered
communications between our components in terms of RPC or Notifications and
always thought about them as message exchange through broker it has
influenced our components architecture. In Murano we use simple
wrapperhttps://github.com/stackforge/murano-common/tree/master/muranocommon/messaging
around Puka https://github.com/majek/puka (RabbitMQ client with most
simple and thoughtful async model) that is used in all our components. We
forked
Puka https://github.com/istalker2/puka since we had specific
requirements to SSL and could not yet merge our
workhttps://github.com/majek/puka/pull/43 back
to master.

Can we abandon our own
wrapperhttps://github.com/stackforge/murano-common/tree/master/muranocommon/messagingaround
our own
fork of Puka https://github.com/istalker2/puka in favor of
*oslo.messaging*? *Yes*, but this migration may be tricky. I believe we
can migrate to *oslo.messaging* in a week or so*.*

I had played with *oslo.messaging* emulating our current communication
patterns with *oslo.messaging*, and I am certain that current
implementation can be migrated to *oslo.messaging**. *But I am not sure
that *oslo.messaging* may be easily suited to all future use-cases that
we plan to cover in a few next releases without major contributions.
Please, try to respond with any questions related to *oslo.messaging*
implementation
and how it can be fitted with certain use-case.

Below, I tried to describe our current use-cases and what specific MQ
features we are using, how they may be implemented with *oslo.messaging *and
with what limitations we will face.

Use-Case
Murano has several components with communications between them based on
messaging queue:
*murano-api* - *murano-conductor:*

1. *murano-api* sends deployment tasks to murano-conductor

*murano-conductor* - *murano-api:*

1. *murano-conductor* reports to *murano-api* task progress
during processing
2. after processing, *murano-conductor* sends results to *murano-api*

*murano-conductor *-* murano-agent:*

1. during task processing *murano-conductor* sends execution plans
with commands to *murano-agent.*

Note: each of mentioned components above may have more than one instance.

One of great messaging queue specific that we heavily use is a idea of
queue itself, messages sent to component will be handled any time soon as
at least one instance would be started. For example, in case of
*murano-agent*, message is sent even before *murano-agent* is started.
Another one is queue life-time, we control life-time of *murano-agent*queues
to exclude overflow of MQ server with queues that is not used
anymore.

One thing is also worse to mention: *murano-conductor* communicates with
several components at the same time: process several tasks at the same
time, during task processing *murano-conductor* sends

Re: [openstack-dev] [Nova][Scheduler] Policy Based Scheduler and Solver Scheduler

2014-02-11 Thread Chris Friesen


On 02/11/2014 03:21 AM, Khanh-Toan Tran wrote:

Second, there is nothing wrong with booting the instances (or

instantiating other

resources) as separate commands as long as we support some kind of
reservation token.


I'm not sure what reservation token would do, is it some kind of informing
the scheduler that the resources would not be initiated until later ?


Like a restaurant reservation, it would claim the resources for use by 
someone at a later date.  That way nobody else can use them.


That way the scheduler would be responsible for determining where the 
resource should be allocated from, and getting a reservation for that 
resource.  It would not have anything to do with actually instantiating 
the instance/volume/etc.



Let's consider a following example:

A user wants to create 2 VMs, a small one with 20 GB RAM, and a big one
with 40 GB RAM in a datacenter consisted of 2 hosts: one with 50 GB RAM
left, and another with 30 GB RAM left, using Filter Scheduler's default
RamWeigher.

If we pass the demand as two commands, there is a chance that the small VM
arrives first. RamWeigher will put it in the 50 GB RAM host, which will be
reduced to 30 GB RAM. Then, when the big VM request arrives, there will be
no space left to host it. As a result, the whole demand is failed.

Now if we can pass the two VMs in a command, SolverScheduler can put their
constraints all together into one big LP as follow (x_uv = 1 if VM u is
hosted in host v, 0 if not):


Yes.  So what I'm suggesting is that we schedule the two VMs as one call 
to the SolverScheduler.  The scheduler then gets reservations for the 
necessary resources and returns them to the caller.  This would be sort 
of like the existing Claim object in nova/compute/claims.py but 
generalized somewhat to other resources as well.


The caller could then boot each instance separately (passing the 
appropriate reservation/claim along with the boot request).  Because the 
caller has a reservation the core code would know it doesn't need to 
schedule or allocate resources, that's already been done.


The advantage of this is that the scheduling and resource allocation is 
done separately from the instantiation.  The instantiation API could 
remain basically as-is except for supporting an optional reservation token.



That responses to your first point, too. If we don't mind that some VMs
are placed and some are not (e.g. they belong to different apps), then
it's OK to pass them to the scheduler without Instance Group. However, if
the VMs are together (belong to an app), then we have to put them into an
Instance Group.


When I think of an Instance Group, I think of 
https://blueprints.launchpad.net/nova/+spec/instance-group-api-extension;. 
  Fundamentally Instance Groups describes a runtime relationship 
between different instances.


The scheduler doesn't necessarily care about a runtime relationship, 
it's just trying to allocate resources efficiently.


In the above example, there is no need for those two instances to 
necessarily be part of an Instance Group--we just want to schedule them 
both at the same time to give the scheduler a better chance of fitting 
them both.


More generally, the more instances I want to start up the more 
beneficial it can be to pass them all to the scheduler at once in order 
to give the scheduler more information.  Those instances could be parts 
of completely independent Instance Groups, or not part of an Instance 
Group at all...the scheduler can still do a better job if it has more 
information to work with.


Chris

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Openstack-stable-maint] Stable gate status?

2014-02-11 Thread Anita Kuno

On 02/11/2014 04:57 AM, Alan Pevec wrote:
 Hi Mark and Anita,
 
 could we declare stable/havana neutron gate jobs good enough at this point?
 There are still random failures as this no-op change shows
 https://review.openstack.org/72576
 but I don't think they're stable/havana specific.
 
 Do we have a list of those somewhere?
 Pulled out where following Neutron patches (IMHO all innocent for gate
 breaking):
 https://review.openstack.org/62206
 https://review.openstack.org/67214
 https://review.openstack.org/70232
 
 I've resubmitted those without Removed from the gate... lines in the
 commit message, waiting for rechecks now.
 
 I'm particularly interested in https://review.openstack.org/#/c/66149/ as a 
 fix for
 https://bugs.launchpad.net/keystone/+bug/1251123
 
 This one is last remaining exception request for 2013.2.2 and is
 waiting for the master change to be reviewed:
 https://review.openstack.org/#/q/Ida39b4699ed6c568609a5121573fc3be5c4ab2f4,n,z
 I hope keystone core could review this one quickly so that backport
 can be updated and merged.
 
 Thanks!
 Alan
 
I will reaffirm here what I had stated in IRC.

If Mark McClain gives his assent for stable/havana patches to be
approved, I will not remove Neutron stable/havana patches from the gate
queue before they start running tests. If after they start running
tests, they demonstrate that they are failing, I will remove them from
the gate as a means to keep the gate flowing. If the stable/havana gate
jobs are indeed stable, I will not be removing any patches that should
be merged.

Adding commit lines is the fastest way to submit a new patchset without
affecting the code in the patch (hence removing it from the gate queue),
so thank you for removing those additional lines in the commit message.

Thank you,
Anita.

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Heat] in-instance update hooks

2014-02-11 Thread Steven Dake


On 02/10/2014 10:22 PM, Clint Byrum wrote:

Hi, so in the previous thread about rolling updates it became clear that
having in-instance control over updates is a more fundamental idea than
I had previously believed. During an update, Heat does things to servers
that may interrupt the server's purpose, and that may cause it to fail
subsequent things in the graph.

Specifically, in TripleO we have compute nodes that we are managing.
Before rebooting a machine, we want to have a chance to live-migrate
workloads if possible, or evacuate in the simpler case, before the node
is rebooted. Also in the case of a Galera DB where we may even be running
degraded, we want to ensure that we have quorum before proceeding.

I've filed a blueprint for this functionality:

https://blueprints.launchpad.net/heat/+spec/update-hooks

I've cobbled together a spec here, and I would very much welcome
edits/comments/etc:

https://etherpad.openstack.org/p/heat-update-hooks

Clint,

I read through your etherpad and think there is a relationship to a use 
case for scaling.  It would be sweet if both these cases could use the 
same model and tooling.  At the moment in an autoscaling group, when you 
want to scale down there is no way to quiesce the node before killing 
the VM.  It is the same problem you have with Galera, except your 
scenario involves an update.


I'm not clear how the proposed design could be made to fit this 
particular use case.  Do you see a way it can fill both roles so we 
don't have two different ways to do essentially the same thing?


Regards
-steve


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Nova][Scheduler] Policy Based Scheduler and Solver Scheduler


Le 11/02/2014 16:09, Chris Friesen a écrit :


Yes.  So what I'm suggesting is that we schedule the two VMs as one 
call to the SolverScheduler.  The scheduler then gets reservations for 
the necessary resources and returns them to the caller.  This would be 
sort of like the existing Claim object in nova/compute/claims.py but 
generalized somewhat to other resources as well.


The caller could then boot each instance separately (passing the 
appropriate reservation/claim along with the boot request). Because 
the caller has a reservation the core code would know it doesn't need 
to schedule or allocate resources, that's already been done.


The advantage of this is that the scheduling and resource allocation 
is done separately from the instantiation.  The instantiation API 
could remain basically as-is except for supporting an optional 
reservation token.


I think you really need to look at what Climate is, but I don't want to 
be boring...


Climate can provide you some way for doing the reservation, and possibly 
you would only need to write a plugin for such a request.


-Sylvain


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [Murano] Community meeting agenda - 02/11/2014

2014-02-11 Thread Alexander Tivelkov

Hi,

This is just a reminder that we are going to have a weekly meeting of
Murano team in IRC (#openstack-meeting-alt) today at 17:00 UTC (9am PST) .

The agenda can be found here:
https://wiki.openstack.org/wiki/Meetings/MuranoAgenda#Agenda

Feel free to add anything you want to discuss.

--
Regards,
Alexander Tivelkov
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Nova][Scheduler] Policy Based Scheduler and Solver Scheduler

2014-02-11 Thread Dina Belova


 Like a restaurant reservation, it would claim the resources for use by
 someone at a later date.  That way nobody else can use them.
 That way the scheduler would be responsible for determining where the
 resource should be allocated from, and getting a reservation for that
 resource.  It would not have anything to do with actually instantiating the
 instance/volume/etc.


Although I'm quite new to topic of Solver Scheduler, it seems to me that in
that case you need to look on Climate project. It aims to provide resource
reservation to OS clouds (and by resource I mean here instance/compute
host/volume/etc.)

And Climate logic is like: create lease - get resources from common pool -
do smth with them when lease start time will come.

I'll say one more time - I'm not really common with this discussion, but it
looks like Climate might help here.

Thanks
Dina


On Tue, Feb 11, 2014 at 7:09 PM, Chris Friesen
chris.frie...@windriver.comwrote:

 On 02/11/2014 03:21 AM, Khanh-Toan Tran wrote:

 Second, there is nothing wrong with booting the instances (or

 instantiating other

 resources) as separate commands as long as we support some kind of
 reservation token.


 I'm not sure what reservation token would do, is it some kind of informing
 the scheduler that the resources would not be initiated until later ?


 Like a restaurant reservation, it would claim the resources for use by
 someone at a later date.  That way nobody else can use them.

 That way the scheduler would be responsible for determining where the
 resource should be allocated from, and getting a reservation for that
 resource.  It would not have anything to do with actually instantiating the
 instance/volume/etc.


  Let's consider a following example:

 A user wants to create 2 VMs, a small one with 20 GB RAM, and a big one
 with 40 GB RAM in a datacenter consisted of 2 hosts: one with 50 GB RAM
 left, and another with 30 GB RAM left, using Filter Scheduler's default
 RamWeigher.

 If we pass the demand as two commands, there is a chance that the small VM
 arrives first. RamWeigher will put it in the 50 GB RAM host, which will be
 reduced to 30 GB RAM. Then, when the big VM request arrives, there will be
 no space left to host it. As a result, the whole demand is failed.

 Now if we can pass the two VMs in a command, SolverScheduler can put their
 constraints all together into one big LP as follow (x_uv = 1 if VM u is
 hosted in host v, 0 if not):


 Yes.  So what I'm suggesting is that we schedule the two VMs as one call
 to the SolverScheduler.  The scheduler then gets reservations for the
 necessary resources and returns them to the caller.  This would be sort of
 like the existing Claim object in nova/compute/claims.py but generalized
 somewhat to other resources as well.

 The caller could then boot each instance separately (passing the
 appropriate reservation/claim along with the boot request).  Because the
 caller has a reservation the core code would know it doesn't need to
 schedule or allocate resources, that's already been done.

 The advantage of this is that the scheduling and resource allocation is
 done separately from the instantiation.  The instantiation API could remain
 basically as-is except for supporting an optional reservation token.


  That responses to your first point, too. If we don't mind that some VMs
 are placed and some are not (e.g. they belong to different apps), then
 it's OK to pass them to the scheduler without Instance Group. However, if
 the VMs are together (belong to an app), then we have to put them into an
 Instance Group.


 When I think of an Instance Group, I think of 
 https://blueprints.launchpad.net/nova/+spec/instance-group-api-extension;.
   Fundamentally Instance Groups describes a runtime relationship between
 different instances.

 The scheduler doesn't necessarily care about a runtime relationship, it's
 just trying to allocate resources efficiently.

 In the above example, there is no need for those two instances to
 necessarily be part of an Instance Group--we just want to schedule them
 both at the same time to give the scheduler a better chance of fitting them
 both.

 More generally, the more instances I want to start up the more beneficial
 it can be to pass them all to the scheduler at once in order to give the
 scheduler more information.  Those instances could be parts of completely
 independent Instance Groups, or not part of an Instance Group at all...the
 scheduler can still do a better job if it has more information to work with.


 Chris

 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev




-- 

Best regards,

Dina Belova

Software Engineer

Mirantis Inc.
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] Hyper-v meeting cancelled.

2014-02-11 Thread Peter Pouliot

Hi all.
We are currently working to bring additional compute power to the hyper-v ci 
infrastructure.
Because this task currently takes precedence I need to cancel the meeting for 
today.

P

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Nova][Scheduler] Policy Based Scheduler and Solver Scheduler

2014-02-11 Thread Khanh-Toan Tran

Thanks, I will look closely at it.



De : Dina Belova [mailto:dbel...@mirantis.com]
Envoyé : mardi 11 février 2014 16:45
À : OpenStack Development Mailing List (not for usage questions)
Objet : Re: [openstack-dev] [Nova][Scheduler] Policy Based Scheduler and
Solver Scheduler



Like a restaurant reservation, it would claim the resources for use by
someone at a later date.  That way nobody else can use them.
That way the scheduler would be responsible for determining where the
resource should be allocated from, and getting a reservation for that
resource.  It would not have anything to do with actually instantiating
the instance/volume/etc.



Although I'm quite new to topic of Solver Scheduler, it seems to me that
in that case you need to look on Climate project. It aims to provide
resource reservation to OS clouds (and by resource I mean here
instance/compute host/volume/etc.)



And Climate logic is like: create lease - get resources from common pool -
do smth with them when lease start time will come.



I'll say one more time - I'm not really common with this discussion, but
it looks like Climate might help here.



Thanks

Dina



On Tue, Feb 11, 2014 at 7:09 PM, Chris Friesen
chris.frie...@windriver.com wrote:

On 02/11/2014 03:21 AM, Khanh-Toan Tran wrote:

Second, there is nothing wrong with booting the instances (or

instantiating other

resources) as separate commands as long as we support some kind of
reservation token.


I'm not sure what reservation token would do, is it some kind of informing
the scheduler that the resources would not be initiated until later ?



Like a restaurant reservation, it would claim the resources for use by
someone at a later date.  That way nobody else can use them.

That way the scheduler would be responsible for determining where the
resource should be allocated from, and getting a reservation for that
resource.  It would not have anything to do with actually instantiating
the instance/volume/etc.



Let's consider a following example:

A user wants to create 2 VMs, a small one with 20 GB RAM, and a big one
with 40 GB RAM in a datacenter consisted of 2 hosts: one with 50 GB RAM
left, and another with 30 GB RAM left, using Filter Scheduler's default
RamWeigher.

If we pass the demand as two commands, there is a chance that the small VM
arrives first. RamWeigher will put it in the 50 GB RAM host, which will be
reduced to 30 GB RAM. Then, when the big VM request arrives, there will be
no space left to host it. As a result, the whole demand is failed.

Now if we can pass the two VMs in a command, SolverScheduler can put their
constraints all together into one big LP as follow (x_uv = 1 if VM u is
hosted in host v, 0 if not):



Yes.  So what I'm suggesting is that we schedule the two VMs as one call
to the SolverScheduler.  The scheduler then gets reservations for the
necessary resources and returns them to the caller.  This would be sort of
like the existing Claim object in nova/compute/claims.py but generalized
somewhat to other resources as well.

The caller could then boot each instance separately (passing the
appropriate reservation/claim along with the boot request).  Because the
caller has a reservation the core code would know it doesn't need to
schedule or allocate resources, that's already been done.

The advantage of this is that the scheduling and resource allocation is
done separately from the instantiation.  The instantiation API could
remain basically as-is except for supporting an optional reservation
token.



That responses to your first point, too. If we don't mind that some VMs
are placed and some are not (e.g. they belong to different apps), then
it's OK to pass them to the scheduler without Instance Group. However, if
the VMs are together (belong to an app), then we have to put them into an
Instance Group.



When I think of an Instance Group, I think of
https://blueprints.launchpad.net/nova/+spec/instance-group-api-extension;
.   Fundamentally Instance Groups describes a runtime relationship
between different instances.

The scheduler doesn't necessarily care about a runtime relationship, it's
just trying to allocate resources efficiently.

In the above example, there is no need for those two instances to
necessarily be part of an Instance Group--we just want to schedule them
both at the same time to give the scheduler a better chance of fitting
them both.

More generally, the more instances I want to start up the more beneficial
it can be to pass them all to the scheduler at once in order to give the
scheduler more information.  Those instances could be parts of completely
independent Instance Groups, or not part of an Instance Group at all...the
scheduler can still do a better job if it has more information to work
with.



Chris

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev







--

Best regards,

Dina Belova

Re: [openstack-dev] [Nova][Scheduler] Policy Based Scheduler and Solver Scheduler

2014-02-11 Thread Yathiraj Udupi (yudupi)

I thought of adding some more points about the Solver Scheduler to this
conversation.
Think of SolverScheduler as a placement decision engine, which gives an
optimal solution for the specified request based on the current
information available at a specific time.  The request could potentially
be a set of instances of the same flavor or different flavors (assuming we
eventually support scheduler APIs that can provide this).
Once the optimal placement decision is known, then we need to allocate the
resources. Currently Nova supports the final allocation of resources
(resource provisioning) one at a time.  I definitely agree there will be
more success in allocating all the requested instances, if there is a
support for reserving the spots as soon as a placement decision is taken
by the SolverScheduler. This is something to explore to add in Nova, using
a local service or external service (need to explore Climate).

If the required set of instances is known, irrespective of whether it is
part of one instance group, or multiple instance groups, you always get a
more optimal solution, if all of this is requested in one shot to the
solver scheduler, as the constraint solving happens in one shot, letting
you know if the entire set of instances is feasible to be placed given the
existing resource capacity.
If it is okay to support partial instantiation of subset of instances,
then it makes sense to provide support to retry one instance group at a
time, when the entire request is not feasible.
To add another point about instance group api implementation for icehouse,
it was decided in the HongKong summit to initially support only flat
instance groups without nesting. Hence if an application requires a big
topology of instances, they can easily belong to multiple instance groups,
and hence if you want the entire application requirement to be satisfied,
the entire set of instances from multiple flat instance groups should be
requested as a single shot to the solver scheduler.  Also, there is
additional work required to add new scheduler APIs to support requesting
instance groups of multiple flavors.

I think I have reiterated some of the points what Chris has mentioned
below.  But Yes, like I had stated earlier in this thread, we need to
separate the decision making phase from the initial request making, and
the final allocation provisioning (or orchestration).  In these phases, a
reservation phase, after the decision making,  will add additional
guarantees to allocate the placed instances.

Thanks,
Yathi. 



On 2/11/14, 7:09 AM, Chris Friesen chris.frie...@windriver.com wrote:

On 02/11/2014 03:21 AM, Khanh-Toan Tran wrote:
 Second, there is nothing wrong with booting the instances (or
 instantiating other
 resources) as separate commands as long as we support some kind of
 reservation token.

 I'm not sure what reservation token would do, is it some kind of
informing
 the scheduler that the resources would not be initiated until later ?

Like a restaurant reservation, it would claim the resources for use by
someone at a later date.  That way nobody else can use them.

That way the scheduler would be responsible for determining where the
resource should be allocated from, and getting a reservation for that
resource.  It would not have anything to do with actually instantiating
the instance/volume/etc.

 Let's consider a following example:

 A user wants to create 2 VMs, a small one with 20 GB RAM, and a big one
 with 40 GB RAM in a datacenter consisted of 2 hosts: one with 50 GB RAM
 left, and another with 30 GB RAM left, using Filter Scheduler's default
 RamWeigher.

 If we pass the demand as two commands, there is a chance that the small
VM
 arrives first. RamWeigher will put it in the 50 GB RAM host, which will
be
 reduced to 30 GB RAM. Then, when the big VM request arrives, there will
be
 no space left to host it. As a result, the whole demand is failed.

 Now if we can pass the two VMs in a command, SolverScheduler can put
their
 constraints all together into one big LP as follow (x_uv = 1 if VM u is
 hosted in host v, 0 if not):

Yes.  So what I'm suggesting is that we schedule the two VMs as one call
to the SolverScheduler.  The scheduler then gets reservations for the
necessary resources and returns them to the caller.  This would be sort
of like the existing Claim object in nova/compute/claims.py but
generalized somewhat to other resources as well.

The caller could then boot each instance separately (passing the
appropriate reservation/claim along with the boot request).  Because the
caller has a reservation the core code would know it doesn't need to
schedule or allocate resources, that's already been done.

The advantage of this is that the scheduling and resource allocation is
done separately from the instantiation.  The instantiation API could
remain basically as-is except for supporting an optional reservation
token.

 That responses to your first point, too. If we don't mind that some VMs
 are

Re: [openstack-dev] [Heat] in-instance update hooks

Excerpts from Steven Dake's message of 2014-02-11 07:19:19 -0800:
 On 02/10/2014 10:22 PM, Clint Byrum wrote:
  Hi, so in the previous thread about rolling updates it became clear that
  having in-instance control over updates is a more fundamental idea than
  I had previously believed. During an update, Heat does things to servers
  that may interrupt the server's purpose, and that may cause it to fail
  subsequent things in the graph.
 
  Specifically, in TripleO we have compute nodes that we are managing.
  Before rebooting a machine, we want to have a chance to live-migrate
  workloads if possible, or evacuate in the simpler case, before the node
  is rebooted. Also in the case of a Galera DB where we may even be running
  degraded, we want to ensure that we have quorum before proceeding.
 
  I've filed a blueprint for this functionality:
 
  https://blueprints.launchpad.net/heat/+spec/update-hooks
 
  I've cobbled together a spec here, and I would very much welcome
  edits/comments/etc:
 
  https://etherpad.openstack.org/p/heat-update-hooks
 Clint,
 
 I read through your etherpad and think there is a relationship to a use 
 case for scaling.  It would be sweet if both these cases could use the 
 same model and tooling.  At the moment in an autoscaling group, when you 
 want to scale down there is no way to quiesce the node before killing 
 the VM.  It is the same problem you have with Galera, except your 
 scenario involves an update.
 
 I'm not clear how the proposed design could be made to fit this 
 particular use case.  Do you see a way it can fill both roles so we 
 don't have two different ways to do essentially the same thing?
 

I see scaling down as an update to a nested stack which contains all of
the members of the scaling group.

So if we start focusing on scaling stacks, as Zane suggested in
the rolling updates thread, then the author of said stack would add
action_hooks to the resources that are scaled.

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Nova][Scheduler] Policy Based Scheduler and Solver Scheduler

2014-02-11 Thread Yathiraj Udupi (yudupi)

Hi Dina,

Thanks for note about Climate logic.  This is something that will be very 
useful, when we will have to schedule from Nova multiple instances (of 
potentially different flavors) as a single request.  If the Solver Scheduler, 
can make a request to the Climate service to reserve the resources soon after 
the placement decision has been made, then the nova provisioning logic can 
handle the resource provisioning using the climate reserved leases.  Regarding 
Solver Scheduler for your reference, just sent another email about this with 
some pointers about it.  Otherwise this is the blueprint - 
https://blueprints.launchpad.net/nova/+spec/solver-scheduler
I guess this is something to explore more and see how Nova provisioning logic 
to work with Climate leases. Or this is something that already works.  I need 
to find out more about Climate.

Thanks,
Yathi.


On 2/11/14, 7:44 AM, Dina Belova 
dbel...@mirantis.commailto:dbel...@mirantis.com wrote:

Like a restaurant reservation, it would claim the resources for use by 
someone at a later date.  That way nobody else can use them.
That way the scheduler would be responsible for determining where the resource 
should be allocated from, and getting a reservation for that resource.  It 
would not have anything to do with actually instantiating the 
instance/volume/etc.

Although I'm quite new to topic of Solver Scheduler, it seems to me that in 
that case you need to look on Climate project. It aims to provide resource 
reservation to OS clouds (and by resource I mean here instance/compute 
host/volume/etc.)

And Climate logic is like: create lease - get resources from common pool - do 
smth with them when lease start time will come.

I'll say one more time - I'm not really common with this discussion, but it 
looks like Climate might help here.

Thanks
Dina


On Tue, Feb 11, 2014 at 7:09 PM, Chris Friesen 
chris.frie...@windriver.commailto:chris.frie...@windriver.com wrote:
On 02/11/2014 03:21 AM, Khanh-Toan Tran wrote:
Second, there is nothing wrong with booting the instances (or
instantiating other
resources) as separate commands as long as we support some kind of
reservation token.

I'm not sure what reservation token would do, is it some kind of informing
the scheduler that the resources would not be initiated until later ?

Like a restaurant reservation, it would claim the resources for use by 
someone at a later date.  That way nobody else can use them.

That way the scheduler would be responsible for determining where the resource 
should be allocated from, and getting a reservation for that resource.  It 
would not have anything to do with actually instantiating the 
instance/volume/etc.


Let's consider a following example:

A user wants to create 2 VMs, a small one with 20 GB RAM, and a big one
with 40 GB RAM in a datacenter consisted of 2 hosts: one with 50 GB RAM
left, and another with 30 GB RAM left, using Filter Scheduler's default
RamWeigher.

If we pass the demand as two commands, there is a chance that the small VM
arrives first. RamWeigher will put it in the 50 GB RAM host, which will be
reduced to 30 GB RAM. Then, when the big VM request arrives, there will be
no space left to host it. As a result, the whole demand is failed.

Now if we can pass the two VMs in a command, SolverScheduler can put their
constraints all together into one big LP as follow (x_uv = 1 if VM u is
hosted in host v, 0 if not):

Yes.  So what I'm suggesting is that we schedule the two VMs as one call to the 
SolverScheduler.  The scheduler then gets reservations for the necessary 
resources and returns them to the caller.  This would be sort of like the 
existing Claim object in nova/compute/claims.py but generalized somewhat to 
other resources as well.

The caller could then boot each instance separately (passing the appropriate 
reservation/claim along with the boot request).  Because the caller has a 
reservation the core code would know it doesn't need to schedule or allocate 
resources, that's already been done.

The advantage of this is that the scheduling and resource allocation is done 
separately from the instantiation.  The instantiation API could remain 
basically as-is except for supporting an optional reservation token.


That responses to your first point, too. If we don't mind that some VMs
are placed and some are not (e.g. they belong to different apps), then
it's OK to pass them to the scheduler without Instance Group. However, if
the VMs are together (belong to an app), then we have to put them into an
Instance Group.

When I think of an Instance Group, I think of 
https://blueprints.launchpad.net/nova/+spec/instance-group-api-extension;.   
Fundamentally Instance Groups describes a runtime relationship between 
different instances.

The scheduler doesn't necessarily care about a runtime relationship, it's just 
trying to allocate resources efficiently.

In the above example, there is no need for those two instances to necessarily 
be part of an

[openstack-dev] [Fuel][TripleO] NIC bonding for OpenStack

2014-02-11 Thread Andrey Danin

Hi Openstackers,

We are working on link aggregation support in Fuel. We wonder what are the
most desirable types of bonding now in datacenters. We had some issues (see
below) with OVS bond in LACP mode, and it turned out that standard Linux
bonding (attached to OVS bridges) was a better option in our setup.

I want to hear your opinion, guys. What types of bonding do you think are
better now in terms of stability and performance, so that we can properly
support them for OpenStack installations.

Also, we are wondering if there any plans to support bonding in TripleO,
and how you guys would like to see it be implemented? What is the general
approach for such complex network configurations for TripleO? We would love
to extract this piece from Fuel and make it fully independent, so that the
larger community can use it and we could work collaboratively on it. Right
now it is actually already granular and can be reused in other projects,
and implemented as a separated puppet module:
https://github.com/stackforge/fuel-library/tree/master/deployment/puppet/l23network
.

Some links with our design considerations:

https://etherpad.openstack.org/p/fuel-bonding-design

https://blueprints.launchpad.net/fuel/+spec/nics-bonding-enabled-from-ui

https://blueprints.launchpad.net/fuel/+spec/nics-bonding-enabled-from-ui

UI mockups:

https://drive.google.com/file/d/0Bw6txZ1qvn9CaDdJS0ZUcW1DeDg/edit?usp=sharing

Description of the problem with LACP we ran into:
https://etherpad.openstack.org/p/LACP_issue

Thanks,


-- 
Andrey Danin
ada...@mirantis.com
skype: gcon.monolake
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Nova][Scheduler] Policy Based Scheduler and Solver Scheduler


Le 11/02/2014 17:23, Yathiraj Udupi (yudupi) a écrit :

Hi Dina,

Thanks for note about Climate logic.  This is something that will be 
very useful, when we will have to schedule from Nova multiple 
instances (of potentially different flavors) as a single request.  If 
the Solver Scheduler, can make a request to the Climate service to 
reserve the resources soon after the placement decision has been made, 
then the nova provisioning logic can handle the resource provisioning 
using the climate reserved leases.  Regarding Solver Scheduler for 
your reference, just sent another email about this with some pointers 
about it.  Otherwise this is the blueprint - 
https://blueprints.launchpad.net/nova/+spec/solver-scheduler
I guess this is something to explore more and see how Nova 
provisioning logic to work with Climate leases. Or this is something 
that already works.  I need to find out more about Climate.


Thanks,
Yathi.




There are possibly 2 ways for creating a lease : either thru the CLI or 
by the python binding.


We implemented these 2 possibilities within the current Climate 0.1 
release :
 - a Nova extension plugin is responsible for creating the lease if a 
VM should be reserved (using the Climate pythonclient binding)
 - an user can request for reserving a compute host using the Climate 
python client directly


Both logics (VM and compute host) are actually referring to 2 distinct 
plugins in the Climate manager, so the actions are completely different.


Based on your use-case, you could imagine a call from the 
SolverScheduler to Climate for creating a lease containing multiple VM 
reservations, and either you would use the Climate VM plugin or you 
would use a dedicated plugin if your need is different.


I don't think that's a huge volume of work, as Climate already defines 
and implements the main features that you need.


-Sylvain
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Mistral]

2014-02-11 Thread Dmitri Zimine

Yes it makes sense, let's draft how it may look;

and also think over implementation implications - now we separate task
parameters, action parameters, and service parameters, we may need to merge
them when instantiating the action.

DZ.

On Feb 11, 2014, at 6:19 AM, Renat Akhmerov rakhme...@mirantis.com wrote:

The only difference I see between std:send-email and something like REST_API
is that a set of parameters for the latter is dynamic (versus std:send-email
where it’s always “recipients”, “subject”, “body”). Even though it’s still
the same protocol (HTTP) but a particular request representation may be
different (i.e. query string, headers, the structure of body in case POST
etc.). But I think that doesn’t cancel the idea of being able to define the
action along with the task itself.

So good point. As for the syntax itself, we need to think it over. In the
snippet you provided “action: std:REST_API”, so we need to make sure not to
have ambiguities in the ways how we can refer actions. A convention could be
“if we don’t use a namespace we assume that there’s a separate action
definition included into the same workbook, otherwise it should be considered
in-place action definition and task property “action” refers to an action
type rather than the action itself”. Does that make sense?

Renat Akhmerov
@ Mirantis Inc.

On 11 Feb 2014, at 16:23, Dmitri Zimine d...@stackstorm.com wrote:

Do we have (or think about) a shorthand to calling REST_API action, without
defining a service?

FULL DSL:

SHORTCUT - may look something like this:

Workflow:
tasks:
timeInToronto:
action:std:REST_API
parameters:
baseUrl: http://api.timezonedb.com;
method: GET
parameters: zone=/America/Torontokey=my_api_key

Why asking:

2) For a one-off web service calls the complete syntax is may be overkill
(but yes, it comes handy for reuse). See examples below.

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Nova][Glance]supporting of v1 and v2 glance APIs in Nova

2014-02-11 Thread Eddie Sheffield

 A few days ago, I met some problems when using 'createimage' feature in
 Nova, we found that using V1 of glanceclient has some problem with
 processing of metadata, the version number and even the glance URIs are
 both hardcoded in Nova.
 
 then, we found the bluepring[1] proposed, and the maillist[2] which talked
 about the topic before, mainly focused on version autodiscovery by keystone
 catalog and config option for nova. But we still need changes in Nova
 because the incompatible behavior between v1 and v2, especially when
 creating and uploading an image file. The review request[3] of the bp is
 abandoned for now.
 
 So, what I want to confirm is, how could this situation be handled? I
 mailed Eddie Sheffield, but got no answer, so bring it up here.
 
 [1]: https://blueprints.launchpad.net/nova/+spec/use-glance-v2-api
 [2]: http://markmail.org/message/uqrpufsmh4qp5pgy
 [4]: https://review.openstack.org/#/c/38414/

Hi Lingxian,

I'm afraid I somehow didn't see the email you sent to me directly. We recently 
held the Glance Mini-summit and this work was discussed. Andrew Laski from the 
Nova team was also in attendance and provided some input from their 
perspective. Essentially we decided that while autodiscovery is desirable, we 
want to roll that functionality into a much-improved python-glanceclient which 
will present a version-agnostic programming api to users of the library. So the 
immediate plan is to go back to the approach outlined in the bp and merge prop 
you reference above. Then in the near future produce the new glanceclient 
followed by updating Nova to use the new library which will address the 
concerns of autodiscovery among other things.

Timeline-wise, I've been a bit covered up with some other work but will be 
getting back to this within a week. There were some concerns about the size of 
the patch so rather than unabandoning the existing one I will be trying to put 
up multiple, smaller patches.

Please let me know if you have any specific concerns or requirements so they 
can be addressed.

Eddie Sheffield
Rackspace Hosting, Inc.
eddie.sheffi...@rackspace.com


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Nova][Scheduler] Policy Based Scheduler and Solver Scheduler

2014-02-11 Thread Dina Belova


 This is something to explore to add in Nova, using
 a local service or external service (need to explore Climate).


I need to find out more about Climate.


Here is Climate Launchpad: https://launchpad.net/climate

That's still really young project, but I believe it'll have great future
speaking about resource reservation. So if you need some kind of
reservation logic, I also believe that should be implemented in Climate (as
it's proposed and currently implemented as Reservation-as-a-Service)

Thanks


On Tue, Feb 11, 2014 at 8:23 PM, Yathiraj Udupi (yudupi)
yud...@cisco.comwrote:

  Hi Dina,

  Thanks for note about Climate logic.  This is something that will be
 very useful, when we will have to schedule from Nova multiple instances (of
 potentially different flavors) as a single request.  If the Solver
 Scheduler, can make a request to the Climate service to reserve the
 resources soon after the placement decision has been made, then the nova
 provisioning logic can handle the resource provisioning using the climate
 reserved leases.  Regarding Solver Scheduler for your reference, just sent
 another email about this with some pointers about it.  Otherwise this is
 the blueprint -
 https://blueprints.launchpad.net/nova/+spec/solver-scheduler
 I guess this is something to explore more and see how Nova provisioning
 logic to work with Climate leases. Or this is something that already works.
  I need to find out more about Climate.

  Thanks,
 Yathi.


   On 2/11/14, 7:44 AM, Dina Belova dbel...@mirantis.com wrote:

Like a restaurant reservation, it would claim the resources for use
 by someone at a later date.  That way nobody else can use them.
 That way the scheduler would be responsible for determining where the
 resource should be allocated from, and getting a reservation for that
 resource.  It would not have anything to do with actually instantiating the
 instance/volume/etc.


  Although I'm quite new to topic of Solver Scheduler, it seems to me that
 in that case you need to look on Climate project. It aims to provide
 resource reservation to OS clouds (and by resource I mean here
 instance/compute host/volume/etc.)

  And Climate logic is like: create lease - get resources from common pool
 - do smth with them when lease start time will come.

  I'll say one more time - I'm not really common with this discussion, but
 it looks like Climate might help here.

  Thanks
 Dina


 On Tue, Feb 11, 2014 at 7:09 PM, Chris Friesen 
 chris.frie...@windriver.com wrote:

 On 02/11/2014 03:21 AM, Khanh-Toan Tran wrote:

  Second, there is nothing wrong with booting the instances (or

 instantiating other

 resources) as separate commands as long as we support some kind of
 reservation token.


 I'm not sure what reservation token would do, is it some kind of
 informing
 the scheduler that the resources would not be initiated until later ?


  Like a restaurant reservation, it would claim the resources for use by
 someone at a later date.  That way nobody else can use them.

 That way the scheduler would be responsible for determining where the
 resource should be allocated from, and getting a reservation for that
 resource.  It would not have anything to do with actually instantiating the
 instance/volume/etc.


  Let's consider a following example:

 A user wants to create 2 VMs, a small one with 20 GB RAM, and a big one
 with 40 GB RAM in a datacenter consisted of 2 hosts: one with 50 GB RAM
 left, and another with 30 GB RAM left, using Filter Scheduler's default
 RamWeigher.

 If we pass the demand as two commands, there is a chance that the small
 VM
 arrives first. RamWeigher will put it in the 50 GB RAM host, which will
 be
 reduced to 30 GB RAM. Then, when the big VM request arrives, there will
 be
 no space left to host it. As a result, the whole demand is failed.

 Now if we can pass the two VMs in a command, SolverScheduler can put
 their
 constraints all together into one big LP as follow (x_uv = 1 if VM u is
 hosted in host v, 0 if not):


  Yes.  So what I'm suggesting is that we schedule the two VMs as one call
 to the SolverScheduler.  The scheduler then gets reservations for the
 necessary resources and returns them to the caller.  This would be sort of
 like the existing Claim object in nova/compute/claims.py but generalized
 somewhat to other resources as well.

 The caller could then boot each instance separately (passing the
 appropriate reservation/claim along with the boot request).  Because the
 caller has a reservation the core code would know it doesn't need to
 schedule or allocate resources, that's already been done.

 The advantage of this is that the scheduling and resource allocation is
 done separately from the instantiation.  The instantiation API could remain
 basically as-is except for supporting an optional reservation token.


  That responses to your first point, too. If we don't mind that some VMs
 are placed and some are not (e.g. they belong to different apps),

[openstack-dev] [Neutron][IPv6] Idea: Floating IPv6 - Without any kind of NAT

2014-02-11 Thread Martinx - ジェームズ

Hello Stackers!

It is very nice to watch the OpenStack evolution in IPv6! Great job guys!!


I have another idea:

Floating IP for IPv6, or just Floating IPv6


With IPv4, as we know, OpenStack have a feature called Floating IP, which
is basically a 1-to-1 NAT rule (within tenant's Namespace q-router). In
IPv4 networks, we need this Floating IP attached to a Instance, to be
able to reach it from the Internet (*I don't like it*). But, what is the
use case for a Floating IP when you have *no NAT** (as it is with IPv6)?!

At first, when with IPv6, I was planning to disable the Floating IP
feature entirely, by removing it from Dashboard and from APIs (even for
IPv4, if FWaaS can in somehow, be able to manage q-router IPv4 NAT rules,
and not only the iptables filter table) and, I just had an idea!

For IPv6, the Floating IP can still be used to allocate more (and more)
IPs to a Instance BUT, instead of creating a NAT rule (like it is for
IPv4), it will configure the DNSMasq (or something like it) to provide more
IPv6 address per MAC / Instance. That way, we can virtually
allocate unlimited IPs (v6) for each Instance!

It will be pretty cool to see the attached Floating IPv6, literally
floating around the tenant subnet, appearing inside the Instances itself
(instead of inside the tenant's Namespace), so, we'll be able to see it
(the Floating IPv6) with ip -6 address command within the attached
Instance!

The only problem I see with this is that, for IPv4, the allocated
Floating IPs
come from the External Network (neutron / --allocation-pool) and, for IPv6,
it will come from the tenant's IPv6 subnet itself... I think... Right?!

---
Why I want tons of IPv6 within each Instance?

A.: Because we can! I mean, we can go back to the days when we had 1
website per 1 public IP (i.e. using IP-Based Virtual Hosts with Apache - I
prefer this approach).

Also, we can try to turn the Floating IPv6, in some kind of Floating
IPv6 Range, this way, we can for example, allocate millions of IPs per
Instance, like this in DHCPv6: range6 2001:db8:1:1::1000
2001:db8:1:1000:1000;...
---

NOTE: I prefer multiple IPs per Instance, instead of 1 IP per Instance,
when using VT, unless, of course, the Instances are based on Docker, so,
with it, I can easily see millions of tiny instances, each of it with its
own IPv6 address, without the overhead of virtualized environment. So, with
Docker, this Floating IPv6 Range doesn't seems to be useful...


* I know that there is NAT66 out there but, who is actually using it?! I'll
never use this thing. Personally I dislike NAT very much, mostly because it
breaks the end-to-end Internet connectivity, effectively kicking you out
from the real Internet, and it is just a workaround created to deal with
IPv4 exaustion.


BTW, please guys, let me know if this isn't the right place to post ideas
for OpenStack / feature requests... I don't want to bloat this list with
undesirable messages.


Best Regards,
Thiago Martins
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [neutron] [group-policy] Changing the meeting time

2014-02-11 Thread Stephen Wong

Hi Kyle,

Almost missed this - sounds good to me.

Thanks,
- Stephen



On Mon, Feb 10, 2014 at 7:30 PM, Kyle Mestery mest...@siliconloons.comwrote:

 Folks:

 I'd like to propose moving the Neutron Group Policy meeting going
 forward, starting with this Thursday. The meeting has been at 1600
 UTC on Thursdays, I'd like to move this to 1700UTC Thursdays
 on #openstack-meeting-alt. If this is a problem for anyone who
 regularly attends this meeting, please reply here. If I don't hear
 any replies by Wednesday, I'll officially move the meeting.

 Thanks!
 Kyle
 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] www.xrefs.info: addition of openflow and SDN

2014-02-11 Thread John Smith

I setup http://www.xrefs.info for open source developers to do cross
reference search based on OpenGrok. It covers projects like linux
kernel from version 0.01 to latest, android, linux packages, BSD,
cloud computing, big data, SDN etc.. Check it out...Let me know if you
have any suggestions or comments...Thx. xrefs.info Admin

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Heat] in-instance update hooks

2014-02-11 Thread Fox, Kevin M

Another scaling down/update use case:
Say I have a pool of ssh servers for users to use (compute cluster login nodes).
Autoscaling up is easy. Just launch a new node and add it to the load balancer.

Scaling down/updating is harder. It should ideally:
 * Set the admin state on the load balancer for the node, ensuring no new 
connections go to the node.
 * Contact the node or the balancer and ensure all outstanding connections are 
complete. Wait for it. This could be a long long time.
 * Destroy or update the node

Thanks,
Kevin

From: Clint Byrum [cl...@fewbar.com]
Sent: Tuesday, February 11, 2014 8:13 AM
To: openstack-dev
Subject: Re: [openstack-dev] [Heat] in-instance update hooks

Excerpts from Steven Dake's message of 2014-02-11 07:19:19 -0800:
 On 02/10/2014 10:22 PM, Clint Byrum wrote:
  Hi, so in the previous thread about rolling updates it became clear that
  having in-instance control over updates is a more fundamental idea than
  I had previously believed. During an update, Heat does things to servers
  that may interrupt the server's purpose, and that may cause it to fail
  subsequent things in the graph.
 
  Specifically, in TripleO we have compute nodes that we are managing.
  Before rebooting a machine, we want to have a chance to live-migrate
  workloads if possible, or evacuate in the simpler case, before the node
  is rebooted. Also in the case of a Galera DB where we may even be running
  degraded, we want to ensure that we have quorum before proceeding.
 
  I've filed a blueprint for this functionality:
 
  https://blueprints.launchpad.net/heat/+spec/update-hooks
 
  I've cobbled together a spec here, and I would very much welcome
  edits/comments/etc:
 
  https://etherpad.openstack.org/p/heat-update-hooks
 Clint,

 I read through your etherpad and think there is a relationship to a use
 case for scaling.  It would be sweet if both these cases could use the
 same model and tooling.  At the moment in an autoscaling group, when you
 want to scale down there is no way to quiesce the node before killing
 the VM.  It is the same problem you have with Galera, except your
 scenario involves an update.

 I'm not clear how the proposed design could be made to fit this
 particular use case.  Do you see a way it can fill both roles so we
 don't have two different ways to do essentially the same thing?


I see scaling down as an update to a nested stack which contains all of
the members of the scaling group.

So if we start focusing on scaling stacks, as Zane suggested in
the rolling updates thread, then the author of said stack would add
action_hooks to the resources that are scaled.

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Neutron][IPv6] Idea: Floating IPv6 - Without any kind of NAT

2014-02-11 Thread Veiga, Anthony


Hello Stackers!

It is very nice to watch the OpenStack evolution in IPv6! Great job guys!!

Thanks!



I have another idea:

Floating IP for IPv6, or just Floating IPv6


With IPv4, as we know, OpenStack have a feature called Floating IP, which is 
basically a 1-to-1 NAT rule (within tenant's Namespace q-router). In IPv4 
networks, we need this Floating IP attached to a Instance, to be able to 
reach it from the Internet (I don't like it). But, what is the use case for a 
Floating IP when you have no NAT* (as it is with IPv6)?!

There are definitely cases for it, and we were planning to address it in a 
future release.


At first, when with IPv6, I was planning to disable the Floating IP feature 
entirely, by removing it from Dashboard and from APIs (even for IPv4, if FWaaS 
can in somehow, be able to manage q-router IPv4 NAT rules, and not only the 
iptables filter table) and, I just had an idea!

For IPv6, the Floating IP can still be used to allocate more (and more) IPs 
to a Instance BUT, instead of creating a NAT rule (like it is for IPv4), it 
will configure the DNSMasq (or something like it) to provide more IPv6 address 
per MAC / Instance. That way, we can virtually allocate unlimited IPs (v6) for 
each Instance!

It will be pretty cool to see the attached Floating IPv6, literally floating 
around the tenant subnet, appearing inside the Instances itself (instead of 
inside the tenant's Namespace), so, we'll be able to see it (the Floating IPv6) 
with ip -6 address command within the attached Instance!

The only problem I see with this is that, for IPv4, the allocated Floating 
IPs come from the External Network (neutron / --allocation-pool) and, for 
IPv6, it will come from the tenant's IPv6 subnet itself... I think... Right?!

I think the real issue here is how neutron handles these from a network/port 
perspective.  In the IPv4 case, the IPs are from and entirely separate block.  
I think that should probably stay the same for IPv6, since you have twofold 
problems:


  1.  You've already issued an RA on the network for a specific address scope.  
Another one, when an interface is already addressed, won't trigger adding a new 
address.
  2.  Routing.  Which address should be the source address? If it's on the same 
network, the majority of distributions will ignore further routes and addresses 
for the purpose of sourcing packets.

Because of the above, it would make sense for floats to end up on a floating 
IP subnet.  However, we'd go back to earlier neutron issues about having 
multiple subnets on a network.  So then, we end up with a new network, a new 
subnet and a new port.  My personal vote is to go this route.


---
Why I want tons of IPv6 within each Instance?

A.: Because we can! I mean, we can go back to the days when we had 1 website 
per 1 public IP (i.e. using IP-Based Virtual Hosts with Apache - I prefer this 
approach).

Also, we can try to turn the Floating IPv6, in some kind of Floating IPv6 
Range, this way, we can for example, allocate millions of IPs per Instance, 
like this in DHCPv6: range6 2001:db8:1:1::1000 2001:db8:1:1000:1000;...

I'd also argue for security/policy/reserved addressing reasons.  Also, let's 
not get into the habit of assigning loads of addresses.  Each network needs to 
be a /64, and going all out means you're back to the exhaustion issue we have 
in IPv4 ;)

---

NOTE: I prefer multiple IPs per Instance, instead of 1 IP per Instance, when 
using VT, unless, of course, the Instances are based on Docker, so, with it, I 
can easily see millions of tiny instances, each of it with its own IPv6 
address, without the overhead of virtualized environment. So, with Docker, this 
Floating IPv6 Range doesn't seems to be useful...


* I know that there is NAT66 out there but, who is actually using it?! I'll 
never use this thing. Personally I dislike NAT very much, mostly because it 
breaks the end-to-end Internet connectivity, effectively kicking you out from 
the real Internet, and it is just a workaround created to deal with IPv4 
exhaustion.

As a long-time IPv6 engineer, I will advocate against NAT wherever I possibly 
can.  +1 to this!



BTW, please guys, let me know if this isn't the right place to post ideas for 
OpenStack / feature requests... I don't want to bloat this list with 
undesirable messages.

It absolutely is!  Also, you might want to join the discussions for the IPv6 
sub-team: https://wiki.openstack.org/wiki/Meetings/Neutron-IPv6-Subteam



Best Regards,
Thiago Martins

-Anthony Veiga
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Fuel][TripleO] NIC bonding for OpenStack

Excerpts from Andrey Danin's message of 2014-02-11 08:42:46 -0800:
 Hi Openstackers,
 
 We are working on link aggregation support in Fuel. We wonder what are the
 most desirable types of bonding now in datacenters. We had some issues (see
 below) with OVS bond in LACP mode, and it turned out that standard Linux
 bonding (attached to OVS bridges) was a better option in our setup.
 
 I want to hear your opinion, guys. What types of bonding do you think are
 better now in terms of stability and performance, so that we can properly
 support them for OpenStack installations.
 
 Also, we are wondering if there any plans to support bonding in TripleO,
 and how you guys would like to see it be implemented? What is the general
 approach for such complex network configurations for TripleO? We would love
 to extract this piece from Fuel and make it fully independent, so that the
 larger community can use it and we could work collaboratively on it. Right
 now it is actually already granular and can be reused in other projects,
 and implemented as a separated puppet module:
 https://github.com/stackforge/fuel-library/tree/master/deployment/puppet/l23network

For Nova baremetal and Ironic, they're going to feed MACs to Neutron which
it would then serve DHCP for. I am not sure how that relates to bonding,
and if the MAC would change for DHCP requests after bonding is configured,
but that would have to be addressed and tested.

Otherwise, it would work largely the same way I think. Just configure the
image to read config parameters from the Heat metadata and subsequently
configure bonding. It is even possible that the facter cfn plugin could
be made to work with a little bit of modification, perhaps allowing you
to forklift the puppet modules into a diskimage-builder element:

https://github.com/oppian/oppian-puppet-modules/blob/master/modules/cfn/lib/facter/cfn.rb

I've heard of no plans for supporting this configuration of servers,
but I see no reason it couldn't be added as an option if it was important
to deployers.

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [infra] Proposing Sergey Lukjanov for infra-core

2014-02-11 Thread Sergey Lukjanov

Thank you all!

I'm glad to help infra team with extremely growing number of CRs to their
projects.


On Tue, Feb 11, 2014 at 2:24 PM, Thierry Carrez thie...@openstack.orgwrote:

 James E. Blair wrote:
  I'm very pleased to propose that we add Sergey Lukjanov to the
  infra-core team.
 
  He is among the top reviewers of projects in openstack-infra, and is
  very familiar with how jenkins-job-builder and zuul are used and
  configured.  He has done quite a bit of work in helping new projects
  through the process and ensuring that changes to the CI system are
  correct.  In addition to providing very helpful reviews he has also
  contributed significant patches to our Python projects illustrating a
  high degree of familiarity with the code base and project direction.
  And as a bonus, we're all looking forward to once again having an
  infra-core member in a non-US time zone!

 +1!

 My only fear is that Sergey is going in too many directions at the same
 time (remember he is also the Savanna^WSahara/Caravan/Batyr/Slonik PTL)
 and might burn out. But so far he has been gracefully handling the
 increasing load... so let's see how that goes :)

 --
 Thierry Carrez (ttx)

 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev




-- 
Sincerely yours,
Sergey Lukjanov
Savanna Technical Lead
Mirantis Inc.
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Nova][Scheduler] Policy Based Scheduler and Solver Scheduler

2014-02-11 Thread Yathiraj Udupi (yudupi)

hi Sylvain and Dina,

Thanks for your pointers about Climate.  I will take a closer look at it and 
try it out.  So after a reservation lease for a VM is made by Climate, who acts 
on it to finally instantiate the VM ? Is it Climate or Nova should act on the 
lease to finally provision the VM.

Thanks,
Yathi.

On 2/11/14, 8:42 AM, Sylvain Bauza 
sylvain.ba...@bull.netmailto:sylvain.ba...@bull.net wrote:

Le 11/02/2014 17:23, Yathiraj Udupi (yudupi) a écrit :
Hi Dina,

Thanks for note about Climate logic.  This is something that will be very 
useful, when we will have to schedule from Nova multiple instances (of 
potentially different flavors) as a single request.  If the Solver Scheduler, 
can make a request to the Climate service to reserve the resources soon after 
the placement decision has been made, then the nova provisioning logic can 
handle the resource provisioning using the climate reserved leases.  Regarding 
Solver Scheduler for your reference, just sent another email about this with 
some pointers about it.  Otherwise this is the blueprint - 
https://blueprints.launchpad.net/nova/+spec/solver-scheduler
I guess this is something to explore more and see how Nova provisioning logic 
to work with Climate leases. Or this is something that already works.  I need 
to find out more about Climate.

Thanks,
Yathi.



There are possibly 2 ways for creating a lease : either thru the CLI or by the 
python binding.

We implemented these 2 possibilities within the current Climate 0.1 release :
 - a Nova extension plugin is responsible for creating the lease if a VM should 
be reserved (using the Climate pythonclient binding)
 - an user can request for reserving a compute host using the Climate python 
client directly

Both logics (VM and compute host) are actually referring to 2 distinct plugins 
in the Climate manager, so the actions are completely different.

Based on your use-case, you could imagine a call from the SolverScheduler to 
Climate for creating a lease containing multiple VM reservations, and either 
you would use the Climate VM plugin or you would use a dedicated plugin if your 
need is different.

I don't think that's a huge volume of work, as Climate already defines and 
implements the main features that you need.

-Sylvain
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [neutron] [group-policy] Changing the meeting time

2014-02-11 Thread Sumit Naiksatam

Hi Kyle,

The new time sounds good to me as well, thanks for initiating this.

~sumit.

On Tue, Feb 11, 2014 at 9:02 AM, Stephen Wong s3w...@midokura.com wrote:
 Hi Kyle,

 Almost missed this - sounds good to me.

 Thanks,
 - Stephen



 On Mon, Feb 10, 2014 at 7:30 PM, Kyle Mestery mest...@siliconloons.com
 wrote:

 Folks:

 I'd like to propose moving the Neutron Group Policy meeting going
 forward, starting with this Thursday. The meeting has been at 1600
 UTC on Thursdays, I'd like to move this to 1700UTC Thursdays
 on #openstack-meeting-alt. If this is a problem for anyone who
 regularly attends this meeting, please reply here. If I don't hear
 any replies by Wednesday, I'll officially move the meeting.

 Thanks!
 Kyle
 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Neutron][LBaaS] Proposal for model

2014-02-11 Thread WICKES, ROGER

[Roger] Hi Stephen! Great job! Obviously your experience is both awesome and 
essential here.

I would ask that we add a historical archive (physically implemented as a log 
file, probably) 
object to your model. When you mentioned sending data off to Ceilometer, that 
triggered 
me to think about one problem I have had to deal with is what packet went 
where?  
in diagnosing errors usually related to having a bug on 1 out of 5 
load-balanced servers, 
usually because of a deployed version mismatch, but could also be due to virus. 
When our
customer sees hey every now and then this image is broken on a web page that 
points
us to an inconsistent farm, and having the ability to trace or see which server 
got that customer's
packet (routed to by the LB) would really help in pinpointing the errant 
server.  

 Benefits of a new model

 If we were to adopt either of these data models, this would enable us to
 eventually support the following feature sets, in the following ways (for
 example):

 Automated scaling of load-balancer services

[Roger] Would the Heat module be called on to add more LB's to the farm?

 I talked about horizontal scaling of load balancers above under High
 Availability, but, at least in the case of a software appliance, vertical
 scaling should also be possible in an active-standby cluster_model by
**

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Heat] in-instance update hooks

Thanks Kevin, great summary.

This is beyond the scope of in-instance notification. I think this is
more like the generic notification API that Thomas Herve suggested in
the rolling updates thread. It can definitely use the same method for
its implementation, and I think it is adjacent to, and not in front of
or behind, the in-instance case.

Excerpts from Fox, Kevin M's message of 2014-02-11 09:22:28 -0800:
 Another scaling down/update use case:
 Say I have a pool of ssh servers for users to use (compute cluster login 
 nodes).
 Autoscaling up is easy. Just launch a new node and add it to the load 
 balancer.
 
 Scaling down/updating is harder. It should ideally:
  * Set the admin state on the load balancer for the node, ensuring no new 
 connections go to the node.
  * Contact the node or the balancer and ensure all outstanding connections 
 are complete. Wait for it. This could be a long long time.
  * Destroy or update the node
 
 Thanks,
 Kevin
 
 From: Clint Byrum [cl...@fewbar.com]
 Sent: Tuesday, February 11, 2014 8:13 AM
 To: openstack-dev
 Subject: Re: [openstack-dev] [Heat] in-instance update hooks
 
 Excerpts from Steven Dake's message of 2014-02-11 07:19:19 -0800:
  On 02/10/2014 10:22 PM, Clint Byrum wrote:
   Hi, so in the previous thread about rolling updates it became clear that
   having in-instance control over updates is a more fundamental idea than
   I had previously believed. During an update, Heat does things to servers
   that may interrupt the server's purpose, and that may cause it to fail
   subsequent things in the graph.
  
   Specifically, in TripleO we have compute nodes that we are managing.
   Before rebooting a machine, we want to have a chance to live-migrate
   workloads if possible, or evacuate in the simpler case, before the node
   is rebooted. Also in the case of a Galera DB where we may even be running
   degraded, we want to ensure that we have quorum before proceeding.
  
   I've filed a blueprint for this functionality:
  
   https://blueprints.launchpad.net/heat/+spec/update-hooks
  
   I've cobbled together a spec here, and I would very much welcome
   edits/comments/etc:
  
   https://etherpad.openstack.org/p/heat-update-hooks
  Clint,
 
  I read through your etherpad and think there is a relationship to a use
  case for scaling.  It would be sweet if both these cases could use the
  same model and tooling.  At the moment in an autoscaling group, when you
  want to scale down there is no way to quiesce the node before killing
  the VM.  It is the same problem you have with Galera, except your
  scenario involves an update.
 
  I'm not clear how the proposed design could be made to fit this
  particular use case.  Do you see a way it can fill both roles so we
  don't have two different ways to do essentially the same thing?
 
 
 I see scaling down as an update to a nested stack which contains all of
 the members of the scaling group.
 
 So if we start focusing on scaling stacks, as Zane suggested in
 the rolling updates thread, then the author of said stack would add
 action_hooks to the resources that are scaled.
 

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Nova][Scheduler] Policy Based Scheduler and Solver Scheduler

2014-02-11 18:28 GMT+01:00 Yathiraj Udupi (yudupi) yud...@cisco.com:



  Thanks for your pointers about Climate.  I will take a closer look at it
 and try it out.  So after a reservation lease for a VM is made by Climate,
 who acts on it to finally instantiate the VM ? Is it Climate or Nova should
 act on the lease to finally provision the VM.



That depends on the plugin mapped with the reservation type. The current
plugin mapped for virtual instances does unshelve the instance at lease
start (because Nova extension shelved it at lease creation).

If you don't like this behaviour, that's quite easy to create another
plugin for another reservation type. When the lease starts, it does run the
lease_create() action defined by the plugin. So, if you don't like
shelving/unshelving, that's not a problem, feel free to add a blueprint
about your needs or create your own plugin.

-Sylvain
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [oslo-notify] notifications consumed by multiple subscribers

2014-02-11 Thread Sanchez, Cristian A

Hi,
I’m planning to use oslo.notify mechanisms to implement a climate blueprint: 
https://blueprints.launchpad.net/climate/+spec/notifications. Ideally, the 
notifications sent by climate should be received by multiple services 
subscribed to the same topic. Is that possible with oslo.notify? And moreover, 
is there any mechanism for removing items from the queue? Or should one 
subscriber be responsible for removing items from it?

Thanks

Cristian

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Murano] Can we migrate to oslo.messaging?

On Tue, Feb 11, 2014 at 9:59 AM, Serg Melikyan smelik...@mirantis.comwrote:

Doug, thank you for your response!

With moving to *oslo.messaging *I think we need to carefully
re-evaluate our communication patterns and try to either fit them to
RPC/Notification, or work on extending *oslo.messaging *with some new
ideas. Adopting a new library is always a challenge.

That makes sense.

OK, great!

As long as we can find a way to make it backwards-compatible, for the
projects that don't use the feature, I don't see a problem in principle.
Experimenting within the Murano repository maybe the easiest way to work
out what those APIs need to be. I'm glad to hear you would contribute that
work back upstream, though. :-)

Doug

On Tue, Feb 11, 2014 at 6:06 PM, Doug Hellmann
doug.hellm...@dreamhost.com wrote:

On Tue, Feb 11, 2014 at 8:05 AM, Serg Melikyan smelik...@mirantis.comwrote:

oslo.messaging http://github.com/openstack/oslo.messaging is a
library that provides RPC and Notifications API, they are part of the same
library for mostly historical reasons. One of the major goals of
*oslo.messaging* is to provide clean RPC and Notification API without
any trace of messaging queue concepts (but two of most advanced drivers
used by oslo.messaging is actually based on AMQP: RabbitMQ and QPID).

We were designing Murano on messaging queue concepts using some
AMQP/RabbitMQ specific features, like queue TTL. Since we never considered
communications between our components in terms of RPC or Notifications and
always thought about them as message exchange through broker it has
influenced our components architecture. In Murano we use simple
wrapperhttps://github.com/stackforge/murano-common/tree/master/muranocommon/messaging
around Puka https://github.com/majek/puka (RabbitMQ client with most
simple and thoughtful async model) that is used in all our components. We
forked
Puka https://github.com/istalker2/puka since we had specific
requirements to SSL and could not yet merge our
workhttps://github.com/majek/puka/pull/43 back
to master.

Can we abandon our own
wrapperhttps://github.com/stackforge/murano-common/tree/master/muranocommon/messagingaround
our own
fork of Puka https://github.com/istalker2/puka in favor of
*oslo.messaging*? *Yes*, but this migration may be tricky. I believe we
can migrate to *oslo.messaging* in a week or so*.*

I had played with *oslo.messaging* emulating our current communication
patterns with *oslo.messaging*, and I am certain that current
implementation can be migrated to *oslo.messaging**. *But I am not sure
that *oslo.messaging* may be easily suited to all future use-cases that
we plan to cover in a few next releases without major contributions.
Please, try to respond with any questions related to *oslo.messaging*
implementation
and how it can be fitted with certain use-case.

Below, I tried to describe our current use-cases and what specific MQ
features we are using, how they may be implemented with
*oslo.messaging *and with what limitations we will face.

Use-Case
Murano has several components with communications between them based on
messaging queue:
*murano-api* - *murano-conductor:*

1. *murano-api* sends deployment tasks to murano-conductor

*murano-conductor* - *murano-api:*

1. *murano-conductor* reports to *murano-api* task progress
during processing
2. after processing, *murano-conductor* sends results to *murano-api*

*murano-conductor *-* murano-agent:*

1. during task processing *murano-conductor* sends execution plans
with commands to *murano-agent.*

Note: each of mentioned components above may have more than one instance.

One of great messaging queue specific that we heavily use is a idea of
queue itself, messages sent to component will be handled any time soon as
at least one instance

Re: [openstack-dev] [oslo-notify] notifications consumed by multiple subscribers

On Tue, Feb 11, 2014 at 1:28 PM, Sanchez, Cristian A 
cristian.a.sanc...@intel.com wrote:

 Hi,
 I'm planning to use oslo.notify mechanisms to implement a climate
 blueprint: https://blueprints.launchpad.net/climate/+spec/notifications.
 Ideally, the notifications sent by climate should be received by multiple
 services subscribed to the same topic. Is that possible with oslo.notify?
 And moreover, is there any mechanism for removing items from the queue? Or
 should one subscriber be responsible for removing items from it?


Each service should subscribe to the same exchange and topic using a
different queue name to receive all of the messages.

The code to set up the listeners is still under review for oslo.messaging.
See the patch series starting with https://review.openstack.org/#/c/57275/16

Doug




 Thanks

 Cristian

 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [infra] Meeting Tuesday February 11th at 19:00 UTC

2014-02-11 Thread Elizabeth Krumbach Joseph

On Mon, Feb 10, 2014 at 7:48 AM, Elizabeth Krumbach Joseph
l...@princessleia.com wrote:
 The OpenStack Infrastructure (Infra) team is hosting our weekly
 meeting tomorrow, Tuesday February 11th, at 19:00 UTC in
 #openstack-meeting

Thanks to everyone who attended, minutes and logs now available here:

Minutes: 
http://eavesdrop.openstack.org/meetings/infra/2014/infra.2014-02-11-19.01.html
Minutes (text):
http://eavesdrop.openstack.org/meetings/infra/2014/infra.2014-02-11-19.01.txt
Log: 
http://eavesdrop.openstack.org/meetings/infra/2014/infra.2014-02-11-19.01.log.html

-- 
Elizabeth Krumbach Joseph || Lyz || pleia2
http://www.princessleia.com

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Fuel][TripleO] NIC bonding for OpenStack

2014-02-11 Thread Dan Prince

- Original Message -
 From: Andrey Danin ada...@mirantis.com
 To: openstack-dev@lists.openstack.org, fuel-dev 
 fuel-...@lists.launchpad.net
 Sent: Tuesday, February 11, 2014 11:42:46 AM
 Subject: [openstack-dev] [Fuel][TripleO] NIC bonding for OpenStack

 Hi Openstackers,

 We are working on link aggregation support in Fuel. We wonder what are the
 most desirable types of bonding now in datacenters. We had some issues (see
 below) with OVS bond in LACP mode, and it turned out that standard Linux
 bonding (attached to OVS bridges) was a better option in our setup.

 I want to hear your opinion, guys. What types of bonding do you think are
 better now in terms of stability and performance, so that we can properly
 support them for OpenStack installations.

 Also, we are wondering if there any plans to support bonding in TripleO,
 and how you guys would like to see it be implemented? What is the general
 approach for such complex network configurations for TripleO?

The OVS bonded approach could work quite well with TripleO since we already use 
an OVS bridge to provide network access. Right now we do most of our OVS 
network configuration via the ensure-bridge script/tool. It takes care of 
creating the OVS bridge and moving our physical NIC onto the bridge so that 
instances have network connectivity in our flat network. I recently re-factored 
this tool so that it uses persistent network configuration files:

 https://review.openstack.org/#/c/69918/

Once we get that in we could simply add in the extra OVSBonding config as 
outlined here and I think we'd have it:

 https://github.com/osrg/openvswitch/blob/master/rhel/README.RHEL#L108

As for standard linux bonding we could do that too... perhaps via another tool 
that runs before ensure-bridge does its thing as we'd still want the ability to 
put the bonded interface on an OVS bridge.

Dan

 We would love to extract this piece from Fuel and make it fully independent, 
 so that the
 larger community can use it and we could work collaboratively on it. Right
 now it is actually already granular and can be reused in other projects,
 and implemented as a separated puppet module:
 https://github.com/stackforge/fuel-library/tree/master/deployment/puppet/l23network
 .

 Some links with our design considerations:

 https://etherpad.openstack.org/p/fuel-bonding-design

 https://blueprints.launchpad.net/fuel/+spec/nics-bonding-enabled-from-ui

 https://blueprints.launchpad.net/fuel/+spec/nics-bonding-enabled-from-ui

 UI mockups:

 https://drive.google.com/file/d/0Bw6txZ1qvn9CaDdJS0ZUcW1DeDg/edit?usp=sharing

 Description of the problem with LACP we ran into:
 https://etherpad.openstack.org/p/LACP_issue

 Thanks,

 --
 Andrey Danin
 ada...@mirantis.com
 skype: gcon.monolake

 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [neutron] [group-policy] Changing the meeting time

2014-02-11 Thread Kyle Mestery

FYI, I’ve made the change on the meeting pages as well [1]. The Neutron
Group Policy meeting is now at 1700UTC Thursday’s on #openstack-meeting-alt.

Thanks!
Kyle

[1] 
https://wiki.openstack.org/wiki/Meetings#Neutron_Group_Policy_Sub-Team_Meeting

On Feb 11, 2014, at 11:30 AM, Sumit Naiksatam sumitnaiksa...@gmail.com wrote:

 Hi Kyle,
 
 The new time sounds good to me as well, thanks for initiating this.
 
 ~sumit.
 
 On Tue, Feb 11, 2014 at 9:02 AM, Stephen Wong s3w...@midokura.com wrote:
 Hi Kyle,
 
Almost missed this - sounds good to me.
 
 Thanks,
 - Stephen
 
 
 
 On Mon, Feb 10, 2014 at 7:30 PM, Kyle Mestery mest...@siliconloons.com
 wrote:
 
 Folks:
 
 I'd like to propose moving the Neutron Group Policy meeting going
 forward, starting with this Thursday. The meeting has been at 1600
 UTC on Thursdays, I'd like to move this to 1700UTC Thursdays
 on #openstack-meeting-alt. If this is a problem for anyone who
 regularly attends this meeting, please reply here. If I don't hear
 any replies by Wednesday, I'll officially move the meeting.
 
 Thanks!
 Kyle
 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
 
 
 
 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
 
 
 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Neutron] Support for multiple provider networks with same VLAN segmentation id

2014-02-11 Thread Aaron Rosen

I believe it would need to be like:

 network_vlan_ranges = physnet1:100:300, phynet2:100:300, phynet3:100:300

Additional comments inline:

On Mon, Feb 10, 2014 at 8:49 PM, Vinay Bannai vban...@gmail.com wrote:

 Bob and Kyle,

 Thanks for your review.
 We looked at this option and it seems it might meet our needs. Here is
 what we intend to do:

 Let's say we have three racks (each rack supports three VLANs - 100, 200
 and 300).
 We create the following config file for the neutron server




 tenant_network_type = vlan
  network_vlan_ranges = physnet1:100:300
  network_vlan_ranges = phynet2:100:300
  network_vlan_ranges = phynet3:100:300
  integration_bridge = br-int
  bridge_mappings = physnet1:br-eth1, physnet2:br-eth1, physnet3:br-eth1
 Is this what you meant?

 Vinay


 On Sun, Feb 9, 2014 at 6:03 PM, Robert Kukura rkuk...@redhat.com wrote:

 On 02/09/2014 12:56 PM, Kyle Mestery wrote:
  On Feb 6, 2014, at 5:24 PM, Vinay Bannai vban...@gmail.com wrote:
 
  Hello Folks,
 
  We are running into a situation where we are not able to create
 multiple provider networks with the same VLAN id. We would like to propose
 a solution to remove this restriction through a configuration option. This
 approach would not conflict with the present behavior where it is not
 possible to create multiple provider networks with the same VLAN id.
 
  The changes should be minimal and would like to propose it for the
 next summit. The use case for this need is documented in the blueprint
 specification.
  Any feedback or comments are welcome.
 
 
 https://blueprints.launchpad.net/neutron/+spec/duplicate-providernet-vlans
 
  Hi Vinay:
 
  This problem seems straightforward enough, though currently you are
 right
  in that we don't allow multiple Neutron networks to have the same
 segmentation
  ID. I've added myself as approver for this BP and look forward to
 further
  discussions of this before and during the upcoming Summit!


I kind of feel like allowing a vlan to span multiple networks is kind of
wonky. I feel like a better abstraction would be if we had better access
control over shared networks between tenants. This way we could explicitly
allow two tenants to share a network. Is this the problem you are trying to
solve though doing it with the same vlan?  How do you plan on enforcing
that mac_addresses are unique on the same physical network?

Multiple networks with network_type of 'vlan' are already allowed to
 have the same segmentation ID with the ml2, openvswitch, or linuxbridge
 plugins - the networks just need to have different physical_network
 names.


This is the same for the NSX plugin as well.


 If they have the same network_type, physical_network, and
 segmentation_id, they are the same network. What else would distinguish
 them from each other?

 Could your use case be addressed by simply using different
 physical_network names for each rack? This would provide independent
 spaces of segmentation_ids for each.

 -Bob

 
  Thanks!
  Kyle
 
  Thanks
  --
  Vinay Bannai
  Email: vban...@gmail.com
  Google Voice: 415 938 7576
  ___
  OpenStack-dev mailing list
  OpenStack-dev@lists.openstack.org
  http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
 
 
  ___
  OpenStack-dev mailing list
  OpenStack-dev@lists.openstack.org
  http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
 


 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev




 --
 Vinay Bannai
 Email: vban...@gmail.com
 Google Voice: 415 938 7576

 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [python-openstacksdk] First meeting scheduled

2014-02-11 Thread Jesse Noller

As I said last week; we’re ready to kickoff and have regular meetings for the 
“unified python SDK” project. The initial meeting is scheduled on the wiki:

https://wiki.openstack.org/wiki/Meetings#python-openstacksdk_Meeting

Date/Time: Feb. 19th - 19:00 UTC / 1pm CST 

IRC channel: #openstack-meeting-3

Meeting Agenda: 
https://wiki.openstack.org/wiki/Meetings/PythonOpenStackSDK

About the project: 
https://wiki.openstack.org/wiki/PythonOpenStackSDK

If you have questions, all of us lurk in #openstack-sdks on freenode!

See you then.

Jesse
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [keystone] role of Domain in VPC definition

2014-02-11 Thread Martin, JC

Ravi,

It seems that the following Blueprint 
https://wiki.openstack.org/wiki/Blueprint-aws-vpc-support

has been approved. 

However, I cannot find a discussion with regard to the merit of using project 
vs. domain, or other mechanism for the implementation.

I have an issue with this approach as it prevents tenants within the same 
domain sharing the same VPC to have projects.

As an example, if you are a large organization on AWS, it is likely that you 
have a large VPC that will be shred by multiple projects. With this proposal, 
we loose that capability, unless I missed something.

JC

On Dec 19, 2013, at 6:10 PM, Ravi Chunduru ravi...@gmail.com wrote:

 Hi,
   We had some internal discussions on role of Domain and VPCs. I would like 
 to expand and understand community thinking of Keystone domain and VPCs.
 
 Is VPC equivalent to Keystone Domain?
 
 If so, as a public cloud provider - I create a Keystone domain and give it to 
 an organization which wants a virtual private cloud.
 
 Now the question is if that organization wants to have  departments wise 
 allocation of resources it is becoming difficult to visualize with existing 
 v3 keystone constructs.
 
 Currently, it looks like each department of an organization cannot have their 
 own resource management with in the organization VPC ( LDAP based user 
 management, network management or dedicating computes etc.,) For us, 
 Openstack Project does not match the requirements of a department of an 
 organization.
 
 I hope you guessed what we wanted - Domain must have VPCs and VPC to have 
 projects.
 
 I would like to know how community see the VPC model in Openstack.
 
 Thanks,
 -Ravi.
 
 
 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Trove] Backup/Restore encryption/decryption issue

2014-02-11 Thread Michael Basnight

Denis Makogon dmako...@mirantis.com writes:

Goodday, OpenStack DВaaS community.

I'd like to start conversation about guestagent security issue related
to backup/restore process. Trove guestagent service uses AES with 256 bit
key (in CBC mode) [1] to encrypt backups which are stored at predefined
Swift container.

As you can see, password is defined in config file [2]. And here comes
problem, this password is used for all tenants/projects that use Trove - it
is a security issue. I would like to suggest Key derivation function [3]
based on static attributes specific for each tenant/project (tenant_id).
KDF would be based upon python implementation of PBKDF2 [4]. Implementation
can be seen here [5].

I do not want to see us writing our own crypto code in Trove. Id much
rather us use barbican for this, assuming it fits the bill. Lets do some
research on barbican before we go write this all.

Also i'm looking forward to give user an ability to pass password for
KDF that would deliver key for backup/restore encryption/decryption, if
ingress password (from user) will be empty, guest will use static
attributes of tenant (tenant_id).

To allow backward compatibility, python-troveclient should be able to pass
old password [1] to guestagent as one of parameters on restore call.

Blueprint already have been registered in Trove launchpad space, [6].

I also foresee porting this feature to oslo-crypt, as part of security
framework (oslo.crypto) extensions.

Again, id rather see us use barbican for this instead of creating oslo-crypt.

Thoughts ?

[1]
https://github.com/openstack/trove/blob/master/trove/guestagent/strategies/backup/base.py#L113-L116
[2]
https://github.com/openstack/trove/blob/master/etc/trove/trove-guestagent.conf.sample#L69
[3] http://en.wikipedia.org/wiki/Key_derivation_function
[4] http://en.wikipedia.org/wiki/PBKDF2
[5] https://gist.github.com/denismakogon/8823279
[6] https://blueprints.launchpad.net/trove/+spec/backup-encryption

Best regards,
Denis Makogon
Mirantis, Inc.
Kharkov, Ukraine
www.mirantis.com
www.mirantis.ru
dmako...@mirantis.com
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

pgp5tp7rQlTnh.pgp
Description: PGP signature
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [python-openstacksdk] First meeting scheduled

2014-02-11 Thread Boris Pavlovic

Jesse,

Why not next approach:
1) Switch all clients to apiclient[1] (internal unification)
2) Unify API of all clients (external unification)
3) Unify work with keystone
4) Graduate apiclient
5) Switch all clients to apiclient lib
6) Make one simple plugable mechanism. e.g. subclass based factory +
steavdore
7) Add one by one subclasse that present client in this factory
8) In one pretty day stop gates  switch to unified client.

This is actually step by step solution that works for community and could
be done independently by tons of developers.


[1]
https://github.com/openstack/oslo-incubator/tree/master/openstack/common/apiclient


Best regards,
Boris Pavlovic


On Wed, Feb 12, 2014 at 12:40 AM, Jesse Noller
jesse.nol...@rackspace.comwrote:

 As I said last week; we're ready to kickoff and have regular meetings for
 the unified python SDK project. The initial meeting is scheduled on the
 wiki:

 https://wiki.openstack.org/wiki/Meetings#python-openstacksdk_Meeting

 Date/Time: Feb. 19th - 19:00 UTC / 1pm CST

 IRC channel: #openstack-meeting-3

 Meeting Agenda:
 https://wiki.openstack.org/wiki/Meetings/PythonOpenStackSDK

 About the project:
 https://wiki.openstack.org/wiki/PythonOpenStackSDK

 If you have questions, all of us lurk in #openstack-sdks on freenode!

 See you then.

 Jesse
 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Trove] Backup/Restore encryption/decryption issue

2014-02-11 Thread Denis Makogon

As we decided at meeting, we wouldn't keep our own implementations of
security stuff, we'll use Barbican as single entry point of delivering
secrets.
I hadn't talked with Barbican team, but since oslo-incubator will (someday)
release oslo.crypto lib for all projects, i think that adding
implementation of new RFC to crypto is a good idea, it would be easy to
re-use it in barbican later and then i will use barbican functionality in
trove for security improvement.

Best regards,

Denis Makogon

Mirantis, Inc.

Kharkov, Ukraine

www.mirantis.com

www.mirantis.ru

dmako...@mirantis.com

2014-02-11 22:58 GMT+02:00 Michael Basnight mbasni...@gmail.com:

Denis Makogon dmako...@mirantis.com writes:

Goodday, OpenStack DВaaS community.

I'd like to start conversation about guestagent security issue
related
to backup/restore process. Trove guestagent service uses AES with 256 bit
key (in CBC mode) [1] to encrypt backups which are stored at predefined
Swift container.

As you can see, password is defined in config file [2]. And here
comes
problem, this password is used for all tenants/projects that use Trove -
it
is a security issue. I would like to suggest Key derivation function [3]
based on static attributes specific for each tenant/project (tenant_id).
KDF would be based upon python implementation of PBKDF2 [4].
Implementation
can be seen here [5].

I do not want to see us writing our own crypto code in Trove. Id much
rather us use barbican for this, assuming it fits the bill. Lets do some
research on barbican before we go write this all.

To allow backward compatibility, python-troveclient should be able to
pass
old password [1] to guestagent as one of parameters on restore call.

Blueprint already have been registered in Trove launchpad space, [6].

I also foresee porting this feature to oslo-crypt, as part of security
framework (oslo.crypto) extensions.

Again, id rather see us use barbican for this instead of creating
oslo-crypt.

Thoughts ?

[1]

https://github.com/openstack/trove/blob/master/trove/guestagent/strategies/backup/base.py#L113-L116
[2]

https://github.com/openstack/trove/blob/master/etc/trove/trove-guestagent.conf.sample#L69
[3] http://en.wikipedia.org/wiki/Key_derivation_function
[4] http://en.wikipedia.org/wiki/PBKDF2
[5] https://gist.github.com/denismakogon/8823279
[6] https://blueprints.launchpad.net/trove/+spec/backup-encryption

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [python-openstacksdk] First meeting scheduled

2014-02-11 Thread Jesse Noller


On Feb 11, 2014, at 3:02 PM, Boris Pavlovic bpavlo...@mirantis.com wrote:

 Jesse,
 
 Why not next approach:
 1) Switch all clients to apiclient[1] (internal unification)

This was discussed several weeks ago here:

http://lists.openstack.org/pipermail/openstack-dev/2014-January/024441.html

This discussion covered everything you mention and more. At that time, it was 
pointed out the blueprint that the internal unification work was being checked 
in against lacked sufficient design, thought towards the end-user (e.g. 
developers). I have spoken to *a lot* of people - across multiple projects and 
teams (and companies) who concur that wrapping the existing clients is 
insufficient and actively *harmful* to the experience of end-users and 
consumers of openstack clouds. More below:

 2) Unify API of all clients (external unification)
 3) Unify work with keystone
 4) Graduate apiclient
 5) Switch all clients to apiclient lib
 6) Make one simple plugable mechanism. e.g. subclass based factory + steavdore
 7) Add one by one subclasse that present client in this factory 
 8) In one pretty day stop gates  switch to unified client.
 
 This is actually step by step solution that works for community and could be 
 done independently by tons of developers.
 

When this last came up, I was Linus’ed:

http://lists.openstack.org/pipermail/openstack-dev/2014-January/024583.html

And not everyone in openstack core is aligned with the common-client-library-2 
work:
https://review.openstack.org/#/q/topic:bp/common-client-library-2,n,z

And it is generally agreed that although “big up-front design” can be a drag, 
the current blueprints and work is confusing and a project like this actually 
requires a deal of design of front to ensure that the API contracts and 
interfaces we expose to users is consistent, usable and not entirely optimized 
for “developers who work on openstack”.

common-client-library-2: 
https://blueprints.launchpad.net/oslo/+spec/common-client-library-2

The blueprint is not a design, or a justification - it’s a todo list and while 
I think code cleanup and reuse is generally a Good Thing, I don’t think 
imposing a series of changes without some coordination and planning across the 
-dev group is going to end up with a good, consistent *end-user* experience. In 
fact, I think the goals are orthogonal - the goals of this BP work seems to be 
to reduce duplicated code; this is an optimization for the OpenStack project.

The python-openstacksdk and python-openstackclient are aimed at a fundamentally 
different (but overlapping set sometimes) audience: developers building 
applications that target openstack deployments, and other end-users. See the 
audience section here:

• Application Developers: Application developers are not OpenStack 
Operators or Developers. These are developers looking to consume a feature-rich 
OpenStack Cloud with its many services. These Developers require a consistent, 
single namespace API (Application Programming Interface) that allows them to 
build and deploy their application with minimal dependencies.

In a perfect scenario: python-openstackclient’s 1 dependency becomes 
python-openstacksdk which in turn has a dependency list of 1 or two fundamental 
libraries (e.g. requests). 

I did propose in the original thread that we join efforts: in fact, we already 
have a fully functioning, unified SDK that *could* be checked in today - but 
without discussing the APIs, design and other items with the community at 
large, I don’t think that that would be successful.


 
 [1] 
 https://github.com/openstack/oslo-incubator/tree/master/openstack/common/apiclient
 
 
 Best regards, 
 Boris Pavlovic 
 
 
 On Wed, Feb 12, 2014 at 12:40 AM, Jesse Noller jesse.nol...@rackspace.com 
 wrote:
 As I said last week; we’re ready to kickoff and have regular meetings for the 
 “unified python SDK” project. The initial meeting is scheduled on the wiki:
 
 https://wiki.openstack.org/wiki/Meetings#python-openstacksdk_Meeting
 
 Date/Time: Feb. 19th - 19:00 UTC / 1pm CST
 
 IRC channel: #openstack-meeting-3
 
 Meeting Agenda:
 https://wiki.openstack.org/wiki/Meetings/PythonOpenStackSDK
 
 About the project:
 https://wiki.openstack.org/wiki/PythonOpenStackSDK
 
 If you have questions, all of us lurk in #openstack-sdks on freenode!
 
 See you then.
 
 Jesse
 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
 
 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Trove] Replication Contract Verbiage

2014-02-11 Thread Michael Basnight

Daniel Salinas imsplit...@gmail.com writes:

 https://wiki.openstack.org/wiki/Trove-Replication-And-Clustering-API#REPLICATION

 I have updated the wiki page to reflect the current proposal for
 replication verbiage with some explanation of the choices.  I would like to
 open discussion here regarding that verbiage.  Without completely
 duplicating everything I just wrote in the wiki here are the proposed words
 that could be used to describe replication between two datastore instances
 of the same type.  Please take a moment to consider them and let me know
 what you think.  I welcome all feedback.

 replicates_from:  This term will be used in an instance that is a slave of
 another instance. It is a clear indicator that it is a slave of another
 instance.

 replicates_to: This term will be used in an instance that has slaves of
 itself. It is a clear indicator that it is a master of one or more
 instances.

Nice work daniel. I think these are quite sane. They are pretty agnostic
to the datastore type. The only thing i remember Stewart Smith saying
was that these may not _both_ _always_ apply to all datastores. So
assuming we have a builtin way to say that a given datastore/replication
type may not support both of these (or may not have a need to expose it
like this).

 writable: This term will be used in an instance to indicate whether it is
 intended to be used for writes. As replication is used commonly to scale
 read operations it is very common to have a read-only slave in many
 datastore types. It is beneficial to the user to be able to see this
 information when viewing the instance details via the api.

Sounds reasonable. But how do we view a multi-tier slave? aka, a slave
to a slave. Is it both read only and writale, so to speak, depending on
where you are in the cluster hierarchy?

 The intention here is to:
 1.  have a clearly defined replication contract between instances.
 2.  allow users to create a topology map simply by querying the api for
 details of instances linked in the replication contracts
 3.  allow the greatest level of flexibility for users when replicating
 their data so that Trove doesn't prescribe how they should make use of
 replication.

 I also think there is value in documenting common replication topologies
 per datastore type with example replication contracts and/or steps to
 recreate them in our api documentation.  There are currently no examples of
 this yet

++

 e.g. To create multi-master replication in mysql...

 As previously stated I welcome all feedback and would love input.

 Regards,

 Daniel Salinas
 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


pgpSmTHKNCxrc.pgp
Description: PGP signature
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [requirements] problems with non overlapping requirements changes

2014-02-11 Thread Sean Dague

A few weeks ago we realized one of the wrecking balls in the gate were
non overlapping requirements changes, like this -
https://review.openstack.org/#/c/72475/

Regular jobs in the gate have to use the OpenStack mirror. Requirements
repo doesn't, because it needs to be able to test things not in the mirror.

So when a requirements job goes into the gate, everything behind it will
be using the new requirements. But the mirror isn't updated until the
requirements change merges.

So if you make a non overlapping change like that, for 1hr (or more)
everything in the wake of the requirements job gets blown up in global
requirements because it can't install that from the mirror.

This issue is partially synthetic, however it does raise a good issue
for continuous deployed environments, because assuming atomic upgrade of
2 code bases isn't a good assumption.

Anyway, the point of this email is we really shouldn't be approving
requirements changes that are disjoint upgrades like that, because they
basically mean they'll trigger 10 - 20 -2s of other people's patches in
the gate.

-Sean

-- 
Sean Dague
Samsung Research America
s...@dague.net / sean.da...@samsung.com
http://dague.net



signature.asc
Description: OpenPGP digital signature
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] oslo.config improvements

2014-02-11 Thread Shaun McCance

On Mon, 2014-02-10 at 10:50 -0500, Doug Hellmann wrote:
 
 2) It locates options based on them being defined at the top
 level of a
 module (not in function or packed into some object), but it
 gets info by
 looking at cfg.CONF, which require registration. This fails if
 an option
 is defined by only registered when some function is called.
 This happens
 right now with the ip_lib_force_root option in Neutron. Both
 the config
 file generator and config docs generator currently crash on
 Neutron
 because of this.
 
 
 The option definition needs to be in a variable at the module top
 level, even if the registration call happens at run time. Can you file
 a bug against Neutron about that?

The option *is* currently defined at the module top level, though it's
only registered when a function is called. _guess_groups in generator.py
raises an error, because it looks for all instantiated options in the
cfg.CONF object, and it's not there.


 3) It's completely incapable of finding options that aren't
 defined or
 registered until some function is called or some object is
 instantiated.
 I don't have a general solution to this, although I do have
 special-case
 code that detects projects that use oslo.messaging and ensures
 those
 options get registered.
 
 
 I did some work to address this under
 https://blueprints.launchpad.net/oslo/+spec/improve-config-discovery-for-docs 
 so that libraries can register entry points that declare configuration 
 options. I think the only library using this feature so far is 
 oslo.messaging, but the other oslo libraries will use it when they graduate 
 from the incubator.

Oh, thanks. That's much cleaner than my workaround for oslo.messaging:

https://review.openstack.org/#/c/68196/



 To address these issues, I'd like to get oslo.config to know
 about the
 defining module for each option. It can get that using
 something like
 this in Opt.__init__:
 
 for frame in inspect.stack():
 mod = inspect.getmodule(frame[0])
 if mod == sys.modules[__name__]:
 continue
 self.module = mod.__name__
 break
 
 
 I'm not sure we want deployers to have to worry about where the option
 is defined. How would you use the information in the documentation? I
 guess we include it in the sample config output?

I agree that deployers shouldn't have to care about the defining module.
The only reason I've proposed it is that the entire architecture of the
sample config generator is geared around poking into modules looking for
options, because it wants to group options by module. This one addition
to Opt.__init__ would make things simpler and faster for the sample
config generator.

As for the docs, I'm not sure if we want that info in there or not. I
wouldn't push hard for it if it's not already there. Whether we'd bother
with it if it is available is something I'd have to ask the rest of the
docs team.

 
 We'd also have to modify __eq__ and __ne__, because it's valid
 for an
 option to be defined in two different modules as long as all
 its
 parameters are the same. Checking vars() equality would trip
 up on the
 module. (There's the further issue that the defining module is
 non-deterministic in those cases.)
 
 
 It's valid for an option to be registered with the same definition
 more than once, but having the definition live in more than one place
 is just asking for maintenance trouble later. Do we actually have this
 situation now?

auth_strategy is defined in both neutron/common/config.py and
neutron/agent/linux/interface.py. It would also be defined in
neutron/agent/metadata/agent.py, except it's defined in a class variable
and only registered by calling main().

  
 
 Then I'd like to get both the config file generator and the
 config docs
 generator using something like the OptionsCache class I wrote
 for the
 config docs generator:
 
 
 http://git.openstack.org/cgit/openstack/openstack-doc-tools/tree/autogenerate_config_docs/common.py#n88
 
 This could be simpler and faster and avoid the crash mentioned
 above
 with the module info attached to each option.
 
 
 Having 2 programs that discover options in 2 ways is a bad thing, so
 yes, let's combine them in oslo.config and let them share code.

Great. So I think there's three things we'll have to figure out:

1) What's the best way to group things by module? The answer might just
be It's not really worth grouping things by module.

2) Is it sufficient to just ask cfg.CONF for options, or do we need to
poke around looking for unregistered options? If the latter, where do

Re: [openstack-dev] [oslo-notify] notifications consumed by multiple subscribers

2014-02-11 Thread Sandy Walsh

The notification system can specify multiple queues to publish to, so each of 
your dependent services can feed from a separate queue. 

However, this is a critical bug in oslo.messaging that has broken this feature. 
https://bugs.launchpad.net/nova/+bug/1277204

Hopefully it'll get fixed quickly and you'll be able to do what you need.

There were plans for a notification consumer in oslo.messaging, but I don't 
know where it stands. I'm working on a standalone notification consumer library 
for rabbit. 

-S


From: Sanchez, Cristian A [cristian.a.sanc...@intel.com]
Sent: Tuesday, February 11, 2014 2:28 PM
To: OpenStack Development Mailing List (not for usage questions)
Subject: [openstack-dev] [oslo-notify] notifications consumed by multiple 
subscribers

Hi,
I’m planning to use oslo.notify mechanisms to implement a climate blueprint: 
https://blueprints.launchpad.net/climate/+spec/notifications. Ideally, the 
notifications sent by climate should be received by multiple services 
subscribed to the same topic. Is that possible with oslo.notify? And moreover, 
is there any mechanism for removing items from the queue? Or should one 
subscriber be responsible for removing items from it?

Thanks

Cristian

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] oslo.config improvements