date:20160523

On Mon, May 23, 2016 at 4:28 PM, Gregory Haynes  wrote:

> On Mon, May 23, 2016, at 05:24 PM, Morgan Fainberg wrote:
>
>
>
> On Mon, May 23, 2016 at 2:57 PM, Gregory Haynes 
> wrote:
>
> On Fri, May 20, 2016, at 07:48 AM, Thierry Carrez wrote:
> > John Dickinson wrote:
> > > [...]
> > >> So the real question we need to answer is... where does OpenStack
> > >> stop, and where does the wider open source community start ? If
> > >> OpenStack is purely an "integration engine", glue code for other
> > >> lower-level technologies like hypervisors, databases, or distributed
> > >> block storage, then the scope is limited, Python should be plenty
> > >> enough, and we don't need to fragment our community. If OpenStack is
> > >> "whatever it takes to reach our mission", then yes we need to add one
> > >> language to cover lower-level/native optimization, because we'll
> > >> need that... and we need to accept the community cost as a
> > >> consequence of that scope choice. Those are the only two options on
> > >> the table.
> > >>
> > >> I'm actually not sure what is the best answer. But I'm convinced we,
> > >> as a community, need to have a clear answer to that. We've been
> > >> avoiding that clear answer until now, creating tension between the
> > >> advocates of an ASF-like collection of tools and the advocates of a
> > >> tighter-integrated "openstack" product. We have created silos and
> > >> specialized areas as we got into the business of developing time-
> > >> series databases or SDNs. As a result, it's not "one community"
> > >> anymore. Should we further encourage that, or should we focus on
> > >> what the core of our mission is, what we have in common, this
> > >> integration engine that binds all those other open source projects
> > >> into one programmable infrastructure solution ?
> > >
> > > You said the answer in your question. OpenStack isn't defined as an
> > > integration engine[3]. The definition of OpenStack is whatever it
> > > takes to fulfill our mission[4][5]. I don't mean that as a tautology.
> > > I mean that we've already gone to the effort of defining OpenStack.
> It's
> > > our mission statement. We're all about building a cloud platform upon
> > > which people can run their apps ("cloud-native" or otherwise), so we
> > > write the software needed to do that.
> > >
> > > So where does OpenStack stop and the wider community start? OpenStack
> > > includes the projects needed to fulfill its mission.
> >
> > I'd totally agree with you if OpenStack was developed in a vacuum. But
> > there is a large number of open source projects and libraries that
> > OpenStack needs to fulfill its mission that are not in "OpenStack": they
> > are external open source projects we depend on. Python, MySQL, libvirt,
> > KVM, Ceph, OpenvSwitch, RabbitMQ... We are not asking that those should
> > be included in OpenStack, and we are not NIHing replacements for those
> > in OpenStack either.
> >
> > So it is not as clear-cut as you present it, and you can approach this
> > dependency question from two directions.
> >
> > One is community-centric: "anything produced by our community is
> > OpenStack". If we are missing a lower-level piece to achieve our mission
> > and are developing it ourselves as a result, then it is OpenStack, even
> > if it ends up being a message queue or a database.
> >
> > The other approach is product-centric: "lower-level pieces are OpenStack
> > dependencies, rather than OpenStack itself". If we are missing a
> > lower-level piece to achieve our mission and are developing it as a
> > result, it could be developed on OpenStack infrastructure by members of
> > the OpenStack community but it is not "OpenStack the product", it's an
> > OpenStack *dependency*. It is not governed by the TC, it can use any
> > language and tool deemed necessary.
> >
> > On this second approach, there is the obvious question of where
> > "lower-level" starts, which as you explained above is not really
> > clear-cut. A good litmus test for it could be whenever Python is not
> > enough. If you can't develop it effectively with the language that is
> > currently sufficient for the rest of OpenStack, then developing it as an
> > OpenStack dependency in whatever language is appropriate might be the
> > solution...
> >
> > That is what I mean by 'scope': where does "OpenStack" stop, and where
> > do "OpenStack dependencies" start ? It is a lot easier and a lot less
> > community-costly to allow additional languages in OpenStack dependencies
> > (we already have plenty there).
> >
>
> I strongly agree with both of the points about what OpenStack is defined
> as. We are  a set of projects attempting to fulfill our mission. In
> doing so, we try to use outside dependencies to help us as much as
> possible. Sometimes we cannot find an outside dependency to satisfy a
> need whether due to optimization needs, licensing issues, usability
> problems, or simply because an

Re: [openstack-dev] [glance] [defcore] [interop] Proposal for a virtual sync dedicated to Import Refactor May 26th

2016-05-23 Thread Mike Perez

On 18:00 May 20, Nikhil Komawar wrote:
> Hello all,
> 
> 
> I want to propose having a dedicated virtual sync next week Thursday May
> 26th at 1500UTC for one hour on the Import Refactor work [1] ongoing in
> Glance. We are making a few updates to the spec; so it would be good to
> have everyone on the same page and soon start merging those spec changes.
> 
> 
> Also, I would like for this sync to be cross project one so that all the
> different stakeholders are aware of the updates to this work even if you
> just want to listen in.
> 
> 
> Please vote with +1, 0, -1. Also, if the time doesn't work please
> propose 2-3 additional time slots.
> 
> 
> We can decide later on the tool and I will setup agenda if we have
> enough interest.

+1

-- 
Mike Perez

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [javascript] Seeking contributors, js-generator-openstack

2016-05-23 Thread Zhang Yujun

Hi, all

I'm happy to join the js-generator-openstack project.

We have listed the todos in story board on
https://storyboard.openstack.org/#!/project/842

Please feel free to take them and make contribution.

--
Yujun Zhang

On Mon, May 23, 2016 at 11:46 PM Michael Krotscheck 
wrote:

> Good idea. Since we've already got a storyboard project, let's start there.
>
> Michael
>
> On Fri, May 20, 2016 at 4:02 PM Zhang Yujun 
> wrote:
>
>> Hi, Michael
>>
>> As you are no longer alone now, we'd better to put things in your head
>> onto documents so that everybody who wish to contribute will know where to
>> go.
>>
>> Besides the technical roadmap, I think we shall need a space for issue
>> tracking and proposal discussion. After we make the project more open to
>> the community, it won't be long that more developers join this project.
>>
>> That's my basic thoughts for the moment.
>>
>> --
>> Yujun
>>
>> On Sat, May 21, 2016 at 1:10 AM Michael Krotscheck 
>> wrote:
>>
>>> Hi there!
>>>
>>> Well, the first thing we need is other reviewers, which is the fastest
>>> way to become a core :). The project page right now is the README.md
>>> file in the project itself. The main reason for this is that the target
>>> audience - javascript engineers - usually find that first via NPM. Most of
>>> the Todo items there have already been done, actually, so the next step
>>> would be to really identify what this project needs to accomplish, group it
>>> into major categories, and start working on it. Off the top of my head,
>>> here's a list:
>>>
>>>
>>>1. Dependency synchronization: Keep a list of semver
>>>global-dependencies.json at the root of the project, and update a 
>>> project's
>>>dependencies if the versions are out of sync.
>>>2. Eslint invocation. Infra's Common Testing Interface states that
>>>all javascript projects must support 'npm run lint', using
>>>eslint-config-openstack. The generator should add/update this to any
>>>project it's run in.
>>>3. nsp invocation. Not strictly necessary, but a postinstall scan of
>>>the project for publicly known vulnerabilities is always a good thing.
>>>
>>> After these pieces, the next step becomes more complicated, as we need
>>> to choose whether the user is creating a web application, or a node
>>> application. This then allows us to switch out which test harness and
>>> runner we're using, so that the `npm test` command can be consistent. Once
>>> this lands, we can start talking about project src/dist directories, how to
>>> best use gulp in each project type, and actual project templates :).
>>>
>>> Is there something in particular you'd like to work on?
>>>
>>> Michael
>>>
>>>
>>> On Thu, May 19, 2016 at 12:39 AM Zhang Yujun 
>>> wrote:
>>>
 Hi, Michael,

 I have several project experience in JavaScript and please let me know
 how I could help on this project?

 Is there a project page?

 Or we shall getting started with gerrit review?

 --
 Yujun

 On Wed, May 18, 2016 at 11:45 PM Michael Krotscheck <
 krotsch...@gmail.com> wrote:

> Hello everyone!
>
> The js-generator-openstack project has been incubated under
> openstack-infra, and is seeking contributors (and cores). The purpose of
> the project is as follows:
>
>- Help manage common project configuration aspects, such as
>licenses, gerrit, authors, and more.
>- Assist in keeping dependencies up-to-date and synchronized
>across javascript projects (JS equivalent of global requirements).
>- Provide all the necessary hooks for OpenStack's JavaScript
>Common Testing Interface.
>- Suggest common tools to use for tasks such as linting, unit
>testing, functional testing, and more.
>- (Newton Stretch) Provide a quick way of bootstrapping a new
>CORS-consuming OpenStack UI.
>
> I'm looking for help- firstly, because right now I'm the only person
> who's willing to review JavaScript amongst the various infra cores, and 
> I'd
> really like more eyeballs on this project. Secondly, because I know that
> I'm not the only person who has opinions about how we should be doing
> JavaScript things.
>
> Come on over to
> https://review.openstack.org/#/q/project:openstack-infra/js-generator-openstack+status:open
>  and
> help me out, would ya? If you've got questions, I'm active in the
> #openstack-javascript channel.
>
> Michael
>

> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe:
> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>

Re: [openstack-dev] [neutron][ovo] NeutronDbObject concurrency issues

2016-05-23 Thread Damon Wang

Hi,

I want to add an option which handle by another project Tooz.

https://github.com/openstack/tooz

with redis or some other drivers, it seems pretty a good choice.

Any comments?

Wei Wang

2016-05-17 6:53 GMT+08:00 Ilya Chukhnakov :

>
> On 16 May 2016, at 20:01, Michał Dulko  wrote:
>
> It's not directly related, but this reminds me of tests done by geguileo
> [1] some time ago that were comparing different methods of preventing DB
> race conditions in concurrent environment. Maybe you'll also find them
> useful as you'll probably need to do something like conditional update
> to increment a revision number.
>
> [1] https://github.com/Akrog/test-cinder-atomic-states
>
>
> Thanks for the link. The SQLA revisions are similar to the
> 'solutions/update_with_where',
> but they use the dedicated column for that [2]. And as long as it is
> properly configured,
> it happens 'automagically' (SQLA will take care of adding proper 'where'
> to 'update').
>
> [2] http://docs.sqlalchemy.org/en/latest/orm/versioning.html
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [OVN] [networking-ovn] [networking-sfc] SFC and OVN

2016-05-23 Thread Ryan Moats

John-
 
My apologies for not answering this sooner - I was giving a two day training on Tues/Wed last week and came back to my son graduating from HS the next day, so thiings have been a bit of a whirlwind here.
 
Looking at the github repos, I like the idea of passing a dictionary from networking-sfc to networking-ovn.  The flow classifiers should be relatively straightforward to map to ovs match rules (famous last words)...
 
I've probably missed an orbit here, but in the ovn-northd implementation, I was expecting to find service chains in the egress and router pipelines in addition to the ingress pipeline (see below for why I think a service chain stage in the egress pipeline makes sense ...)
 
Also, in the ovn-northd implementation, I'm a little disturbed to see the ingress side of the service chain sending packets to output ports - I think that a more
scalable (and more "ovs" approach) would be to match the egress side of a port pair in the chaining stage of the ingress pipeline, with an action that  set the input port register.  Then the egress pipeline would have a chaining stage where the output port register would be set based on the ingress port of the next port pair in the chain and the packet being punted to the proper output port in the last table.  That should automagically build your function chain and provider the bases for bucketizing multiple ingress ports for the next port group to support hash based load balancing.
 
Does that make sense?
 
Ryan
 
- Original message -From: John McDowall To: Ryan Moats/Omaha/IBM@IBMUSCc: "disc...@openvswitch.org" , "OpenStack Development Mailing List" Subject: Re: [OVN] [networking-ovn] [networking-sfc] SFC and OVNDate: Wed, May 18, 2016 3:55 PM 
Ryan,
 
OK all three repos and now aligned with their masters. I have done some simple level system tests and I can steer traffic to a single VNF.  Note: some additional changes to networking-sfc to catch-up with their changes.
 
https://github.com/doonhammer/networking-sfc 
https://github.com/doonhammer/networking-ovn
https://github.com/doonhammer/ovs 
 
The next tasks I see are:
 
 
Decouple networking-sfc and networking-ovn. I am thinking that I will pass a nested port-chain dictionary holding port-pairs/port-pair-groups/flow-classifiers from networking-sfc to networking-ovn.Align the interface between networking-ovn and ovs/ovn to match the nested dictionary in 1.Modify the ovn-nb schema and ovn-northd.c to march the port-chain model.Add ability to support chain of port-pairsThink about flow-classifiers and how best to map them, today I just map the logical-port and ignore everything else.
 
Any other suggestions/feedback?
 
Regards
 
John
  
From: Ryan Moats Date: Wednesday, May 11, 2016 at 1:39 PMTo: John McDowall Cc: "disc...@openvswitch.org" , OpenStack Development Mailing List Subject: Re: [OVN] [networking-ovn] [networking-sfc] SFC and OVN 
  
John McDowall  wrote on 05/11/2016 12:37:40 PM:> From: John McDowall > To: Ryan Moats/Omaha/IBM@IBMUS> Cc: "disc...@openvswitch.org" , "OpenStack> Development Mailing List" > Date: 05/11/2016 12:37 PM> Subject: Re: [OVN] [networking-ovn] [networking-sfc] SFC and OVN>> Ryan,>> I have done networking-sfc the files that I see as changed/added are:>> devstack/settings   Modified Runtime setting to pick up OVN Driver> networking_sfc/db/migration/alembic_migrations/versions/mitaka/> expand/5a475fc853e6_ovs_data_model.py Hack to work around> flow_classifier issue – need to resolve with SFC team.> networking_sfc/services/sfc/drivers/ovn/__init__.py   Added for OVN Driver> networking_sfc/services/sfc/drivers/ovn/driver.py     Added ovn driver file> setup.cfg Inserted OVN driver entry>> I am currently working to clean up ovs/ovn.>> Regards>> JohnI can confirm that the networking-sfc rebase goes in clean againstmaster for me :) - Looking forward to ovs ...Ryan 
 


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Magnum][Octavia] Need help for LBaaS v2

2016-05-23 Thread 王华

Hi Michael,

Thank you for your help. Do you need any other logs to debug this problem?

Regards,
Wanghua

On Tue, May 24, 2016 at 12:31 AM, Michael Johnson 
wrote:

> Hi Wanghua,
>
> From the o-cw log, it looks like the amphora service VM did not boot
> properly or the network is not configured correctly.  We can see that
> Nova said the VM went active, but the amphora-agent inside the image
> never became reachable.  I would check the nova instance console log
> to make sure the instance finished booting and check that the octavia
> management network is properly setup such that the amphora-agent can
> be reached.
>
> Michael
>
>
> On Sun, May 22, 2016 at 8:31 PM, 王华  wrote:
> > Hi all,
> >
> > Previously Magnum used LBaaS v1. Now LBaaS v1 is deprecated, so we want
> to
> > replace it by LBaaS v2. But I met a problem in my patch
> > https://review.openstack.org/#/c/314060/. I could not figure out why it
> > didn't work. It seems there are some errors in
> >
> http://logs.openstack.org/60/314060/5/check/gate-functional-dsvm-magnum-k8s/6e3795e/logs/screen-o-cw.txt.gz
> .
> > Can anyone in Octavia team help me to figure out why my patch doesn't
> work
> > in the gate?  Thank you.
> >
> > Regards,
> > Wanghua
> >
> >
> __
> > OpenStack Development Mailing List (not for usage questions)
> > Unsubscribe:
> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> >
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [openstack-operators][cinder] max_concurrent_builds in Cinder

2016-05-23 Thread John Griffith

On Mon, May 23, 2016 at 8:32 AM, Ivan Kolodyazhny  wrote:

> Hi developers and operators,
> I would like to get any feedback from you about my idea before I'll start
> work on spec.
>
> In Nova, we've got max_concurrent_builds option [1] to set 'Maximum number
> of instance builds to run concurrently' per each compute. There is no
> equivalent Cinder.
>
> Why do we need it for Cinder? IMO, it could help us to address following
> issues:
>
>- Creation of N volumes at the same time increases a lot of resource
>usage by cinder-volume service. Image caching feature [2] could help us a
>bit in case when we create volume form image. But we still have to upload N
>images to the volumes backend at the same time.
>- Deletion on N volumes at parallel. Usually, it's not very hard task
>for Cinder, but if you have to delete 100+ volumes at once, you can fit
>different issues with DB connections, CPU and memory usages. In case of
>LVM, it also could use 'dd' command to cleanup volumes.
>- It will be some kind of load balancing in HA mode: if cinder-volume
>process is busy with current operations, it will not catch message from
>RabbitMQ and other cinder-volume service will do it.
>- From users perspective, it seems that better way is to create/delete
>N volumes a bit slower than fail after X volumes were created/deleted.
>
>
> [1]
> https://github.com/openstack/nova/blob/283da2bbb74d0a131a66703516d16698a05817c7/nova/conf/compute.py#L161-L163
> [2]
> https://specs.openstack.org/openstack/cinder-specs/specs/liberty/image-volume-cache.html
>
> Regards,
> Ivan Kolodyazhny,
> http://blog.e0ne.info/
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
> Just curious about a couple things:  Is this attempting to solve a
problem in the actual Cinder Volume Service or is this trying to solve
problems with backends that can't keep up and deliver resources under heavy
load?  I get the copy-image to volume, that's a special case that certainly
does impact Cinder services and the Cinder node itself, but there's already
throttling going on there, at least in terms of IO allowed.

Also, I'm curious... would the exiting API Rate Limit configuration achieve
the same sort of thing you want to do here?  Granted it's not selective but
maybe it's worth mentioning.

If we did do something like this I would like to see it implemented as a
driver config; but that wouldn't help if the problem lies in the Rabbit or
RPC space.  That brings me back to wondering about exactly where we want to
solve problems and exactly which.  If delete is causing problems like you
describe I'd suspect we have an issue in our DB code (too many calls to
start with) and that we've got some overhead elsewhere that should be
eradicated.  Delete is a super simple operation on the Cinder side of
things (and most back ends) so I'm a bit freaked out thinking that it's
taxing resources heavily.

Thanks,
John
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [nova] Suggest to not deprecate os-migrations API

2016-05-23 Thread Zhenyu Zheng

Hi All,

According to
https://github.com/openstack/nova/blob/master/releasenotes/notes/os-migrations-ef225e5b309d5497.yaml
, we are going to deprecate the old os-migrations API, and two new APIs:
/servers/{server uuid}/migrations and /servers/{server
uuid}/migrations/{migration id} is added.

As we can see, the newly added APIs cannot work if we don't know which
instance is migrating. If our user uses HA applications or DRS applications
such as openstack-watcher, automatic migrations can take place, we may not
know which instance is migrating. And an API like the old os-migrations
will be a really good choice to use, we can get all the current running
migrations in simply one call. So I suggest not deprecate this API.

Any thoughts?

BR,
Kevin Zheng
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [tricircle] Launch work group for cross-pod L2 networking feature

2016-05-23 Thread Yipei Niu

Hi, all,

Since I have to attend the dissertation defense of my master degree on
Wednesday morning, I cannot attend the group meeting. Sorry about that. If
it is inconvenient to reschedule the meeting, I will keep online in
#openstack-tricircle so as to receive the meeting log.

Best regards,
Yipei
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [PTLs][all][mentoring] Mentors needed in specific technical areas

2016-05-23 Thread Sean McGinnis

Hi Emily,

You have a couple spots in your message that look like they were
supposed to be hyperlinks. Do you have those links to the guidelines and
sign up form?

Great program - glad to see it's taking off!

Thanks,
Sean

On Mon, May 23, 2016 at 10:24:57AM -0400, Emily K Hugenbruch wrote:
> 
> 
> Hi,
> The lightweight mentoring program sponsored by the Women of OpenStack has
> really taken off, and we have about 35 mentees looking for technical help
> that we don't have mentors for.  We're asking for help from the PTLs to
> announce the mentoring program in team meetings then direct people to the
> guidelines (here and here) and signup form if they're interested.
> 
> Mentors should be regular contributors to a project, with an interest in
> helping new people and about 4 hours a month for mentoring.  They do not
> have to be women; the program is just sponsored by WoO, we welcome all
> mentees and mentors.
> 
> These are the projects/areas where we especially need mentors:
>   Cinder
>   Containers
>   Documentation
>   Glance
>   Keystone
>   Murano
>   Neutron
>   Nova
>   Ops
>   Searchlight
>   Telemetry
>   TripleO
>   Trove
> 
> If you have any questions you can contact me, or ask on openstack-women
> where the mentoring committee hangs out.
> Thanks!
> Emily Hugenbruch
> IRC: ekhugen

> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [openstack-operators][cinder] max_concurrent_builds in Cinder

2016-05-23 Thread Sean McGinnis

On Mon, May 23, 2016 at 05:32:45PM +0300, Ivan Kolodyazhny wrote:
> Hi developers and operators,
> I would like to get any feedback from you about my idea before I'll start
> work on spec.
> 
> In Nova, we've got max_concurrent_builds option [1] to set 'Maximum number
> of instance builds to run concurrently' per each compute. There is no
> equivalent Cinder.

I do think it would be worth having some sort of operation throttling.

So are you thinking something more the "maximum number of volume builds
to run concurrently"? As you point out below, there are other operations
besides the creation that can cause problems when run in bulk.

I do like the idea!

> 
> Why do we need it for Cinder? IMO, it could help us to address following
> issues:
> 
>- Creation of N volumes at the same time increases a lot of resource
>usage by cinder-volume service. Image caching feature [2] could help us a
>bit in case when we create volume form image. But we still have to upload N
>images to the volumes backend at the same time.
>- Deletion on N volumes at parallel. Usually, it's not very hard task
>for Cinder, but if you have to delete 100+ volumes at once, you can fit
>different issues with DB connections, CPU and memory usages. In case of
>LVM, it also could use 'dd' command to cleanup volumes.
>- It will be some kind of load balancing in HA mode: if cinder-volume
>process is busy with current operations, it will not catch message from
>RabbitMQ and other cinder-volume service will do it.
>- From users perspective, it seems that better way is to create/delete N
>volumes a bit slower than fail after X volumes were created/deleted.
> 
> 
> [1]
> https://github.com/openstack/nova/blob/283da2bbb74d0a131a66703516d16698a05817c7/nova/conf/compute.py#L161-L163
> [2]
> https://specs.openstack.org/openstack/cinder-specs/specs/liberty/image-volume-cache.html
> 
> Regards,
> Ivan Kolodyazhny,
> http://blog.e0ne.info/

> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [neutron] DHCP Agent Scheduling for Segments

2016-05-23 Thread Carl Baldwin

It would be worth looking at the code that adds AZ awareness to DHCP
scheduling.  There might be something to learn.

However, by design, segments and AZs are orthogonal concepts that don't
force any kind of correlation.  There are segments that span AZs and AZs
that span segments.  I really want to be careful not to make assumptions.

Another very important distinction is that with multiple AZs today, each
DHCP server is generally available to other AZs.  We just wanted a way to
have multiple servers in different failure domains.  With routed segments,
it is different, we need to be sure there is a DHCP server in each segment
with a DHCP enabled subnet.  Otherwise, DHCP will be unreachable from it.

Carl
On May 22, 2016 7:32 AM, "Gary Kotton"  wrote:

> Today the DHCP schedulers have support for ‘AZ hints’. My understanding is
> that a segment is a subset of an AZ. So why are we not able to leverage
> that logic or make it more generic?
> Thanks
> Gary
>
> On 5/22/16, 4:02 AM, "Carl Baldwin"  wrote:
>
> >On Fri, May 20, 2016 at 1:44 PM, Brandon Logan
> > wrote:
> >> On Thu, 2016-05-19 at 14:16 -0600, Carl Baldwin wrote:
> >>> On Wed, May 18, 2016 at 1:36 PM, Kevin Benton 
> wrote:
> >>> >>I may have wrongly assumed that segments MAY have the possibility of
> being
> >>> >> l2 adjacent, even if the entire network they are in is not, which
> would mean
> >>> >> that viewing and scheduling these in the context of a segment could
> be
> >>> >> useful.
> >>> >
> >>> > Segments could be L2 adjacent, but I think it would be pretty
> uncommon for a
> >>> > DHCP agent to have access to multiple L2 adjacent segments for the
> same
> >>> > network. But even if that happens, the main use case I see for the
> scheduler
> >>> > API is taking networks off of dead agents, agents going under
> maintenance,
> >>> > or agents under heavy load. With the introduction of segments, all
> of those
> >>> > are still possible via the network-based API.
> >>>
> >>> I think I agree with this.  Let's not change the API at all to begin
> >>> with.  I do think this means that the current API should work with or
> >>> without segments.  I'm not sure that the current approach of doing
> >>> scheduling for segments completely independently of scheduling for
> >>> networks works for this.  Does it?
> >>>
> >>
> >> I still think it does, but we can make it work without making them
> >> separate.  My original plan was to keep them together, but that ended up
> >> causing some unclean code and also the possibility of requiring an
> >> interface change, which would break out-of-tree schedulers like bgp,
> >> that just got moved out of tree.  If I can devise an alternative to
> >> breaking that interface, then I'll go forward without separate
> >> schedulers.
> >
> >I think we've got a general idea of how it should behave.  Let's see
> >what you come up with.
> >
> >Carl
> >
> >__
> >OpenStack Development Mailing List (not for usage questions)
> >Unsubscribe:
> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> >http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all][tc] Languages vs. Scope of "OpenStack"

2016-05-23 Thread Geoff O'Callaghan

Surely openstack is defined by it’s capabilities and interfaces rather than 
it’s internals.  Given the simplistic view that openstack is a collection of 
micro services connected by well defined api’s does it really matter what code 
is used inside that micro service (or macro service )?   If there is a 
community willing to bring and support code in language ‘x’,  isn’t that better 
than worrying about the off chance of existing developers wishing to move 
projects and not knowing the target language?Is there a fear that we’ll end 
up with a fork of nova (or others) written in rust ?
If we’re not open to evolving then we’ll die.

Just throwing in a different perspective.

Sorry about the top-post but it applies in general to the below discussion
Tks
Geoff

> On 24 May 2016, at 9:28 AM, Gregory Haynes  wrote:
> 
> On Mon, May 23, 2016, at 05:24 PM, Morgan Fainberg wrote:
>>  
>>  
>> On Mon, May 23, 2016 at 2:57 PM, Gregory Haynes > > wrote:
>> On Fri, May 20, 2016, at 07:48 AM, Thierry Carrez wrote:
>> > John Dickinson wrote:
>> > > [...]
>> > >> So the real question we need to answer is... where does OpenStack
>> > >> stop, and where does the wider open source community start ? If
>> > >> OpenStack is purely an "integration engine", glue code for other
>> > >> lower-level technologies like hypervisors, databases, or distributed
>> > >> block storage, then the scope is limited, Python should be plenty
>> > >> enough, and we don't need to fragment our community. If OpenStack is
>> > >> "whatever it takes to reach our mission", then yes we need to add one
>> > >> language to cover lower-level/native optimization, because we'll
>> > >> need that... and we need to accept the community cost as a
>> > >> consequence of that scope choice. Those are the only two options on
>> > >> the table.
>> > >>
>> > >> I'm actually not sure what is the best answer. But I'm convinced we,
>> > >> as a community, need to have a clear answer to that. We've been
>> > >> avoiding that clear answer until now, creating tension between the
>> > >> advocates of an ASF-like collection of tools and the advocates of a
>> > >> tighter-integrated "openstack" product. We have created silos and
>> > >> specialized areas as we got into the business of developing time-
>> > >> series databases or SDNs. As a result, it's not "one community"
>> > >> anymore. Should we further encourage that, or should we focus on
>> > >> what the core of our mission is, what we have in common, this
>> > >> integration engine that binds all those other open source projects
>> > >> into one programmable infrastructure solution ?
>> > >
>> > > You said the answer in your question. OpenStack isn't defined as an
>> > > integration engine[3]. The definition of OpenStack is whatever it
>> > > takes to fulfill our mission[4][5]. I don't mean that as a tautology.
>> > > I mean that we've already gone to the effort of defining OpenStack. It's
>> > > our mission statement. We're all about building a cloud platform upon
>> > > which people can run their apps ("cloud-native" or otherwise), so we
>> > > write the software needed to do that.
>> > >
>> > > So where does OpenStack stop and the wider community start? OpenStack
>> > > includes the projects needed to fulfill its mission.
>> >
>> > I'd totally agree with you if OpenStack was developed in a vacuum. But
>> > there is a large number of open source projects and libraries that
>> > OpenStack needs to fulfill its mission that are not in "OpenStack": they
>> > are external open source projects we depend on. Python, MySQL, libvirt,
>> > KVM, Ceph, OpenvSwitch, RabbitMQ... We are not asking that those should
>> > be included in OpenStack, and we are not NIHing replacements for those
>> > in OpenStack either.
>> >
>> > So it is not as clear-cut as you present it, and you can approach this
>> > dependency question from two directions.
>> >
>> > One is community-centric: "anything produced by our community is
>> > OpenStack". If we are missing a lower-level piece to achieve our mission
>> > and are developing it ourselves as a result, then it is OpenStack, even
>> > if it ends up being a message queue or a database.
>> >
>> > The other approach is product-centric: "lower-level pieces are OpenStack
>> > dependencies, rather than OpenStack itself". If we are missing a
>> > lower-level piece to achieve our mission and are developing it as a
>> > result, it could be developed on OpenStack infrastructure by members of
>> > the OpenStack community but it is not "OpenStack the product", it's an
>> > OpenStack *dependency*. It is not governed by the TC, it can use any
>> > language and tool deemed necessary.
>> >
>> > On this second approach, there is the obvious question of where
>> > "lower-level" starts, which as you explained above is not really
>> > clear-cut. A good litmus test for it could be whenever Python is not
>> > enough. If you can't develop it

Re: [openstack-dev] [all] when are your mid-cycle meetups?

2016-05-23 Thread Nikhil Komawar



On 5/23/16 5:57 PM, Anita Kuno wrote:
> On 05/23/2016 05:36 PM, Nikhil Komawar wrote:
>> FYI,
>>
>> Glance one was cancelled (so can't put on the wiki), as we are
>> preferring the virtual one for more participation.
>>
>> http://lists.openstack.org/pipermail/openstack-dev/2016-May/095591.html
> Did you add it to the wiki page for virtual sprints?
> https://wiki.openstack.org/wiki/VirtualSprints

idnk. thanks!

>
>> On 5/23/16 5:18 PM, Doug Hellmann wrote:
>>> We are trying to help the organizers of the community bug squash event
>>> choose good dates for their Newton event. We would like to avoid any
>>> sprints or mid-cycles, but there aren't very many listed in the wiki
>>> [1]. If your team is planning an event, please list it so we can do our
>>> best to avoid scheduling conflicts.
>>>
>>> Thanks,
>>> Doug
>>>
>>> [1] https://wiki.openstack.org/wiki/Sprints
>>>
>>> __
>>> OpenStack Development Mailing List (not for usage questions)
>>> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>>
>> __
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

-- 

Thanks,
Nikhil


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all][tc] Languages vs. Scope of "OpenStack"

2016-05-23 Thread Erno Kuvaja

On Tue, May 24, 2016 at 12:28 AM, Gregory Haynes 
wrote:

> On Mon, May 23, 2016, at 05:24 PM, Morgan Fainberg wrote:
>
>
>
> On Mon, May 23, 2016 at 2:57 PM, Gregory Haynes 
> wrote:
>
> On Fri, May 20, 2016, at 07:48 AM, Thierry Carrez wrote:
> > John Dickinson wrote:
> > > [...]
> > >> So the real question we need to answer is... where does OpenStack
> > >> stop, and where does the wider open source community start ? If
> > >> OpenStack is purely an "integration engine", glue code for other
> > >> lower-level technologies like hypervisors, databases, or distributed
> > >> block storage, then the scope is limited, Python should be plenty
> > >> enough, and we don't need to fragment our community. If OpenStack is
> > >> "whatever it takes to reach our mission", then yes we need to add one
> > >> language to cover lower-level/native optimization, because we'll
> > >> need that... and we need to accept the community cost as a
> > >> consequence of that scope choice. Those are the only two options on
> > >> the table.
> > >>
> > >> I'm actually not sure what is the best answer. But I'm convinced we,
> > >> as a community, need to have a clear answer to that. We've been
> > >> avoiding that clear answer until now, creating tension between the
> > >> advocates of an ASF-like collection of tools and the advocates of a
> > >> tighter-integrated "openstack" product. We have created silos and
> > >> specialized areas as we got into the business of developing time-
> > >> series databases or SDNs. As a result, it's not "one community"
> > >> anymore. Should we further encourage that, or should we focus on
> > >> what the core of our mission is, what we have in common, this
> > >> integration engine that binds all those other open source projects
> > >> into one programmable infrastructure solution ?
> > >
> > > You said the answer in your question. OpenStack isn't defined as an
> > > integration engine[3]. The definition of OpenStack is whatever it
> > > takes to fulfill our mission[4][5]. I don't mean that as a tautology.
> > > I mean that we've already gone to the effort of defining OpenStack.
> It's
> > > our mission statement. We're all about building a cloud platform upon
> > > which people can run their apps ("cloud-native" or otherwise), so we
> > > write the software needed to do that.
> > >
> > > So where does OpenStack stop and the wider community start? OpenStack
> > > includes the projects needed to fulfill its mission.
> >
> > I'd totally agree with you if OpenStack was developed in a vacuum. But
> > there is a large number of open source projects and libraries that
> > OpenStack needs to fulfill its mission that are not in "OpenStack": they
> > are external open source projects we depend on. Python, MySQL, libvirt,
> > KVM, Ceph, OpenvSwitch, RabbitMQ... We are not asking that those should
> > be included in OpenStack, and we are not NIHing replacements for those
> > in OpenStack either.
> >
> > So it is not as clear-cut as you present it, and you can approach this
> > dependency question from two directions.
> >
> > One is community-centric: "anything produced by our community is
> > OpenStack". If we are missing a lower-level piece to achieve our mission
> > and are developing it ourselves as a result, then it is OpenStack, even
> > if it ends up being a message queue or a database.
> >
> > The other approach is product-centric: "lower-level pieces are OpenStack
> > dependencies, rather than OpenStack itself". If we are missing a
> > lower-level piece to achieve our mission and are developing it as a
> > result, it could be developed on OpenStack infrastructure by members of
> > the OpenStack community but it is not "OpenStack the product", it's an
> > OpenStack *dependency*. It is not governed by the TC, it can use any
> > language and tool deemed necessary.
> >
> > On this second approach, there is the obvious question of where
> > "lower-level" starts, which as you explained above is not really
> > clear-cut. A good litmus test for it could be whenever Python is not
> > enough. If you can't develop it effectively with the language that is
> > currently sufficient for the rest of OpenStack, then developing it as an
> > OpenStack dependency in whatever language is appropriate might be the
> > solution...
> >
> > That is what I mean by 'scope': where does "OpenStack" stop, and where
> > do "OpenStack dependencies" start ? It is a lot easier and a lot less
> > community-costly to allow additional languages in OpenStack dependencies
> > (we already have plenty there).
> >
>
> I strongly agree with both of the points about what OpenStack is defined
> as. We are  a set of projects attempting to fulfill our mission. In
> doing so, we try to use outside dependencies to help us as much as
> possible. Sometimes we cannot find an outside dependency to satisfy a
> need whether due to optimization needs, licensing issues, usability
> problems, or simply because an

Re: [openstack-dev] [oslo.service] Lifecycle Hooks

2016-05-23 Thread Joshua Harlow


Doug Hellmann wrote:

Excerpts from Kanagaraj Manickam's message of 2016-05-18 19:48:13 +0530:

DIms,

Use case could be anything, which would needed by either
operator/community, who wants to perform an required task before and

after

service is started. This requirement is very generic by nature, and I
believe it will be very useful.

Would like to give the sample use cases from from Operator&  OpenStack
community side as below.
Operator side, any pre/post actions could be hooked which is the
requirement for them. Simple example would be, one who wants to create an
journal of start/stop details like time, number of worker,

configurations,

etc in a common portal, this life-cycle hook would help.

Is that information not available in the logs?

[Kanagaraj M] its available in log, but one who wants to collect these info
in centralized portal,
it would help.


OpenStack community side, sample use cases would be:
1. Most of the OpenStack components starts TextGuruMeditation, logging
while those components are get started. These tasks could be provided as
life cycle hooks and all OpenStack components could start to leverage it.
All of those are things that need to be built into the app in a way that
they are started at the right time, rather than at an arbitrary point
defined by the plugin order a deployer has specified.

[Kanagaraj M] while i investigated the OpenStack service cmd, mostly
it follows the similar pattern on use these utils, so i thought, it would be
good to provide an plugin, which take care of it instead every service
code does take care of it. helps to reduces maintenance effort.


Except that we don't want deployers to turn them off, and we need to
control the initialization order, and so we don't want them to be
specified through a configuration option.


2. For automatically discovering the OpenStack deployment, this hooks

will

be very useful. Auto-discover-hook would report to pre-defined

destinations

while starting/stopping the service.

Doing that usefully is going to require passing information to the hook
so it knows where it is running (what service, what port, etc.). None of
the APIs for doing this have been described yet. Do you have any plans
put together?

[Kanagaraj M] I am trying to get all of these information from oslo.confg
global config variable. As we discussed about namos during austin summit,
namos does collect these details
https://github.com/openstack/os-namos/blob/master/os_namos/sync.py#L124


There are 2 issues with having a plugin access config values directly.

Configuration options are owned by the code that defines them, and
are not considered a public "API" for other parts of the code. That
means an application or library developer is free to change the
name or location of a configuration option, without regard to code
that might be trying to use it from outside of the owning module.
oslo.config has support for renames so that *deployers* aren't
broken, but we don't do anything to prevent code that accesses
private values from breaking.  So, you don't want to build any
assumptions into the plugins that they will be able to see configuration
values.

Second, options do not "exist" as far as oslo.config is concerned
until they are registered.  The registration step may happen at
runtime in code that has not executed before the plugin is loaded. So
even if we ignore the "private" nature of the option, there is a timing
issue.


It feels very much like you're advocating that we create a thing like
paste-deploy for non-WSGI apps: allow anyone to insert anything into
the executing process for any purpose and with little control on the
application authors' part. That introduces potential stability issues,
and a lot of questions that haven't been answered. For example:
Does service startup block until all of the plugins are done? If not,
do we need to have a timeout management system built in or do we run
the plugins in their own thread/subprocess so they can run in the
background?
Can a plugin change the execution of the service in some way (you
mentioned having a plugin download config files when we spoke at the
summit, is that still something you want to slip in this way instead of
putting it into oslo.config)?
Can a plugin cause the service to not start at all by exiting?
What happens if one plugin fails but others succeed? Do we keep running?
What information about the service can a plugin see? It's running in the
same process, so it could see the configuration object, for example.
It would only be able to see configuration options it adds (yes, that
would work) or that were registered by the application before the plugin
is run. So, not necessarily all options but potentially many, with more
and more as apps shift to pre-registering all of their options in one
place. Assuming these are things the deployer has selectively installed,
maybe that's OK. OTOH, it does open another security surface.
What happens when a service is told to restart its workers? Do

Re: [openstack-dev] [all][tc] Languages vs. Scope of "OpenStack"

On Mon, May 23, 2016, at 05:24 PM, Morgan Fainberg wrote:
>
>
> On Mon, May 23, 2016 at 2:57 PM, Gregory Haynes
>  wrote:
>> On Fri, May 20, 2016, at 07:48 AM, Thierry Carrez wrote:
>> > John Dickinson wrote:
>> > > [...]
>> > >> So the real question we need to answer is... where does
>> > >> OpenStack
>> > >> stop, and where does the wider open source community start ? If
>> > >> OpenStack is purely an "integration engine", glue code for other
>> > >> lower-level technologies like hypervisors, databases, or
>> > >> distributed
>> > >> block storage, then the scope is limited, Python should be
>> > >> plenty
>> > >> enough, and we don't need to fragment our community. If
>> > >> OpenStack is
>> > >> "whatever it takes to reach our mission", then yes we need to
>> > >> add one
>> > >> language to cover lower-level/native optimization, because we'll
>> > >> need that... and we need to accept the community cost as a
>> > >> consequence of that scope choice. Those are the only two options
>> > >> on
>> > >> the table.
>> > >>
>> > >> I'm actually not sure what is the best answer. But I'm convinced
>> > >> we,
>> > >> as a community, need to have a clear answer to that. We've been
>> > >> avoiding that clear answer until now, creating tension between
>> > >> the
>> > >> advocates of an ASF-like collection of tools and the advocates
>> > >> of a
>> > >> tighter-integrated "openstack" product. We have created silos
>> > >> and
>> > >> specialized areas as we got into the business of developing time-
>> > >> series databases or SDNs. As a result, it's not "one community"
>> > >> anymore. Should we further encourage that, or should we focus on
>> > >> what the core of our mission is, what we have in common, this
>> > >> integration engine that binds all those other open source
>> > >> projects
>> > >> into one programmable infrastructure solution ?
>> > >
>> > > You said the answer in your question. OpenStack isn't defined
>> > > as an
>> > > integration engine[3]. The definition of OpenStack is whatever it
>> > > takes to fulfill our mission[4][5]. I don't mean that as a
>> > > tautology.
>> > > I mean that we've already gone to the effort of defining
>> > > OpenStack. It's
>> > > our mission statement. We're all about building a cloud platform
>> > > upon
>> > > which people can run their apps ("cloud-native" or otherwise),
>> > > so we
>> > > write the software needed to do that.
>> > >
>> > > So where does OpenStack stop and the wider community start?
>> > > OpenStack
>> > > includes the projects needed to fulfill its mission.
>> >
>> > I'd totally agree with you if OpenStack was developed in a
>> > vacuum. But
>> > there is a large number of open source projects and libraries that
>> > OpenStack needs to fulfill its mission that are not in
>> > "OpenStack": they
>> > are external open source projects we depend on. Python, MySQL,
>> > libvirt,
>> > KVM, Ceph, OpenvSwitch, RabbitMQ... We are not asking that those
>> > should
>> > be included in OpenStack, and we are not NIHing replacements for
>> > those
>> > in OpenStack either.
>> >
>> > So it is not as clear-cut as you present it, and you can
>> > approach this
>> > dependency question from two directions.
>> >
>> > One is community-centric: "anything produced by our community is
>> > OpenStack". If we are missing a lower-level piece to achieve our
>> > mission
>> > and are developing it ourselves as a result, then it is
>> > OpenStack, even
>> > if it ends up being a message queue or a database.
>> >
>> > The other approach is product-centric: "lower-level pieces are
>> > OpenStack
>> > dependencies, rather than OpenStack itself". If we are missing a
>> > lower-level piece to achieve our mission and are developing it as a
>> > result, it could be developed on OpenStack infrastructure by
>> > members of
>> > the OpenStack community but it is not "OpenStack the product",
>> > it's an
>> > OpenStack *dependency*. It is not governed by the TC, it can
>> > use any
>> > language and tool deemed necessary.
>> >
>> > On this second approach, there is the obvious question of where
>> > "lower-level" starts, which as you explained above is not really
>> > clear-cut. A good litmus test for it could be whenever Python
>> > is not
>> > enough. If you can't develop it effectively with the language
>> > that is
>> > currently sufficient for the rest of OpenStack, then developing it
>> > as an
>> > OpenStack dependency in whatever language is appropriate might
>> > be the
>> > solution...
>> >
>> > That is what I mean by 'scope': where does "OpenStack" stop, and
>> > where
>> > do "OpenStack dependencies" start ? It is a lot easier and a
>> > lot less
>> > community-costly to allow additional languages in OpenStack
>> > dependencies
>> > (we already have plenty there).
>> >
>>
>> I strongly agree with both of the points about what OpenStack is
>> defined
>> as. We are  a set of projects attempting to fulfill our mission. In
>> doing so, we try to use

Re: [openstack-dev] [all][tc] Languages vs. Scope of "OpenStack"

On Mon, May 23, 2016 at 2:57 PM, Gregory Haynes  wrote:

> On Fri, May 20, 2016, at 07:48 AM, Thierry Carrez wrote:
> > John Dickinson wrote:
> > > [...]
> > >> So the real question we need to answer is... where does OpenStack
> > >> stop, and where does the wider open source community start ? If
> > >> OpenStack is purely an "integration engine", glue code for other
> > >> lower-level technologies like hypervisors, databases, or distributed
> > >> block storage, then the scope is limited, Python should be plenty
> > >> enough, and we don't need to fragment our community. If OpenStack is
> > >> "whatever it takes to reach our mission", then yes we need to add one
> > >> language to cover lower-level/native optimization, because we'll
> > >> need that... and we need to accept the community cost as a
> > >> consequence of that scope choice. Those are the only two options on
> > >> the table.
> > >>
> > >> I'm actually not sure what is the best answer. But I'm convinced we,
> > >> as a community, need to have a clear answer to that. We've been
> > >> avoiding that clear answer until now, creating tension between the
> > >> advocates of an ASF-like collection of tools and the advocates of a
> > >> tighter-integrated "openstack" product. We have created silos and
> > >> specialized areas as we got into the business of developing time-
> > >> series databases or SDNs. As a result, it's not "one community"
> > >> anymore. Should we further encourage that, or should we focus on
> > >> what the core of our mission is, what we have in common, this
> > >> integration engine that binds all those other open source projects
> > >> into one programmable infrastructure solution ?
> > >
> > > You said the answer in your question. OpenStack isn't defined as an
> > > integration engine[3]. The definition of OpenStack is whatever it
> > > takes to fulfill our mission[4][5]. I don't mean that as a tautology.
> > > I mean that we've already gone to the effort of defining OpenStack.
> It's
> > > our mission statement. We're all about building a cloud platform upon
> > > which people can run their apps ("cloud-native" or otherwise), so we
> > > write the software needed to do that.
> > >
> > > So where does OpenStack stop and the wider community start? OpenStack
> > > includes the projects needed to fulfill its mission.
> >
> > I'd totally agree with you if OpenStack was developed in a vacuum. But
> > there is a large number of open source projects and libraries that
> > OpenStack needs to fulfill its mission that are not in "OpenStack": they
> > are external open source projects we depend on. Python, MySQL, libvirt,
> > KVM, Ceph, OpenvSwitch, RabbitMQ... We are not asking that those should
> > be included in OpenStack, and we are not NIHing replacements for those
> > in OpenStack either.
> >
> > So it is not as clear-cut as you present it, and you can approach this
> > dependency question from two directions.
> >
> > One is community-centric: "anything produced by our community is
> > OpenStack". If we are missing a lower-level piece to achieve our mission
> > and are developing it ourselves as a result, then it is OpenStack, even
> > if it ends up being a message queue or a database.
> >
> > The other approach is product-centric: "lower-level pieces are OpenStack
> > dependencies, rather than OpenStack itself". If we are missing a
> > lower-level piece to achieve our mission and are developing it as a
> > result, it could be developed on OpenStack infrastructure by members of
> > the OpenStack community but it is not "OpenStack the product", it's an
> > OpenStack *dependency*. It is not governed by the TC, it can use any
> > language and tool deemed necessary.
> >
> > On this second approach, there is the obvious question of where
> > "lower-level" starts, which as you explained above is not really
> > clear-cut. A good litmus test for it could be whenever Python is not
> > enough. If you can't develop it effectively with the language that is
> > currently sufficient for the rest of OpenStack, then developing it as an
> > OpenStack dependency in whatever language is appropriate might be the
> > solution...
> >
> > That is what I mean by 'scope': where does "OpenStack" stop, and where
> > do "OpenStack dependencies" start ? It is a lot easier and a lot less
> > community-costly to allow additional languages in OpenStack dependencies
> > (we already have plenty there).
> >
>
> I strongly agree with both of the points about what OpenStack is defined
> as. We are  a set of projects attempting to fulfill our mission. In
> doing so, we try to use outside dependencies to help us as much as
> possible. Sometimes we cannot find an outside dependency to satisfy a
> need whether due to optimization needs, licensing issues, usability
> problems, or simply because an outside project doesn't exist. That is
> when things become less clear-cut and we might need to develop software
> not purely/directly related to

Re: [openstack-dev] [all] when are your mid-cycle meetups?

2016-05-23 Thread Anita Kuno

On 05/23/2016 05:36 PM, Nikhil Komawar wrote:
> FYI,
> 
> Glance one was cancelled (so can't put on the wiki), as we are
> preferring the virtual one for more participation.
> 
> http://lists.openstack.org/pipermail/openstack-dev/2016-May/095591.html

Did you add it to the wiki page for virtual sprints?
https://wiki.openstack.org/wiki/VirtualSprints

> 
> On 5/23/16 5:18 PM, Doug Hellmann wrote:
>> We are trying to help the organizers of the community bug squash event
>> choose good dates for their Newton event. We would like to avoid any
>> sprints or mid-cycles, but there aren't very many listed in the wiki
>> [1]. If your team is planning an event, please list it so we can do our
>> best to avoid scheduling conflicts.
>>
>> Thanks,
>> Doug
>>
>> [1] https://wiki.openstack.org/wiki/Sprints
>>
>> __
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> 
> 
> 
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> 


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all][tc] Languages vs. Scope of "OpenStack"

On Fri, May 20, 2016, at 07:48 AM, Thierry Carrez wrote:
> John Dickinson wrote:
> > [...]
> >> So the real question we need to answer is... where does OpenStack
> >> stop, and where does the wider open source community start ? If
> >> OpenStack is purely an "integration engine", glue code for other
> >> lower-level technologies like hypervisors, databases, or distributed
> >> block storage, then the scope is limited, Python should be plenty
> >> enough, and we don't need to fragment our community. If OpenStack is
> >> "whatever it takes to reach our mission", then yes we need to add one
> >> language to cover lower-level/native optimization, because we'll
> >> need that... and we need to accept the community cost as a
> >> consequence of that scope choice. Those are the only two options on
> >> the table.
> >>
> >> I'm actually not sure what is the best answer. But I'm convinced we,
> >> as a community, need to have a clear answer to that. We've been
> >> avoiding that clear answer until now, creating tension between the
> >> advocates of an ASF-like collection of tools and the advocates of a
> >> tighter-integrated "openstack" product. We have created silos and
> >> specialized areas as we got into the business of developing time-
> >> series databases or SDNs. As a result, it's not "one community"
> >> anymore. Should we further encourage that, or should we focus on
> >> what the core of our mission is, what we have in common, this
> >> integration engine that binds all those other open source projects
> >> into one programmable infrastructure solution ?
> >
> > You said the answer in your question. OpenStack isn't defined as an
> > integration engine[3]. The definition of OpenStack is whatever it
> > takes to fulfill our mission[4][5]. I don't mean that as a tautology.
> > I mean that we've already gone to the effort of defining OpenStack. It's
> > our mission statement. We're all about building a cloud platform upon
> > which people can run their apps ("cloud-native" or otherwise), so we
> > write the software needed to do that.
> >
> > So where does OpenStack stop and the wider community start? OpenStack
> > includes the projects needed to fulfill its mission.
> 
> I'd totally agree with you if OpenStack was developed in a vacuum. But 
> there is a large number of open source projects and libraries that 
> OpenStack needs to fulfill its mission that are not in "OpenStack": they 
> are external open source projects we depend on. Python, MySQL, libvirt, 
> KVM, Ceph, OpenvSwitch, RabbitMQ... We are not asking that those should 
> be included in OpenStack, and we are not NIHing replacements for those 
> in OpenStack either.
> 
> So it is not as clear-cut as you present it, and you can approach this 
> dependency question from two directions.
> 
> One is community-centric: "anything produced by our community is 
> OpenStack". If we are missing a lower-level piece to achieve our mission 
> and are developing it ourselves as a result, then it is OpenStack, even 
> if it ends up being a message queue or a database.
> 
> The other approach is product-centric: "lower-level pieces are OpenStack 
> dependencies, rather than OpenStack itself". If we are missing a 
> lower-level piece to achieve our mission and are developing it as a 
> result, it could be developed on OpenStack infrastructure by members of 
> the OpenStack community but it is not "OpenStack the product", it's an 
> OpenStack *dependency*. It is not governed by the TC, it can use any 
> language and tool deemed necessary.
> 
> On this second approach, there is the obvious question of where 
> "lower-level" starts, which as you explained above is not really 
> clear-cut. A good litmus test for it could be whenever Python is not 
> enough. If you can't develop it effectively with the language that is 
> currently sufficient for the rest of OpenStack, then developing it as an 
> OpenStack dependency in whatever language is appropriate might be the 
> solution...
> 
> That is what I mean by 'scope': where does "OpenStack" stop, and where 
> do "OpenStack dependencies" start ? It is a lot easier and a lot less 
> community-costly to allow additional languages in OpenStack dependencies 
> (we already have plenty there).
> 

I strongly agree with both of the points about what OpenStack is defined
as. We are  a set of projects attempting to fulfill our mission. In
doing so, we try to use outside dependencies to help us as much as
possible. Sometimes we cannot find an outside dependency to satisfy a
need whether due to optimization needs, licensing issues, usability
problems, or simply because an outside project doesn't exist. That is
when things become less clear-cut and we might need to develop software
not purely/directly related to fulfilling our mission.

In the product-centric approach I worry that we are going to be paying
many of the costs which the existing TC resolutions hoped to prevent. We
still will have to maintain and debug these

Re: [openstack-dev] [all] when are your mid-cycle meetups?

2016-05-23 Thread Nikhil Komawar

FYI,

Glance one was cancelled (so can't put on the wiki), as we are
preferring the virtual one for more participation.

http://lists.openstack.org/pipermail/openstack-dev/2016-May/095591.html

On 5/23/16 5:18 PM, Doug Hellmann wrote:
> We are trying to help the organizers of the community bug squash event
> choose good dates for their Newton event. We would like to avoid any
> sprints or mid-cycles, but there aren't very many listed in the wiki
> [1]. If your team is planning an event, please list it so we can do our
> best to avoid scheduling conflicts.
>
> Thanks,
> Doug
>
> [1] https://wiki.openstack.org/wiki/Sprints
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

-- 

Thanks,
Nikhil

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [all] when are your mid-cycle meetups?

We are trying to help the organizers of the community bug squash event
choose good dates for their Newton event. We would like to avoid any
sprints or mid-cycles, but there aren't very many listed in the wiki
[1]. If your team is planning an event, please list it so we can do our
best to avoid scheduling conflicts.

Thanks,
Doug

[1] https://wiki.openstack.org/wiki/Sprints

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] profiling - eg neutron-vpn-agent

2016-05-23 Thread Rick Jones


On 05/23/2016 01:33 PM, Kevin Benton wrote:

I've found that the issue is that if you interrupt with ctrl-C it won't
write the profile. However, sending it a SIGTERM with the 'kill' command
did the trick when I was using cprofile. I think oslo service calls
os.exit right on SIGINT so the profiler doesn't get a chance to write out.


SIGTERM does seem to result in a file being written-out.  Thanks.  That 
gets me one step closer.


happy benchmarking,

rick jones


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [tacker] any agenda item for tomorrow's IRC meeting ?

2016-05-23 Thread Sridhar Ramaswamy

Tackers -

Does anyone have a specific agenda item for tomorrow's IRC weekly meeting?
I know many specs are in flight and reviews are happening in gerrit. If I
don't hear any specific topics to discuss by 10:00 UTC I'll cancel the
meeting and give an hour back.

- Sridhar
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all][tc] Languages vs. Scope of "OpenStack"

On Mon, May 23, 2016, at 02:57 PM, Sean Dague wrote:
> On 05/23/2016 03:34 PM, Gregory Haynes wrote:
> > 
> > On Mon, May 23, 2016, at 11:48 AM, Doug Hellmann wrote:
> >> Excerpts from Chris Dent's message of 2016-05-23 17:07:36 +0100:
> >>> On Mon, 23 May 2016, Doug Hellmann wrote:
>  Excerpts from Chris Dent's message of 2016-05-20 14:16:15 +0100:
> > I don't think language does (or should) have anything to do with it.
> >
> > The question is whether or not the tool (whether service or
> > dependent library) is useful to and usable outside the openstack-stack.
> > For example gnocchi is useful to openstack but you can use it with other
> > stuff, therefore _not_ openstack. More controversially: swift can be
> > usefully used all by its lonesome: _not_ openstack.
> 
> > 
> > Making a tool which is useful outside of the OpenStack context just
> > seems like good software engineering - it seems odd that we would try
> > and ensure our tools do not fit this description. Fortunately, many (or
> > even most) of the tools we create *are* useful outside of the OpenStack
> > world - pbr, git-review, diskimage-builder, (I hope) many of the oslo
> > libraries. This is really a question of defining useful interfaces more
> > than anything else, not a statement of whether a tool is part of our
> > community.
> 
> Only if you are willing to pay the complexity and debt cost of having
> optional backends all over the place.
> 
> For instance, I think we're well beyond that point that Keystone being
> optional should be a thing anywhere (and it is a thing in a number of
> places). Keystone should be our auth system, all projects 100% depend on
> it, and if you have different site needs, put that into a Keystone
> backend.
> 

Services and Projects seem to be getting conflated here. IIUC Your two
points apply only to services - we certainly aren't paying any
complexity costs for making pbr optional and the same could be said for
many of our tools.

I don't have a ton of context for why some services are electing to pay
the cost of making Keystone optional. The point I was hoping to make is
that there is value in defining an interface which is useful outside of
OpenStack, and this is a very common pattern with many of our tools. I
completely agree that there are additional costs to doing so at times,
and obviously they have to be weighed against the benefits. That is
really a problem-specific issue, though.

> Most of the oslo libraries require other oslo libraries, which is fine.
> They aren't trying to solve the general purpose case of logging or
> configuration or db access. They are trying to solve a specific set of
> patterns that are applicable to OpenStack projects.
> 

This is true up to a point - there isn't any inherent value in
overfitting a problem to be OpenStack specific. To beat on the pbr
hammer some more - we created a tool that fulfills our needs and making
it in a way where others can use it didn't cost us anything. This isn't
always the case but sometimes it is, and there is absolutely value in
making a tool which others can use.

>   -Sean
> 
> -- 
> Sean Dague
> http://dague.net

Cheers,
Greg

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all][tc] Languages vs. Scope of "OpenStack"

2016-05-23 Thread Chris Dent


On Mon, 23 May 2016, Gregory Haynes wrote:

Excerpts from Chris Dent's message of 2016-05-20 14:16:15 +0100:

I don't think language does (or should) have anything to do with it.

The question is whether or not the tool (whether service or
dependent library) is useful to and usable outside the openstack-stack.
For example gnocchi is useful to openstack but you can use it with other
stuff, therefore _not_ openstack. More controversially: swift can be
usefully used all by its lonesome: _not_ openstack.


Making a tool which is useful outside of the OpenStack context just
seems like good software engineering - it seems odd that we would try
and ensure our tools do not fit this description. Fortunately, many (or
even most) of the tools we create *are* useful outside of the OpenStack
world - pbr, git-review, diskimage-builder, (I hope) many of the oslo
libraries. This is really a question of defining useful interfaces more
than anything else, not a statement of whether a tool is part of our
community.


Defining what the community is is what we're trying to do here, yes?

Based on conversations I've had without people _outside_ the
community, those people think that OpenStack is a community building
a thing called OpenStack but they are universally confused about
what OpenStack _is_.

Despite the great usefulness of those things I'm pretty sure the
OpenStack those people are looking for is neither pbr, not git-review
nor diskimage-builder. Those tools are critical to the development
of OpenStack, our attention to them is critical. But now much to
they impact the definition of OpenStack?

So, yet another way to frame the original question (in a loaded way)
may be: Are we trying to come up with a way of defining the community
that lets us carry on doing what we've been doing, haphazardly, or
are we trying to get the process of defining the community to bring
us to a point where we have some useful constraints that allow us to
more effectively reach goal X?

Begging, of course: What's X?

(To me, an unfettered big tent is a great way to keep riding the
great OpenStack enterprise boondoggle, but I'm not sure it's
resulting in a great experience for humans who aren't on that
train.)

--
Chris Dent   (╯°□°)╯︵┻━┻http://anticdent.org/
freenode: cdent tw: @anticdent__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] profiling - eg neutron-vpn-agent

2016-05-23 Thread Kevin Benton

I've found that the issue is that if you interrupt with ctrl-C it won't
write the profile. However, sending it a SIGTERM with the 'kill' command
did the trick when I was using cprofile. I think oslo service calls os.exit
right on SIGINT so the profiler doesn't get a chance to write out.
On May 23, 2016 13:23, "Rick Jones"  wrote:

> Folks -
>
> I'm looking for suggestions on how to go about profiling the likes of
> neutron-vpn-agent from Liberty.  I have a simple little test - nothing
> stressful - just create 1000 ports with floating IPs on a single private
> network with a CVR router :)  This seems to put the Liberty
> neutron-vpn-agent in situations where it will spend 100% of its time in
> user space doing something.  As the number of ports increases, so too does
> the length of time in this mode.  It ends up spending tens of minutes
> therein t the exclusion of doing anything else.
>
> So, I'd like to profile it.  To see what it thinks it is doing.
>
> I have tried running neutron-vpn-agent "by hand" on the controller hosting
> the vrouter using the "A typical profiling session with python 2.5 loos
> like this" part of
> https://wiki.python.org/moin/PythonSpeed/PerformanceTips#Profiling_Code
> (no, I'm not using Python 2.5, that is simply where web searching has lead
> me from the peanut gallery :) )
>
> Alas, it seems that ^C'ing it doesn't have the profile written-out.
>
> So I'm looking for other methods.
>
> happy benchmarking,
>
> rick jones
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all][tc] Languages vs. Scope of "OpenStack"

2016-05-23 Thread Joshua Harlow


Sean Dague wrote:

On 05/23/2016 03:34 PM, Gregory Haynes wrote:

On Mon, May 23, 2016, at 11:48 AM, Doug Hellmann wrote:

Excerpts from Chris Dent's message of 2016-05-23 17:07:36 +0100:

On Mon, 23 May 2016, Doug Hellmann wrote:

Excerpts from Chris Dent's message of 2016-05-20 14:16:15 +0100:

I don't think language does (or should) have anything to do with it.

The question is whether or not the tool (whether service or
dependent library) is useful to and usable outside the openstack-stack.
For example gnocchi is useful to openstack but you can use it with other
stuff, therefore _not_ openstack. More controversially: swift can be
usefully used all by its lonesome: _not_ openstack.

Making a tool which is useful outside of the OpenStack context just
seems like good software engineering - it seems odd that we would try
and ensure our tools do not fit this description. Fortunately, many (or
even most) of the tools we create *are* useful outside of the OpenStack
world - pbr, git-review, diskimage-builder, (I hope) many of the oslo
libraries. This is really a question of defining useful interfaces more
than anything else, not a statement of whether a tool is part of our
community.


Only if you are willing to pay the complexity and debt cost of having
optional backends all over the place.


We seem to do this quite well even without building tools/projects that 
work outside of openstack ;)


Or are you saying that using such library/project(s) u have to accept 
that there will be many optional backends that you will likely not/never 
use that exist in said library/project (and thus are akin to dead weight)?




For instance, I think we're well beyond that point that Keystone being
optional should be a thing anywhere (and it is a thing in a number of
places). Keystone should be our auth system, all projects 100% depend on
it, and if you have different site needs, put that into a Keystone backend.

Most of the oslo libraries require other oslo libraries, which is fine.
They aren't trying to solve the general purpose case of logging or
configuration or db access. They are trying to solve a specific set of
patterns that are applicable to OpenStack projects.


I just took a quick stab at annotating which ones (I think are) useable 
outside of openstack (without say bringing in the configuration 
ideology/pattern that oslo.config adds) and made the following:


automaton (useable)
cliff (useable)
cookiecutter (useable)
debtcollector (useable)
futurist (useable)
osprofiler (useable?)
oslo.cache
oslo.concurrency
oslo.context
oslo.config
oslo-cookiecutter
oslo.db
oslo.i18n
oslo.log
oslo.messaging
oslo.middleware
oslo.policy (useable?)
oslo.privsep (useable?)
oslo.reports
oslo.rootwrap
oslo.serialization (useable)
oslo.service
oslosphinx
oslotest (useable)
oslo.utils (useable)
oslo.versionedobjects
oslo.vmware
hacking (useable)
pbr (useable)
pyCADF (useable?)
stevedore (useable)
taskflow (useable)
tooz (useable)

So out of 33 that's about half (~17) that I think are useable outside 
without to many patterns/ideologies being forced on non-openstack folks 
(if your external project accepts the pattern of oslo.config then the 
number increases).




-Sean



__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] profiling - eg neutron-vpn-agent

2016-05-23 Thread Rick Jones


Folks -

I'm looking for suggestions on how to go about profiling the likes of 
neutron-vpn-agent from Liberty.  I have a simple little test - nothing 
stressful - just create 1000 ports with floating IPs on a single private 
network with a CVR router :)  This seems to put the Liberty 
neutron-vpn-agent in situations where it will spend 100% of its time in 
user space doing something.  As the number of ports increases, so too 
does the length of time in this mode.  It ends up spending tens of 
minutes therein t the exclusion of doing anything else.


So, I'd like to profile it.  To see what it thinks it is doing.

I have tried running neutron-vpn-agent "by hand" on the controller 
hosting the vrouter using the "A typical profiling session with python 
2.5 loos like this" part of 
https://wiki.python.org/moin/PythonSpeed/PerformanceTips#Profiling_Code 
(no, I'm not using Python 2.5, that is simply where web searching has 
lead me from the peanut gallery :) )


Alas, it seems that ^C'ing it doesn't have the profile written-out.

So I'm looking for other methods.

happy benchmarking,

rick jones

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all][tc] Languages vs. Scope of "OpenStack"

2016-05-23 Thread Sean Dague

On 05/23/2016 03:34 PM, Gregory Haynes wrote:
> 
> On Mon, May 23, 2016, at 11:48 AM, Doug Hellmann wrote:
>> Excerpts from Chris Dent's message of 2016-05-23 17:07:36 +0100:
>>> On Mon, 23 May 2016, Doug Hellmann wrote:
 Excerpts from Chris Dent's message of 2016-05-20 14:16:15 +0100:
> I don't think language does (or should) have anything to do with it.
>
> The question is whether or not the tool (whether service or
> dependent library) is useful to and usable outside the openstack-stack.
> For example gnocchi is useful to openstack but you can use it with other
> stuff, therefore _not_ openstack. More controversially: swift can be
> usefully used all by its lonesome: _not_ openstack.

> 
> Making a tool which is useful outside of the OpenStack context just
> seems like good software engineering - it seems odd that we would try
> and ensure our tools do not fit this description. Fortunately, many (or
> even most) of the tools we create *are* useful outside of the OpenStack
> world - pbr, git-review, diskimage-builder, (I hope) many of the oslo
> libraries. This is really a question of defining useful interfaces more
> than anything else, not a statement of whether a tool is part of our
> community.

Only if you are willing to pay the complexity and debt cost of having
optional backends all over the place.

For instance, I think we're well beyond that point that Keystone being
optional should be a thing anywhere (and it is a thing in a number of
places). Keystone should be our auth system, all projects 100% depend on
it, and if you have different site needs, put that into a Keystone backend.

Most of the oslo libraries require other oslo libraries, which is fine.
They aren't trying to solve the general purpose case of logging or
configuration or db access. They are trying to solve a specific set of
patterns that are applicable to OpenStack projects.

-Sean

-- 
Sean Dague
http://dague.net

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [new][oslo] pbr 1.10.0 release

We are psyched to announce the release of:

pbr 1.10.0: Python Build Reasonableness

With source available at:

http://git.openstack.org/cgit/openstack-dev/pbr

With package available at:

https://pypi.python.org/pypi/pbr

Please report issues through launchpad:

http://bugs.launchpad.net/pbr

For more details, please see below.

Changes in pbr 1.9.1..1.10.0


7f02669 File is wrongly marked as executable
8778c64 Fix wsgiref script use with oslo.config
dcc49c5 Update Preversioning explanation to avoid double that
ef759ed Correct server test
b87e4c0 Fix soabi tests with pypy

Diffstat (except docs and test files)
-

pbr/packaging.py  |  17 ++-
6 files changed, 107 insertions(+), 94 deletions(-)




__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [ironic] No meeting next monday, May 30

2016-05-23 Thread Jim Rollenhagen

Hi team,

There's a US holiday next Monday (May 30) and as such I'm cancelling the
corresponding weekly ironic meeting.

See you all the following week.

// jim

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all][tc] Languages vs. Scope of "OpenStack"

On Mon, May 23, 2016, at 11:48 AM, Doug Hellmann wrote:
> Excerpts from Chris Dent's message of 2016-05-23 17:07:36 +0100:
> > On Mon, 23 May 2016, Doug Hellmann wrote:
> > > Excerpts from Chris Dent's message of 2016-05-20 14:16:15 +0100:
> > >> I don't think language does (or should) have anything to do with it.
> > >>
> > >> The question is whether or not the tool (whether service or
> > >> dependent library) is useful to and usable outside the openstack-stack.
> > >> For example gnocchi is useful to openstack but you can use it with other
> > >> stuff, therefore _not_ openstack. More controversially: swift can be
> > >> usefully used all by its lonesome: _not_ openstack.
> > >

Making a tool which is useful outside of the OpenStack context just
seems like good software engineering - it seems odd that we would try
and ensure our tools do not fit this description. Fortunately, many (or
even most) of the tools we create *are* useful outside of the OpenStack
world - pbr, git-review, diskimage-builder, (I hope) many of the oslo
libraries. This is really a question of defining useful interfaces more
than anything else, not a statement of whether a tool is part of our
community.

> > > Add keystone, cinder, and ironic to that list.
> > 
> > Hmmm. You can, but would people want to (that is, would it be a sound
> > choice?)? Or _do_ people? Maybe that's the distinction? As far as I
> 
> Yes, I'm aware of cases of each of those projects being used without
> "the rest" of OpenStack. I used keystone like that to secure some
> internal APIs myself.
> 

This has become  a very popular way of using Ironic as well. We even
have an OpenStack project (bifrost) which is used to deploy Ironic in
this fashion.

Cheers,
Greg

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Keystone] Welcome Keystone to the World of Python 3

On Mon, May 23, 2016 at 12:03 PM, Doug Hellmann 
wrote:

> Excerpts from Morgan Fainberg's message of 2016-05-23 11:55:48 -0700:
> > On Mon, May 23, 2016 at 7:54 AM, Doug Hellmann 
> > wrote:
> >
> > > Excerpts from Morgan Fainberg's message of 2016-05-20 20:58:00 -0700:
> > > > We've gone through all of our test cases and all of our code base. At
> > > this
> > > > point Keystone is no longer skipping any of the tests (which do tend
> to
> > > > test the entire request stack) and we are properly gating on being
> > > > Python3.4 compatible.
> > > >
> > > > I want to thank everyone who has put in effort in the last few weeks
> to
> > > > punt the last of the patches though the gate. It would not have been
> > > doable
> > > > without those hacking on LdapPool, doing test cleanup, and those
> > > > reviewing/trying the code out.
> > > >
> > > > If you run across issues with Keystone and Python3, please let us
> know.
> > > >
> > > > A sincere thanks to the entire Keystone team involved in this
> multicycle
> > > > effort.
> > > >
> > > > --Morgan
> > >
> > > Is this for unit or functional tests?
> > >
> > >
> > Our unit tests and functional tests cross over significantly. The
> majority
> > of keystone's unit tests are really "stand up a full keystone and test
> the
> > API as though you were a client".
> >
> > This is not DSVM and true "functional" tests (most of the restful test
> > cases will move over to "functional" once it's working) are not fully
> setup
> > yet.
> >
> > In short "unit tests", except our unit tests are mostly "functional" in
> > nature.
> >
> > --Morgan
>
> Ah, OK, I was wondering if you were using the python 3 features of
> devstack and how well they were working for you.
>
>
Thats the next plan, of course.


> Doug
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Keystone] Welcome Keystone to the World of Python 3

Excerpts from Morgan Fainberg's message of 2016-05-23 11:55:48 -0700:
> On Mon, May 23, 2016 at 7:54 AM, Doug Hellmann 
> wrote:
> 
> > Excerpts from Morgan Fainberg's message of 2016-05-20 20:58:00 -0700:
> > > We've gone through all of our test cases and all of our code base. At
> > this
> > > point Keystone is no longer skipping any of the tests (which do tend to
> > > test the entire request stack) and we are properly gating on being
> > > Python3.4 compatible.
> > >
> > > I want to thank everyone who has put in effort in the last few weeks to
> > > punt the last of the patches though the gate. It would not have been
> > doable
> > > without those hacking on LdapPool, doing test cleanup, and those
> > > reviewing/trying the code out.
> > >
> > > If you run across issues with Keystone and Python3, please let us know.
> > >
> > > A sincere thanks to the entire Keystone team involved in this multicycle
> > > effort.
> > >
> > > --Morgan
> >
> > Is this for unit or functional tests?
> >
> >
> Our unit tests and functional tests cross over significantly. The majority
> of keystone's unit tests are really "stand up a full keystone and test the
> API as though you were a client".
> 
> This is not DSVM and true "functional" tests (most of the restful test
> cases will move over to "functional" once it's working) are not fully setup
> yet.
> 
> In short "unit tests", except our unit tests are mostly "functional" in
> nature.
> 
> --Morgan

Ah, OK, I was wondering if you were using the python 3 features of
devstack and how well they were working for you.

Doug

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [openstack-operators][cinder] max_concurrent_builds in Cinder

2016-05-23 Thread Alex Meade

This sounds like a good idea to me. The queue doesn't handle this since we
just read everything off immediately anyways. I have seen issues where
customers have to write scripts that build 5 volumes, sleep, then build
more until they get >100 volumes. Just because a Cinder volume service will
clobber itself.

-Alex

On Mon, May 23, 2016 at 10:32 AM, Ivan Kolodyazhny  wrote:

> Hi developers and operators,
> I would like to get any feedback from you about my idea before I'll start
> work on spec.
>
> In Nova, we've got max_concurrent_builds option [1] to set 'Maximum number
> of instance builds to run concurrently' per each compute. There is no
> equivalent Cinder.
>
> Why do we need it for Cinder? IMO, it could help us to address following
> issues:
>
>- Creation of N volumes at the same time increases a lot of resource
>usage by cinder-volume service. Image caching feature [2] could help us a
>bit in case when we create volume form image. But we still have to upload N
>images to the volumes backend at the same time.
>- Deletion on N volumes at parallel. Usually, it's not very hard task
>for Cinder, but if you have to delete 100+ volumes at once, you can fit
>different issues with DB connections, CPU and memory usages. In case of
>LVM, it also could use 'dd' command to cleanup volumes.
>- It will be some kind of load balancing in HA mode: if cinder-volume
>process is busy with current operations, it will not catch message from
>RabbitMQ and other cinder-volume service will do it.
>- From users perspective, it seems that better way is to create/delete
>N volumes a bit slower than fail after X volumes were created/deleted.
>
>
> [1]
> https://github.com/openstack/nova/blob/283da2bbb74d0a131a66703516d16698a05817c7/nova/conf/compute.py#L161-L163
> [2]
> https://specs.openstack.org/openstack/cinder-specs/specs/liberty/image-volume-cache.html
>
> Regards,
> Ivan Kolodyazhny,
> http://blog.e0ne.info/
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Keystone] Welcome Keystone to the World of Python 3

On Mon, May 23, 2016 at 7:54 AM, Doug Hellmann 
wrote:

> Excerpts from Morgan Fainberg's message of 2016-05-20 20:58:00 -0700:
> > We've gone through all of our test cases and all of our code base. At
> this
> > point Keystone is no longer skipping any of the tests (which do tend to
> > test the entire request stack) and we are properly gating on being
> > Python3.4 compatible.
> >
> > I want to thank everyone who has put in effort in the last few weeks to
> > punt the last of the patches though the gate. It would not have been
> doable
> > without those hacking on LdapPool, doing test cleanup, and those
> > reviewing/trying the code out.
> >
> > If you run across issues with Keystone and Python3, please let us know.
> >
> > A sincere thanks to the entire Keystone team involved in this multicycle
> > effort.
> >
> > --Morgan
>
> Is this for unit or functional tests?
>
>
Our unit tests and functional tests cross over significantly. The majority
of keystone's unit tests are really "stand up a full keystone and test the
API as though you were a client".

This is not DSVM and true "functional" tests (most of the restful test
cases will move over to "functional" once it's working) are not fully setup
yet.

In short "unit tests", except our unit tests are mostly "functional" in
nature.

--Morgan


> Doug
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [craton] meeting notes

2016-05-23 Thread sean roberts

Hello project team members. We have one week until M1. We need to discuss
some details of what should be in the Newton release of Craton. Since no
one has commented on the project name, I am going to assume we all agree to
the name Craton. I will push the meeting patch to get it published.



On Mon, May 16, 2016 at 8:51 AM, sean roberts 
wrote:

> first team meeting notes here
> https://etherpad.openstack.org/p/craton-meeting-2016-05-16
>
> We have two weeks until M1, which is a good date to finalize what features
> we will have in the newton release.
>
> We proposed the project name Craton to stick. Let's propose an alternative
> here quickly if anyone wants.
>
> We agreed to hold meetbot IRC meetings 07:30 PDT. The #openstack-meeting-4
> slot is available.
>
> We agreed to move all email to openstack-dev@lists.openstack.org with the
> subject starting with [craton]
>
> The chat.freenode.net IRC channel #craton is up and running, but not
> logged yet.
>
> We want to target getting together face to face around the time of the yet
> to be scheduled Operators Mid-Cycle. 19 July Watcher mid-cycle in
> hillsboro, OR may be the area and time.
>
> I am happy to patch both the IRC meeting and craton logging patches, as
> soon as we decide.
>
> I would recommend that we decide on these items today or tomorrow, so we
> can get started on important project work.
>
> ~ sean
>



-- 

~ sean
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [craton] meeting notes

2016-05-23 Thread sean roberts

yes it is.

On Mon, May 16, 2016 at 9:36 AM, Tim Bell  wrote:

> Thanks for the notes.
>
>
>
> Is the agreement to work using Craton as a base for the fleet management ?
>
>
>
> Tim
>
>
>
> *From: *sean roberts 
> *Reply-To: *"OpenStack Development Mailing List (not for usage
> questions)" 
> *Date: *Monday 16 May 2016 at 17:51
> *To: *"openstack-dev@lists.openstack.org" <
> openstack-dev@lists.openstack.org>
> *Subject: *[openstack-dev] [craton] meeting notes
>
>
>
> first team meeting notes here
> https://etherpad.openstack.org/p/craton-meeting-2016-05-16
>
>
>
> We have two weeks until M1, which is a good date to finalize what features
> we will have in the newton release.
>
>
>
> We proposed the project name Craton to stick. Let's propose an alternative
> here quickly if anyone wants.
>
>
>
> We agreed to hold meetbot IRC meetings 07:30 PDT. The #openstack-meeting-4
> slot is available.
>
>
>
> We agreed to move all email to openstack-dev@lists.openstack.org with the
> subject starting with [craton]
>
>
>
> The chat.freenode.net IRC channel #craton is up and running, but not
> logged yet.
>
>
>
> We want to target getting together face to face around the time of the yet
> to be scheduled Operators Mid-Cycle. 19 July Watcher mid-cycle in
> hillsboro, OR may be the area and time.
>
>
>
> I am happy to patch both the IRC meeting and craton logging patches, as
> soon as we decide.
>
>
>
> I would recommend that we decide on these items today or tomorrow, so we
> can get started on important project work.
>
>
>
> ~ sean
>
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>


-- 

~ sean
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [new][ironic] ironic-python-agent 1.2.1 release (mitaka)

We are jazzed to announce the release of:

ironic-python-agent 1.2.1: Ironic Python Agent Ramdisk

This release is part of the mitaka stable release series.

For more details, please see below.

1.2.1
^


New Features


* The "has_carrier" flag was added to the network interface
  information.


Bug Fixes
*

* Fixes bug where the user specified disk_label is ignored with
  partition images in agent drivers.

* Inspection code now waits up to 1 minute for all NICs to get their
  IP addresses. Otherwise inspection fails for some users due to race
  between DIB DHCP code and IPA startup. See
  https://bugs.launchpad.net/bugs/1564954 for details.

* During inspection wait only for a PXE booting NIC to get its IP by
  default. Introduce a new "inspection_dhcp_all_interfaces" option to
  enable waiting for all interfaces instead.

* Stop checking the "has_carrier" field when waiting for NIC's to
  get IP addresses, as it might be set to "False" when the interface
  is being configured.

Changes in ironic-python-agent 1.2.0..1.2.1
---

f5b7ea2 Add script to install missing tinyipa dependencies
ed978f3 [inspection] wait for the PXE DHCP by default and remove the carrier 
check
aa9c061 Stop using git:// and be nice to people behind proxy servers
6ece00d Install qemu-image from backports repo
3fba1ee Wait for the interfaces to get IP addresses before inspection
88f33a3 Correct link to enabling agent drivers
237b76e Fix full_trusty_build once and for all
d3e112c Append BRANCH_PATH to filenames of build output
e949a1e Add disk_label support for partition images
8910634 Update .gitreview for stable/mitaka

Diffstat (except docs and test files)
-

.gitreview |   1 +
Dockerfile |   7 +-
imagebuild/coreos/full_trusty_build.sh |  24 -
imagebuild/tinyipa/Makefile|   8 +-
imagebuild/tinyipa/build-iso.sh|   2 +-
imagebuild/tinyipa/build-tinyipa.sh|   5 +-
imagebuild/tinyipa/finalise-tinyipa.sh |   2 +
imagebuild/tinyipa/install-deps.sh |  17 
ironic_python_agent/cmd/agent.py   |  15 +++
ironic_python_agent/extensions/standby.py  |   6 +-
ironic_python_agent/hardware.py|  20 +++-
ironic_python_agent/inspector.py   |  58 +++
.../notes/disk-label-fix-536897e41a4d817f.yaml |   5 +
.../inspection-wait-for-ips-223e39b65fef31bd.yaml  |   7 ++
...nspection-wait-for-ips-v2-146016f758d7010c.yaml |   8 ++
20 files changed, 366 insertions(+), 27 deletions(-)




__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [new][keystone] keystonemiddleware 4.4.1 release (mitaka)

We are pleased to announce the release of:

keystonemiddleware 4.4.1: Middleware for OpenStack Identity

This release is part of the mitaka stable release series.

With source available at:

http://git.openstack.org/cgit/openstack/keystonemiddleware

With package available at:

https://pypi.python.org/pypi/keystonemiddleware

Please report issues through launchpad:

http://bugs.launchpad.net/keystonemiddleware

For more details, please see below.

Changes in keystonemiddleware 4.4.0..4.4.1
--

0997d44 Updated from global requirements
b1c8fa9 Updated from global requirements
fa8ac82 Remove bandit.yaml in favor of defaults
c34f4cc Create signing_dir upon first usage

Diffstat (except docs and test files)
-

bandit.yaml| 119 -
keystonemiddleware/audit.py|  13 ++-
keystonemiddleware/auth_token/__init__.py  |   4 +-
keystonemiddleware/auth_token/_request.py  |   3 +-
keystonemiddleware/auth_token/_signing_dir.py  |  21 ++--
keystonemiddleware/echo/__main__.py|   3 +-
requirements.txt   |   2 +-
test-requirements.txt  |   2 +-
tox.ini|   7 +-
10 files changed, 56 insertions(+), 148 deletions(-)


Requirements updates


diff --git a/requirements.txt b/requirements.txt
index 0a9f2f3..aa2d0c1 100644
--- a/requirements.txt
+++ b/requirements.txt
@@ -14 +14 @@ pycadf!=2.0.0,>=1.1.0 # Apache-2.0
-python-keystoneclient!=1.8.0,!=2.1.0,>=1.6.0 # Apache-2.0
+python-keystoneclient!=1.8.0,!=2.1.0,<3.0.0,>=1.6.0 # Apache-2.0
diff --git a/test-requirements.txt b/test-requirements.txt
index 357d49e..35bb567 100644
--- a/test-requirements.txt
+++ b/test-requirements.txt
@@ -8 +8 @@ coverage>=3.6 # Apache-2.0
-fixtures>=1.3.1 # Apache-2.0/BSD
+fixtures<2.0,>=1.3.1 # Apache-2.0/BSD



__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [new][keystone] keystoneauth1 2.4.1 release (mitaka)

We are jazzed to announce the release of:

keystoneauth1 2.4.1: Authentication Library for OpenStack Identity

This release is part of the mitaka stable release series.

With source available at:

http://git.openstack.org/cgit/openstack/keystoneauth

With package available at:

https://pypi.python.org/pypi/keystoneauth1

Please report issues through launchpad:

http://bugs.launchpad.net/keystoneauth

For more details, please see below.

Changes in keystoneauth1 2.4.0..2.4.1
-

dd58541 Updated from global requirements
bc2354d Swap the order of username deprecation
73475f0 fix OrderedDict mutated during iteration

Diffstat (except docs and test files)
-

keystoneauth1/fixture/keystoneauth_betamax.py  | 2 +-
keystoneauth1/loading/_plugins/identity/generic.py | 5 ++---
keystoneauth1/loading/_plugins/identity/v2.py  | 7 +++
keystoneauth1/loading/_plugins/identity/v3.py  | 5 ++---
setup.cfg  | 2 +-
test-requirements.txt  | 2 +-
7 files changed, 11 insertions(+), 14 deletions(-)


Requirements updates


diff --git a/test-requirements.txt b/test-requirements.txt
index ad06c08..bf7f328 100644
--- a/test-requirements.txt
+++ b/test-requirements.txt
@@ -10 +10 @@ discover # BSD
-fixtures>=1.3.1 # Apache-2.0/BSD
+fixtures<2.0,>=1.3.1 # Apache-2.0/BSD



__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] Gerrit downtime on Friday 2016-06-03 at 20:00 UTC

2016-05-23 Thread Elizabeth K. Joseph

Hi everyone,

On Friday, June 3, from approximately 20:00 through 24:00 UTC Gerrit
will be unavailable while we rename some projects.

Currently, we plan on renaming the following projects:

openstack/openstack-ansible-ironic -> openstack/openstack-ansible-os_ironic

openstack-infra/ansible-puppet -> openstack-infra/ansible-role-puppet

This list is subject to change. If you need a rename, please be sure
to get your change in soon so we can review it and add it to
https://wiki.openstack.org/wiki/Meetings/InfraTeamMeeting#Upcoming_Project_Renames

Existing reviews, project watches, etc, should all be carried over.

If you have any questions about the maintenance, please reply here or
contact us in #openstack-infra on freenode.

-- 
Elizabeth Krumbach Joseph || Lyz || pleia2

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [Infra] Meeting Tuesday May 24th at 19:00 UTC

2016-05-23 Thread Elizabeth K. Joseph

Hi everyone,

The OpenStack Infrastructure (Infra) team is having our next weekly
meeting on Tuesday May 24th, at 19:00 UTC in #openstack-meeting

Meeting agenda available here:
https://wiki.openstack.org/wiki/Meetings/InfraTeamMeeting#Agenda_for_next_meeting

Anyone is welcome to to add agenda items and everyone interested in
the project infrastructure and process surrounding automated testing
and deployment is encouraged to attend.

In case you missed it or would like a refresher, the meeting minutes
and full logs from our last meeting are available:

Minutes: 
http://eavesdrop.openstack.org/meetings/infra/2016/infra.2016-05-17-19.01.html
Minutes (text):
http://eavesdrop.openstack.org/meetings/infra/2016/infra.2016-05-17-19.01.txt
Log: 
http://eavesdrop.openstack.org/meetings/infra/2016/infra.2016-05-17-19.01.log.html

-- 
Elizabeth Krumbach Joseph || Lyz || pleia2

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [new][neutron] neutron-lib 0.2.0 release (newton)

We are delighted to announce the release of:

neutron-lib 0.2.0: Neutron shared routines and utilities

This release is part of the newton release series.

With source available at:

http://git.openstack.org/cgit/openstack/neutron-lib

With package available at:

https://pypi.python.org/pypi/neutron-lib

Please report issues through launchpad:

http://bugs.launchpad.net/neutron

For more details, please see below.

Changes in neutron-lib 0.1.0..0.2.0
---

4dcbeb2 Updated from global requirements
d70cb29 Add IPv6_LLA_PREFIX constant
c09ff0b Remove ICMPV6_ALLOWED_TYPES
d33b62b Maintain ATTR_NOT_SPECIFIED constant with deepcopy
da9870c Add constants for additional ICMPv6 types
55498ff Define legacy ICMPv6 protocol name 'icmpv6' for backward compaty

Diffstat (except docs and test files)
-

neutron_lib/constants.py | 55 +---
requirements.txt |  4 ++--
2 files changed, 49 insertions(+), 10 deletions(-)


Requirements updates


diff --git a/requirements.txt b/requirements.txt
index e91e710..388462c 100644
--- a/requirements.txt
+++ b/requirements.txt
@@ -6 +6 @@ pbr>=1.6 # Apache-2.0
-Babel!=2.3.0,!=2.3.1,!=2.3.2,!=2.3.3,>=1.3 # BSD
+Babel>=2.3.4 # BSD
@@ -14 +14 @@ oslo.messaging>=4.5.0 # Apache-2.0
-oslo.service>=1.0.0 # Apache-2.0
+oslo.service>=1.10.0 # Apache-2.0



__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [puppet] weekly meeting #82

2016-05-23 Thread Emilien Macchi

Hi Puppeteers!

We'll have our weekly meeting tomorrow at 3pm UTC on
#openstack-meeting-4.

Here's a first agenda:
https://etherpad.openstack.org/p/puppet-openstack-weekly-meeting-20160524

Feel free to add more topics, and any outstanding bug and patch.

See you tomorrow!
Thanks,
-- 
Emilien Macchi

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [neutron] [networking-sfc] Adding use case to wiki?

2016-05-23 Thread Duarte Cardoso, Igor

Thanks Louis,

Best regards,
Igor.

From: Henry Fourie [mailto:louis.fou...@huawei.com]
Sent: Monday, May 23, 2016 5:59 PM
To: OpenStack Development Mailing List (not for usage questions) 

Subject: Re: [openstack-dev] [neutron] [networking-sfc] Adding use case to wiki?

Igor,
   Add them here https://wiki.openstack.org/wiki/Neutron/ServiceChainUseCases

-  Louis

From: Duarte Cardoso, Igor [mailto:igor.duarte.card...@intel.com]
Sent: Monday, May 23, 2016 9:14 AM
To: OpenStack Development Mailing List (not for usage questions)
Subject: [openstack-dev] [neutron] [networking-sfc] Adding use case to wiki?

Hi networking-sfc,

As discussed at the last meeting I was to add the SFC Encapsulation use case to 
the networking-sfc wiki:
#action paul, john igor to add use-cases to wiki

Where exactly is the place to insert use cases in the wiki? Is there one 
already?
None of these seem to have any section for that purpose:
https://wiki.openstack.org/wiki/Neutron/ServiceInsertionAndChaining
https://wiki.openstack.org/wiki/Meetings/ServiceFunctionChainingMeeting
https://wiki.openstack.org/wiki/Neutron/APIForServiceChaining

Best regards,
Igor.

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [mistral] Team meeting minues/log - 05/23/2016

2016-05-23 Thread Renat Akhmerov

Thanks for joining today’s meeting!

Meeting minutes and log:
http://eavesdrop.openstack.org/meetings/mistral/2016/mistral.2016-05-23-16.01.html
 

http://eavesdrop.openstack.org/meetings/mistral/2016/mistral.2016-05-23-16.01.log.html
 


The next meeting will be on May 30th.

Renat Akhmerov
@Nokia

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [neutron] [networking-sfc] Adding use case to wiki?

2016-05-23 Thread Henry Fourie

Igor,
   Add them here https://wiki.openstack.org/wiki/Neutron/ServiceChainUseCases

-Louis

From: Duarte Cardoso, Igor [mailto:igor.duarte.card...@intel.com]
Sent: Monday, May 23, 2016 9:14 AM
To: OpenStack Development Mailing List (not for usage questions)
Subject: [openstack-dev] [neutron] [networking-sfc] Adding use case to wiki?

Hi networking-sfc,

As discussed at the last meeting I was to add the SFC Encapsulation use case to 
the networking-sfc wiki:
#action paul, john igor to add use-cases to wiki

Where exactly is the place to insert use cases in the wiki? Is there one 
already?
None of these seem to have any section for that purpose:
https://wiki.openstack.org/wiki/Neutron/ServiceInsertionAndChaining
https://wiki.openstack.org/wiki/Meetings/ServiceFunctionChainingMeeting
https://wiki.openstack.org/wiki/Neutron/APIForServiceChaining

Best regards,
Igor.

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all][tc] Languages vs. Scope of "OpenStack"

Excerpts from Chris Dent's message of 2016-05-23 17:07:36 +0100:
> On Mon, 23 May 2016, Doug Hellmann wrote:
> > Excerpts from Chris Dent's message of 2016-05-20 14:16:15 +0100:
> >> I don't think language does (or should) have anything to do with it.
> >>
> >> The question is whether or not the tool (whether service or
> >> dependent library) is useful to and usable outside the openstack-stack.
> >> For example gnocchi is useful to openstack but you can use it with other
> >> stuff, therefore _not_ openstack. More controversially: swift can be
> >> usefully used all by its lonesome: _not_ openstack.
> >
> > Add keystone, cinder, and ironic to that list.
> 
> Hmmm. You can, but would people want to (that is, would it be a sound
> choice?)? Or _do_ people? Maybe that's the distinction? As far as I

Yes, I'm aware of cases of each of those projects being used without
"the rest" of OpenStack. I used keystone like that to secure some
internal APIs myself.

> understand things, it's not unusual for standalone swift
> installations to exist and it's something of an openstack joke that
> there are "lots of rules about how things are done, but swift is
> special". Something like gnocchi was designed from the outset to be
> separable.
> 
> I don't really know. I'm firmly in the camp that OpenStack needs to
> be smaller and more tightly focused if a unitary thing called OpenStack
> expects to be any good. So I'm curious about and interested in
> strategies for figuring out where the boundaries are.
> 
> So that, of course, leads back to the original question: Is OpenStack
> supposed to be a unitary.
> 
> If it's _not_, they yes, we need some fairly arbitrary (as in, it's
> okay if we pull them out of thin air without relation to the quality
> of the product, because there is no product!) but concrete guidelines
> that delineate in or out. We can just choose to choose them.
> 
> > As you say, there are a lot of good reasons to strive for loose
> > coupling between components. On the other hand, whether or not an
> > individual component can or should be used by itself isn't a
> > sufficient line when we're talking about the nature of OpenStack
> > as a whole, especially if we want interoperability between deployments.
> 
> If we want interoperability between deployments (is that yet
> universally agreed?) then optionality is a useful guideline for

It's the basis of much if not all of the DefCore trademark work.
So there are still some folks who don't like it, but it's an official
goal.

> whether something is OpenStack[1] or not. If it's optional, then it
> can't be part of OpenStack because if one install has it and another
> does not, then interoperability is shot.

Not quite. With the different trademark programs in place, the
stance is "if you're calling your X API 'OpenStack' it needs to run
our code and it needs to pass the interop tests for the related
capabilities." Each trademark program has a different list of
projects to which it applies.  So it's more prix fixe with an option
of the number of courses rather than a la carte (I may need lunch).

> 
> So that seems like a good second question: Do we all agree that the
> thing called OpenStack at cloud A is the same thing as OpenStack
> at cloud B?
> 
> (There's plenty of talk about this concept, but I'm pretty sure
> there isn't agreement. If we're not working towards real agreement,
> and we're not going to follow that agreement if we ever reach it,
> what are we doing?)

The DefCore team is working to establish the definition, and has a
well-defined process for documenting and modifying the definition over
time.

Doug

> 
> [1] Being useful-to or usable-by OpenStack is a whole 'nother deal.
> 

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Magnum][Octavia] Need help for LBaaS v2

2016-05-23 Thread Michael Johnson

Hi Wanghua,

From the o-cw log, it looks like the amphora service VM did not boot
properly or the network is not configured correctly.  We can see that
Nova said the VM went active, but the amphora-agent inside the image
never became reachable.  I would check the nova instance console log
to make sure the instance finished booting and check that the octavia
management network is properly setup such that the amphora-agent can
be reached.

Michael

On Sun, May 22, 2016 at 8:31 PM, 王华  wrote:
> Hi all,
>
> Previously Magnum used LBaaS v1. Now LBaaS v1 is deprecated, so we want to
> replace it by LBaaS v2. But I met a problem in my patch
> https://review.openstack.org/#/c/314060/. I could not figure out why it
> didn't work. It seems there are some errors in
> http://logs.openstack.org/60/314060/5/check/gate-functional-dsvm-magnum-k8s/6e3795e/logs/screen-o-cw.txt.gz.
> Can anyone in Octavia team help me to figure out why my patch doesn't work
> in the gate?  Thank you.
>
> Regards,
> Wanghua
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Freezer] Replace Gnu Tar with DAR

2016-05-23 Thread Fausto Marzi

If that would work, the only difference would probably be that when
restoring, the deleted files between levels are restore too. No difference
for incremental backups as --listed-incremental is still used. So when the
restore of that level fails, it is restarted without --incremental related
options. It is a possible workaround.

A possible solution would be to not wrap binaries any more and provide
similar functionalities from python code, so the code would also be more
portable. I'd rather invest time on binary dependency removal rather than
use other binaries. Just my opinion.


On Mon, May 23, 2016 at 6:03 PM, Dieterly, Deklan 
wrote:

> Then it would not be an incremental backup/restore. This problem arises
> when doing incremental backup and restores.
> --
> Deklan Dieterly
>
> Senior Systems Software Engineer
> HPE
>
>
>
>
> From:  Fausto Marzi 
> Reply-To:  "OpenStack Development Mailing List (not for usage questions)"
> 
> Date:  Wednesday, May 18, 2016 at 5:22 AM
> To:  "OpenStack Development Mailing List (not for usage questions)"
> 
> Subject:  Re: [openstack-dev] [Freezer] Replace Gnu Tar with DAR
>
>
> >Hi Deklan,
> >
> >what happen if the extract is executed without --listed-incremental or
> >--incremental options?
> >
> >
> >Does the issue still happen?
> >
> >
> >Thanks,
> >
> >Fausto
> >
> >
> >On Sat, May 14, 2016 at 12:56 AM, Dieterly, Deklan
> > wrote:
> >
> >When using incremental backups, tar will not handle removing a dir and
> >then renaming another dir to the removed dir.
> >
> >
> >dek@dek-HP-Z620-Workstation:~/backup-test$ tar --extract
> >--listed-incrementa=/dev/null --file backup.2.tar
> >tar: Cannot rename Œbackup/dir1¹ to Œbackup/dir2¹: Directory not empty
> >tar: Exiting with failure status due to previous errors
> >
> >
> >
> >Here are the steps to reproduce.
> >
> > 1845  mkdir backup
> > 1846  mkdir backup/dir1
> > 1847  mkdir backup/dir2
> > 1848  echo "aa" > backup/dir1/dir1-file1
> > 1849  echo "aa" > backup/dir2/dir2-file1
> > 1852  tar --create --file=backup.tar --listed-incremental=./listed-incr
> >backup
> > 1854  rm -rf backup/dir2
> > 1855  mv backup/dir1 backup/dir2
> > 1856  tar --create --file=backup.2.tar --listed-incremental=./listed-incr
> >backup
> > 1859  tar --extract --listed-incrementa=/dev/null --file backup.tar
> > 1861  tar --extract --listed-incrementa=/dev/null --file backup.2.tar
> >
> >
> >This seems to be a well known, long-standing issue with tar.
> >--
> >Deklan Dieterly
> >
> >Senior Systems Software Engineer
> >HPE
> >
> >
> >
> >
> >On 5/13/16, 4:33 PM, "Fox, Kevin M"  wrote:
> >
> >>Whats the issue?
> >>
> >>From: Dieterly, Deklan [deklan.diete...@hpe.com]
> >>Sent: Friday, May 13, 2016 3:07 PM
> >>To: openstack-dev@lists.openstack.org
> >>Subject: [openstack-dev] [Freezer] Replace Gnu Tar with DAR
> >>
> >>Does anybody see any issues if Freezer used DAR instead of Gnu Tar? DAR
> >>seems to handle a particular use case that Freezer has while Gnu Tar does
> >>not.
> >>--
> >>Deklan Dieterly
> >>
> >>Senior Systems Software Engineer
> >>HPE
> >>
> >>
> >>_
> >>_
> >>OpenStack Development Mailing List (not for usage questions)
> >>Unsubscribe:
> >openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> >
> >>http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> >>
> >>_
> >>_
> >>OpenStack Development Mailing List (not for usage questions)
> >>Unsubscribe:
> >openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> >
> >>http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> >
> >
> >__
> >OpenStack Development Mailing List (not for usage questions)
> >Unsubscribe:
> >openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> >
> >http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> >
> >
> >
> >
> >
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [neutron] [networking-sfc] Adding use case to wiki?

2016-05-23 Thread Duarte Cardoso, Igor

Hi networking-sfc,

As discussed at the last meeting I was to add the SFC Encapsulation use case to 
the networking-sfc wiki:
#action paul, john igor to add use-cases to wiki

Where exactly is the place to insert use cases in the wiki? Is there one 
already?
None of these seem to have any section for that purpose:
https://wiki.openstack.org/wiki/Neutron/ServiceInsertionAndChaining
https://wiki.openstack.org/wiki/Meetings/ServiceFunctionChainingMeeting
https://wiki.openstack.org/wiki/Neutron/APIForServiceChaining

Best regards,
Igor.

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [new][keystone] keystonemiddleware 4.5.1 release (newton)

We are eager to announce the release of:

keystonemiddleware 4.5.1: Middleware for OpenStack Identity

This release is part of the newton release series.

With source available at:

http://git.openstack.org/cgit/openstack/keystonemiddleware

With package available at:

https://pypi.python.org/pypi/keystonemiddleware

Please report issues through launchpad:

http://bugs.launchpad.net/keystonemiddleware

For more details, please see below.

Changes in keystonemiddleware 4.5.0..4.5.1
--

f55b033 Fix AttributeError on cached-invalid token checks

Diffstat (except docs and test files)
-

keystonemiddleware/auth_token/__init__.py  |  2 +-
.../unit/auth_token/test_auth_token_middleware.py  | 18 ++
2 files changed, 19 insertions(+), 1 deletion(-)




__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all][tc] Languages vs. Scope of "OpenStack"

2016-05-23 Thread Chris Dent


On Mon, 23 May 2016, Doug Hellmann wrote:

Excerpts from Chris Dent's message of 2016-05-20 14:16:15 +0100:

I don't think language does (or should) have anything to do with it.

The question is whether or not the tool (whether service or
dependent library) is useful to and usable outside the openstack-stack.
For example gnocchi is useful to openstack but you can use it with other
stuff, therefore _not_ openstack. More controversially: swift can be
usefully used all by its lonesome: _not_ openstack.


Add keystone, cinder, and ironic to that list.


Hmmm. You can, but would people want to (that is, would it be a sound
choice?)? Or _do_ people? Maybe that's the distinction? As far as I
understand things, it's not unusual for standalone swift
installations to exist and it's something of an openstack joke that
there are "lots of rules about how things are done, but swift is
special". Something like gnocchi was designed from the outset to be
separable.

I don't really know. I'm firmly in the camp that OpenStack needs to
be smaller and more tightly focused if a unitary thing called OpenStack
expects to be any good. So I'm curious about and interested in
strategies for figuring out where the boundaries are.

So that, of course, leads back to the original question: Is OpenStack
supposed to be a unitary.

If it's _not_, they yes, we need some fairly arbitrary (as in, it's
okay if we pull them out of thin air without relation to the quality
of the product, because there is no product!) but concrete guidelines
that delineate in or out. We can just choose to choose them.


As you say, there are a lot of good reasons to strive for loose
coupling between components. On the other hand, whether or not an
individual component can or should be used by itself isn't a
sufficient line when we're talking about the nature of OpenStack
as a whole, especially if we want interoperability between deployments.


If we want interoperability between deployments (is that yet
universally agreed?) then optionality is a useful guideline for
whether something is OpenStack[1] or not. If it's optional, then it
can't be part of OpenStack because if one install has it and another
does not, then interoperability is shot.

So that seems like a good second question: Do we all agree that the
thing called OpenStack at cloud A is the same thing as OpenStack
at cloud B?

(There's plenty of talk about this concept, but I'm pretty sure
there isn't agreement. If we're not working towards real agreement,
and we're not going to follow that agreement if we ever reach it,
what are we doing?)

[1] Being useful-to or usable-by OpenStack is a whole 'nother deal.

--
Chris Dent   (╯°□°)╯︵┻━┻http://anticdent.org/
freenode: cdent tw: @anticdent__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Freezer] Replace Gnu Tar with DAR

2016-05-23 Thread Dieterly, Deklan

Then it would not be an incremental backup/restore. This problem arises
when doing incremental backup and restores.
-- 
Deklan Dieterly

Senior Systems Software Engineer
HPE




From:  Fausto Marzi 
Reply-To:  "OpenStack Development Mailing List (not for usage questions)"

Date:  Wednesday, May 18, 2016 at 5:22 AM
To:  "OpenStack Development Mailing List (not for usage questions)"

Subject:  Re: [openstack-dev] [Freezer] Replace Gnu Tar with DAR


>Hi Deklan,
>
>what happen if the extract is executed without --listed-incremental or
>--incremental options?
>
>
>Does the issue still happen?
>
>
>Thanks,
>
>Fausto
>
>
>On Sat, May 14, 2016 at 12:56 AM, Dieterly, Deklan
> wrote:
>
>When using incremental backups, tar will not handle removing a dir and
>then renaming another dir to the removed dir.
>
>
>dek@dek-HP-Z620-Workstation:~/backup-test$ tar --extract
>--listed-incrementa=/dev/null --file backup.2.tar
>tar: Cannot rename Œbackup/dir1¹ to Œbackup/dir2¹: Directory not empty
>tar: Exiting with failure status due to previous errors
>
>
>
>Here are the steps to reproduce.
>
> 1845  mkdir backup
> 1846  mkdir backup/dir1
> 1847  mkdir backup/dir2
> 1848  echo "aa" > backup/dir1/dir1-file1
> 1849  echo "aa" > backup/dir2/dir2-file1
> 1852  tar --create --file=backup.tar --listed-incremental=./listed-incr
>backup
> 1854  rm -rf backup/dir2
> 1855  mv backup/dir1 backup/dir2
> 1856  tar --create --file=backup.2.tar --listed-incremental=./listed-incr
>backup
> 1859  tar --extract --listed-incrementa=/dev/null --file backup.tar
> 1861  tar --extract --listed-incrementa=/dev/null --file backup.2.tar
>
>
>This seems to be a well known, long-standing issue with tar.
>--
>Deklan Dieterly
>
>Senior Systems Software Engineer
>HPE
>
>
>
>
>On 5/13/16, 4:33 PM, "Fox, Kevin M"  wrote:
>
>>Whats the issue?
>>
>>From: Dieterly, Deklan [deklan.diete...@hpe.com]
>>Sent: Friday, May 13, 2016 3:07 PM
>>To: openstack-dev@lists.openstack.org
>>Subject: [openstack-dev] [Freezer] Replace Gnu Tar with DAR
>>
>>Does anybody see any issues if Freezer used DAR instead of Gnu Tar? DAR
>>seems to handle a particular use case that Freezer has while Gnu Tar does
>>not.
>>--
>>Deklan Dieterly
>>
>>Senior Systems Software Engineer
>>HPE
>>
>>
>>_
>>_
>>OpenStack Development Mailing List (not for usage questions)
>>Unsubscribe: 
>openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
>
>>http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>>_
>>_
>>OpenStack Development Mailing List (not for usage questions)
>>Unsubscribe: 
>openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
>
>>http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
>__
>OpenStack Development Mailing List (not for usage questions)
>Unsubscribe: 
>openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
>
>http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
>
>
>

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [magnum][kuryr] Nested containers networking

2016-05-23 Thread Hongbin Lu

Hi Kuryr team,

I want to start this ML to sync up the latest status of the nested container 
networking implementation. Could I know who is implementing this feature in 
Kuryr side and how Magnum team could help in this efforts? In addition, I 
wonder if it makes sense to establish cross-project liaisons between Kuryr and 
Magnum. Magnum relies on Kuryr to implement several important features so I 
think it is helpful to setup a communication channel between both teams. 
Thoughts?

Best regards,
Hongbin
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [javascript] Seeking contributors, js-generator-openstack

2016-05-23 Thread Michael Krotscheck

Good idea. Since we've already got a storyboard project, let's start there.

Michael

On Fri, May 20, 2016 at 4:02 PM Zhang Yujun 
wrote:

> Hi, Michael
>
> As you are no longer alone now, we'd better to put things in your head
> onto documents so that everybody who wish to contribute will know where to
> go.
>
> Besides the technical roadmap, I think we shall need a space for issue
> tracking and proposal discussion. After we make the project more open to
> the community, it won't be long that more developers join this project.
>
> That's my basic thoughts for the moment.
>
> --
> Yujun
>
> On Sat, May 21, 2016 at 1:10 AM Michael Krotscheck 
> wrote:
>
>> Hi there!
>>
>> Well, the first thing we need is other reviewers, which is the fastest
>> way to become a core :). The project page right now is the README.md
>> file in the project itself. The main reason for this is that the target
>> audience - javascript engineers - usually find that first via NPM. Most of
>> the Todo items there have already been done, actually, so the next step
>> would be to really identify what this project needs to accomplish, group it
>> into major categories, and start working on it. Off the top of my head,
>> here's a list:
>>
>>
>>1. Dependency synchronization: Keep a list of semver
>>global-dependencies.json at the root of the project, and update a 
>> project's
>>dependencies if the versions are out of sync.
>>2. Eslint invocation. Infra's Common Testing Interface states that
>>all javascript projects must support 'npm run lint', using
>>eslint-config-openstack. The generator should add/update this to any
>>project it's run in.
>>3. nsp invocation. Not strictly necessary, but a postinstall scan of
>>the project for publicly known vulnerabilities is always a good thing.
>>
>> After these pieces, the next step becomes more complicated, as we need to
>> choose whether the user is creating a web application, or a node
>> application. This then allows us to switch out which test harness and
>> runner we're using, so that the `npm test` command can be consistent. Once
>> this lands, we can start talking about project src/dist directories, how to
>> best use gulp in each project type, and actual project templates :).
>>
>> Is there something in particular you'd like to work on?
>>
>> Michael
>>
>>
>> On Thu, May 19, 2016 at 12:39 AM Zhang Yujun 
>> wrote:
>>
>>> Hi, Michael,
>>>
>>> I have several project experience in JavaScript and please let me know
>>> how I could help on this project?
>>>
>>> Is there a project page?
>>>
>>> Or we shall getting started with gerrit review?
>>>
>>> --
>>> Yujun
>>>
>>> On Wed, May 18, 2016 at 11:45 PM Michael Krotscheck <
>>> krotsch...@gmail.com> wrote:
>>>
 Hello everyone!

 The js-generator-openstack project has been incubated under
 openstack-infra, and is seeking contributors (and cores). The purpose of
 the project is as follows:

- Help manage common project configuration aspects, such as
licenses, gerrit, authors, and more.
- Assist in keeping dependencies up-to-date and synchronized across
javascript projects (JS equivalent of global requirements).
- Provide all the necessary hooks for OpenStack's JavaScript Common
Testing Interface.
- Suggest common tools to use for tasks such as linting, unit
testing, functional testing, and more.
- (Newton Stretch) Provide a quick way of bootstrapping a new
CORS-consuming OpenStack UI.

 I'm looking for help- firstly, because right now I'm the only person
 who's willing to review JavaScript amongst the various infra cores, and I'd
 really like more eyeballs on this project. Secondly, because I know that
 I'm not the only person who has opinions about how we should be doing
 JavaScript things.

 Come on over to
 https://review.openstack.org/#/q/project:openstack-infra/js-generator-openstack+status:open
  and
 help me out, would ya? If you've got questions, I'm active in the
 #openstack-javascript channel.

 Michael

>>>
 __
 OpenStack Development Mailing List (not for usage questions)
 Unsubscribe:
 openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

>>>
>>> __
>>> OpenStack Development Mailing List (not for usage questions)
>>> Unsubscribe:
>>> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>
>> __
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe:
>>

[openstack-dev] [magnum] FW: Mentors needed in specific technical areas

2016-05-23 Thread Hongbin Lu

FYI,

If you interest to be a mentor for containers or other areas, check below...

Best regards,
Hongbin

From: Emily K Hugenbruch [mailto:ekhugenbr...@us.ibm.com]
Sent: May-23-16 10:25 AM
To: openstack-dev@lists.openstack.org
Subject: [openstack-dev] [PTLs][all][mentoring] Mentors needed in specific 
technical areas

Hi,
The lightweight mentoring program sponsored by the Women of OpenStack has 
really taken off, and we have about 35 mentees looking for technical help that 
we don't have mentors for. We're asking for help from the PTLs to announce the 
mentoring program in team meetings then direct people to the guidelines 
(here
 and 
here)
 and signup form if 
they're interested.

Mentors should be regular contributors to a project, with an interest in 
helping new people and about 4 hours a month for mentoring. They do not have to 
be women; the program is just sponsored by WoO, we welcome all mentees and 
mentors.

These are the projects/areas where we especially need mentors:

 *   Cinder
 *   Containers
 *   Documentation
 *   Glance
 *   Keystone
 *   Murano
 *   Neutron
 *   Nova
 *   Ops
 *   Searchlight
 *   Telemetry
 *   TripleO
 *   Trove
If you have any questions you can contact me, or ask on openstack-women where 
the mentoring committee hangs out.
Thanks!
Emily Hugenbruch
IRC: ekhugen
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all][tc] Languages vs. Scope of "OpenStack"

Excerpts from Chris Dent's message of 2016-05-20 14:16:15 +0100:
> On Fri, 20 May 2016, Thierry Carrez wrote:
> 
> > The other approach is product-centric: "lower-level pieces are OpenStack 
> > dependencies, rather than OpenStack itself". If we are missing a 
> > lower-level 
> > piece to achieve our mission and are developing it as a result, it could be 
> > developed on OpenStack infrastructure by members of the OpenStack community 
> > but it is not "OpenStack the product", it's an OpenStack *dependency*. It 
> > is 
> > not governed by the TC, it can use any language and tool deemed necessary.
> >
> > On this second approach, there is the obvious question of where 
> > "lower-level" 
> > starts, which as you explained above is not really clear-cut. A good litmus 
> > test for it could be whenever Python is not enough. If you can't develop it 
> > effectively with the language that is currently sufficient for the rest of 
> > OpenStack, then developing it as an OpenStack dependency in whatever 
> > language 
> > is appropriate might be the solution...
> 
> I don't think language does (or should) have anything to do with it.
> 
> The question is whether or not the tool (whether service or
> dependent library) is useful to and usable outside the openstack-stack.
> For example gnocchi is useful to openstack but you can use it with other
> stuff, therefore _not_ openstack. More controversially: swift can be
> usefully used all by its lonesome: _not_ openstack.

Add keystone, cinder, and ironic to that list.

> Not being in OpenStack (where "in" means "of the product") is good
> for OpenStack, good for the project and good for opensource in general:
> 
> * Outside the OpenStack bubble, looking in, one can see a bunch of
>complexity and a bunch of bad architecture decisions but rarely
>see the good stuff that is actually there, so it is easy enough to walk
>away. Good stuff that a larger audience could benefit from may get
>dismissed, if that good stuff has an opportunity to have an
>independent identity, it can be useful.
> 
> * A project that is used by a larger and more diverse audience
>(people-wise and technology-wise) will of necessity be more
>robust.
> 
> * A project that defines itself as independent will be required to
>have strong and narrow contracts to satisfy its diverse audiences.
> 
> * A project that has those strong and narrow contracts can use what
>ever language it likes and still be useful and nobody really needs
>to care all that deeply except for the people making it. If they
>want to be in a language that infra doesn't want to support,
>that's fine: there are plenty of other ways to do CI.
> 
> * An openstack which is more narrow is far easier for people to
>comprehend and contemplate.
> 
> * A broad opensource world that has lots of nice things is a better
>one.

As you say, there are a lot of good reasons to strive for loose
coupling between components. On the other hand, whether or not an
individual component can or should be used by itself isn't a
sufficient line when we're talking about the nature of OpenStack
as a whole, especially if we want interoperability between deployments.

Most of our services add value on top of some other existing project
(either by integrating them or abstracting them or both).  What
would that story look like if we said that because keystone can be
used by itself, it is not part of OpenStack? Would it make sense
to still require deployments that want to use the trademark to use
keystone for authentication?

Doug

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [kubernetes]

2016-05-23 Thread Fox, Kevin M

+1 for using k8s to do work where possible.

-1 for trying to shoehorn a feature in so that k8s can deal with stuff its not 
ready to handle. We need to ensure Operators have everything they need in order 
to successfully operate their cloud.

The current upgrade stuff in k8s is focused around replacing one, usually 
stateless, thing for another. It never had Database Schema upgrades in mind.  
It is great to use for minor version bumps. It is insufficient for major 
OpenStack upgrades. If you follow the OpenStack release notes, they tend to be 
quite linear, in a workflow. K8s isn't designed for that. Hence the need for a 
tool outside of k8s to drive the creation/upgrading of Deployments and Jobs in 
the proper order.

Init containers also do not look like a good fit. As far as I can gather from 
the spec, they are intended to init something on a node when a pod is spawned. 
This is a very different thing from upgrading a shared database's schema. I 
don't believe they should be used for that.

I've upgraded many OpenStack clouds over the years. One of the things that has 
bit me from time to time is a failed schema update and having to tweak code and 
then rerun schema upgrades. This will continue to happen and needs to be 
covered. The Job's workflow as discussed in the spec allows an operator to do 
just that. Hiding it in an init container makes that much harder for Operators.

As an Op, we need the ability to tweak the workflow as needed and run/rerun 
only the pieces that we need.

Thanks,
Kevin

From: Ryan Hallisey [rhall...@redhat.com]
Sent: Sunday, May 22, 2016 12:50 PM
To: OpenStack Development Mailing List (not for usage questions)
Subject: [openstack-dev]  [kolla][kolla-kubernetes][kubernetes]

Hi all,

At the Kolla meeting last week, I brought up some of the challenges around the 
bootstrapping
process in Kubernetes.  The main highlight revolved around how the 
bootstrapping process will
work.

Currently, in the kolla-kubernetes spec [1], the process for bootstrapping 
involves
outside orchestration running Kubernetes 'Jobs' that will handle the database 
initialization,
creating users, etc...  One of the flaws in this approach, is that 
kolla-kubernetes can't use
native Kubernetes upgrade tooling. Kubernetes does upgrades as a single action 
that scales
down running containers and replaces them with the upgraded containers. So 
instead of having
Kubernetes manage the upgrade, it would be guided by an external engine.  Not 
saying this is
a bad thing, but it does loosen the control Kubernetes would have over stack 
management.

Kubernetes does have some incoming new features that are a step in the right 
direction to
allow for kolla-kubernetes to make complete use of Kubernetes tooling like init 
containers [2].
There is also the introduction to wait.for conditions in the kubectl [3].

   kubectl get pod my-pod --wait --wait-for="pod-running"

Upgrades will be in the distant future for kolla-kubernetes, but I want to make 
sure the
community maintains an open mind about bootstrap/upgrades since there are 
potentially many
options that could come down the road.

I encourage everyone to add your input to the spec!

Thanks,
Ryan

[1] SPEC - https://review.openstack.org/#/c/304182/
[2] Init containers - https://github.com/kubernetes/kubernetes/pull/23567
[3] wait.for kubectl - https://github.com/kubernetes/kubernetes/issues/1899

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [all][glare][api] How's going to use Glare?

2016-05-23 Thread Mikhail Fedosin

Hello! Some time ago we created new service called Glare that brings
support of artifacts in OpenStack. It is based on Glance v2 API and adapts
it to the possibility of using different artifact types, like containers,
various templates, application packages, but not just VM images only. You
can check these slides
https://docs.google.com/presentation/d/1WQoBenlp-0vD1t7mpPgQuepDmlPUXq2LOfRYnurZx74/edit#slide=id.p
and this presentation https://www.youtube.com/watch?v=XgpEdycRp9Y to learn
more about Glare and its mission.

We have been developing stable Glare version since February and today I'm
happy to present you the final specification of v1 API
https://review.openstack.org/#/c/283136/. Currently we work with Heat,
App-Catalog and Murano teams to implement features they need, but also I'll
be glad to get some feedback from other projects which are interested in
using Glare and catalogization their binaries.

Also it will be awesome if you can review the spec and leave your opinions
as well (it's kind of huge, but it's only because there are a lot of use
cases and examples of the API). There we tried to satisfy all API-WG,
DefCore and a common sense requirements to make this API as much RESTful as
possible.

Thanks in advance!

Best regards,
Mikhail Fedosin
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [horizon] [antiddos] New project antiddos

Excerpts from adriano fialho araujo's message of 2016-05-20 11:45:09 -0300:
> Hello, fellow developers!
> 
> i need to create a new project to sell antiDDoS the horizon . what do you
> recommend me as the first start of the project?
> Does anyone know any documentation to start a project in django on
> OpenStack standard.
> 
> thank you

The instructions for setting up the infrastructure for new projects is
in the "Project Creator's Guide" section of the Infra Manual. See
http://docs.openstack.org/infra/manual/creators.html

Doug

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [nova] next notification subteam meeting

2016-05-23 Thread Balázs Gibizer

Hi, 

The next notification subteam meeting will be held on 2016.05.24 17:00 UTC [1] 
on #openstack-meeting-4.

Cheers,
Gibi

[1] https://www.timeanddate.com/worldclock/fixedtime.html?iso=20160524T17 

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Keystone] Welcome Keystone to the World of Python 3

Excerpts from Morgan Fainberg's message of 2016-05-20 20:58:00 -0700:
> We've gone through all of our test cases and all of our code base. At this
> point Keystone is no longer skipping any of the tests (which do tend to
> test the entire request stack) and we are properly gating on being
> Python3.4 compatible.
> 
> I want to thank everyone who has put in effort in the last few weeks to
> punt the last of the patches though the gate. It would not have been doable
> without those hacking on LdapPool, doing test cleanup, and those
> reviewing/trying the code out.
> 
> If you run across issues with Keystone and Python3, please let us know.
> 
> A sincere thanks to the entire Keystone team involved in this multicycle
> effort.
> 
> --Morgan

Is this for unit or functional tests?

Doug

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [openstack-operators][cinder] max_concurrent_builds in Cinder

2016-05-23 Thread Ivan Kolodyazhny

Hi developers and operators,
I would like to get any feedback from you about my idea before I'll start
work on spec.

In Nova, we've got max_concurrent_builds option [1] to set 'Maximum number
of instance builds to run concurrently' per each compute. There is no
equivalent Cinder.

Why do we need it for Cinder? IMO, it could help us to address following
issues:

- Creation of N volumes at the same time increases a lot of resource
usage by cinder-volume service. Image caching feature [2] could help us a
bit in case when we create volume form image. But we still have to upload N
images to the volumes backend at the same time.
- Deletion on N volumes at parallel. Usually, it's not very hard task
for Cinder, but if you have to delete 100+ volumes at once, you can fit
different issues with DB connections, CPU and memory usages. In case of
LVM, it also could use 'dd' command to cleanup volumes.
- It will be some kind of load balancing in HA mode: if cinder-volume
process is busy with current operations, it will not catch message from
RabbitMQ and other cinder-volume service will do it.
- From users perspective, it seems that better way is to create/delete N
volumes a bit slower than fail after X volumes were created/deleted.

[1]
https://github.com/openstack/nova/blob/283da2bbb74d0a131a66703516d16698a05817c7/nova/conf/compute.py#L161-L163
[2]
https://specs.openstack.org/openstack/cinder-specs/specs/liberty/image-volume-cache.html

Regards,
Ivan Kolodyazhny,
http://blog.e0ne.info/
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [PTLs][all][mentoring] Mentors needed in specific technical areas

2016-05-23 Thread Emily K Hugenbruch



Hi,
The lightweight mentoring program sponsored by the Women of OpenStack has
really taken off, and we have about 35 mentees looking for technical help
that we don't have mentors for.  We're asking for help from the PTLs to
announce the mentoring program in team meetings then direct people to the
guidelines (here and here) and signup form if they're interested.

Mentors should be regular contributors to a project, with an interest in
helping new people and about 4 hours a month for mentoring.  They do not
have to be women; the program is just sponsored by WoO, we welcome all
mentees and mentors.

These are the projects/areas where we especially need mentors:
  Cinder
  Containers
  Documentation
  Glance
  Keystone
  Murano
  Neutron
  Nova
  Ops
  Searchlight
  Telemetry
  TripleO
  Trove

If you have any questions you can contact me, or ask on openstack-women
where the mentoring committee hangs out.
Thanks!
Emily Hugenbruch
IRC: ekhugen
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [glance] [defcore] [interop] Proposal for a virtual sync dedicated to Import Refactor May 26th