Re: [openstack-dev] [Nova] live migration sub-team meeting

2015-11-10 Thread Qiao,Liyong

hi Paul

+1, will attend for this meeting.

On 2015年11月10日 19:48, Murray, Paul (HP Cloud) wrote:


Thank you for the prompting Michael. I was chasing up a couple of key 
people to make sure they were available.


The IRC meeting should be Tuesdays at 1400 UTC on #openstack-meeting-3 
starting next week (too late for today).


I will get that sorted out with infra and send another email to 
confirm. I will also sort out all the specs and patches that I know 
about today. More information will be included about that too.


Paul

*From:*Michael Still [mailto:mi...@stillhq.com]
*Sent:* 09 November 2015 21:34
*To:* OpenStack Development Mailing List (not for usage questions)
*Subject:* Re: [openstack-dev] [Nova] live migration sub-team meeting

So, its been a week. What time are we picking?

Michael

On Thu, Nov 5, 2015 at 10:46 PM, Murray, Paul (HP Cloud) 
mailto:pmur...@hpe.com>> wrote:


> > Most team members expressed they would like a regular IRC
meeting for
> > tracking work and raising blocking issues. Looking at the
contributors
> > here [2], most of the participants seem to be in the European
> > continent (in time zones ranging from UTC to UTC+3) with a few
in the
> > US (please correct me if I am wrong). That suggests that a
time around
> > 1500 UTC makes sense.
> >
> > I would like to invite suggestions for a day and time for a weekly
> > meeting -
>
> Maybe you could create a quick Doodle poll to reach a rough
consensus on
> day/time:
>
> http://doodle.com/

Yes, of course, here's the poll:

http://doodle.com/poll/rbta6n3qsrzcqfbn




__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe:
openstack-dev-requ...@lists.openstack.org?subject:unsubscribe

http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



--

Rackspace Australia



__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


--
BR, Eli(Li Yong)Qiao

<>__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Neutron][IPAM] ML2 don't call ipam driver remove_subnet function

2015-11-10 Thread Gary Kotton
Hi,
This should be resolved with this path - https://review.openstack.org/239885
Good luck
Gary

From: thanh le giang mailto:legiangth...@gmail.com>>
Reply-To: OpenStack List 
mailto:openstack-dev@lists.openstack.org>>
Date: Wednesday, November 11, 2015 at 5:20 AM
To: OpenStack List 
mailto:openstack-dev@lists.openstack.org>>
Subject: [openstack-dev] [Neutron][IPAM] ML2 don't call ipam driver 
remove_subnet function

Dear folks

I have met a problem when implement IPAM driver, ML2 doesn't call remove_subnet 
function of ipam driver because ML2 doesn't use delete_subnet function of 
NeutronDbPluginV2

Now I workaround by using "SUBNET BEFORE_DELETE" event to notify external IPAM 
that subnet will be deleted, but I think it's not a good solution. According to 
my understanding this event should be used for checking subnet is in-used or 
not. I think ML2 should call IPAM driver for this situation or provide 
additional "SUBNET AFTER_DELETE" event

Thanks,
Thanh

Email: legiangt...@gmail.com
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] Introducing Lwood - a weekly summary of openstack-dev traffic

2015-11-10 Thread Hugh Blemings

On 10/11/2015 19:50, Thierry Carrez wrote:

Hugh Blemings wrote:

[...]

 http://hugh.blemings.id.au/category/Lwood/feed/


You should federate that one to Planet OpenStack!

See instructions at:
https://wiki.openstack.org/wiki/AddingYourBlog


Thanks for the reminder - have done so now (assuming my git-fu is still 
ok :)


Cheers,
Hugh



__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


[openstack-dev] [HA][RabbitMQ][messaging][Pacemaker][operators] Improved OCF resource agent for dynamic active-active mirrored clustering

2015-11-10 Thread bdobrelia
Thank you Andrew.
Answers below.
>>>
Sounds interesting, can you give any comment about how it differs to the 
other[i] upstream agent?
Am I right that this one is effectively A/P and wont function without some kind 
of shared storage?
Any particular reason you went down this path instead of full A/A?

[i] 
https://github.com/ClusterLabs/resource-agents/blob/master/heartbeat/rabbitmq-cluster
<<<
It is based on multistate clone notifications. It requries nothing shared but 
Corosync info base CIB where all Pacemaker resources stored anyway. And it is 
fully A/A. All running rabbit nodes may process AMQP connections. Master state 
is only for a cluster initial point at wich other slaves may join to it.
Note, here you can find events flow charts as well [0]
[0] https://www.rabbitmq.com/pacemaker.html


Regards,

Bogdan__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Fuel][Fuel-QA][Fuel-TechDebt] Code Quality: Do Not Hardcode - Fix Things Instead

2015-11-10 Thread Aleksandr Didenko
+1 from me

On Tue, Nov 10, 2015 at 6:38 PM, Stanislaw Bogatkin 
wrote:

> I think that it is excellent thought.
> +1
>
> On Tue, Nov 10, 2015 at 6:52 PM, Vladimir Kuklin 
> wrote:
>
>> Folks
>>
>> I wanted to raise awareness about one of the things I captured while
>> doing reviews recently - we are sacrificing quality to bugfixing and
>> feature development velocity, essentially moving from one heap to another -
>> from bugs/features to 'tech-debt' bugs.
>>
>> I understand that we all have deadlines and need to meet them. But,
>> folks, let's create the following policy:
>>
>> 1) do not introduce hacks/workarounds/kludges if it is possible.
>> 2) while fixing things if you have a hack/workaround/kludge that you need
>> to work with - think of removing it instead of enhancing and extending it.
>> If it is possible - fix it. Do not let our technical debt grow.
>> 3) if there is no way to avoid kludge addition/enhancing, if there is no
>> way to remove it - please, add a 'TODO/FIXME' line above it, so that we can
>> collect them in the future and fix them gradually.
>>
>> I suggest to add this requirement into code-review policy.
>>
>> What do you think about this?
>>
>> --
>> Yours Faithfully,
>> Vladimir Kuklin,
>> Fuel Library Tech Lead,
>> Mirantis, Inc.
>> +7 (495) 640-49-04
>> +7 (926) 702-39-68
>> Skype kuklinvv
>> 35bk3, Vorontsovskaya Str.
>> Moscow, Russia,
>> www.mirantis.com 
>> www.mirantis.ru
>> vkuk...@mirantis.com
>>
>> __
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe:
>> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>>
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [nova][cinder] About rebuilding volume-backed instances.

2015-11-10 Thread Clint Byrum
Excerpts from Zhenyu Zheng's message of 2015-11-09 00:21:23 -0800:
> Hi,
> 
> Thanks for the reply, for some scenario, launching an new instance is
> easier. But for production deployment, an instance may contain a large
> number of data such as keypairs, metadata, bdm etc. and it may have
> multiple internet interfaces that are connected to multiple networks. That
> is to say, for operations like recovery, change/update operating system, to
> build an new instance is a lot more "expensive" than rebuild it. And one
> instance may have some volumes that are marked as delete_on_terminate =
> True, if that is the situation, build an new instance will not save the
> user data in user volume, but rebuild can protect them.
> 
> So, I think this is a quite reasonable demand for OpenStack.
>

I completely respect the fact that you have real users and they are
asking for this. I am sure you're doing what you can to convince them to
build cloud-ready applications.

Unfortunately, you're trying to work around misuse of the cloud API's,
not missing features from them. Don't use those volume types, and don't
build systems that rely on single ports and interfaces. IMO rebuild is a
misguided concept (something that took me a long time to realize). It
requires one service (Nova) to take all of the responsibility for
cross-service reinitialization and that doesn't happen so you get weird
disconnects like this one. Heat would be a better choice, as you can
simply deploy a new template, which will replace the old instance in
a relatively safe way including detaching and re-attaching volumes and
any VIP's.

So, to be clear:

*DO* build systems that store data on volumes that are _not_ the root
disk. Boot a new instance, initialize the configuration using your
tools, and then move the volume attachment from the old to the new.

*DO* build systems that use DNS or a VIP to communicate so that new
ports can be allocated and attached to the new instance while the old
one is stil active.

> On Mon, Nov 9, 2015 at 3:28 PM, Clint Byrum  wrote:
> 
> > Excerpts from Zhenyu Zheng's message of 2015-11-08 23:04:59 -0800:
> > > Hi All,
> > >
> > > Currently, we have strong demands about "rebuilding"(or actions like
> > > rebuilding) volume-backed instances. As in production deployment, volume
> > > backed instance is widely used. Users have the demands of performing the
> > > rebuild(recovery) action for root device while maintain instance UUID
> > sorts
> > > of information, many users also wants to keep the volume uuid unchanged.
> > >
> > > Nova side doesn't support using Rebuild API directly for volume backed
> > > instances (the volume will not change). And Nova side also doesn't
> > support
> > > detaching root device, that means we cannot performing volume
> > > backup/restore from cinder side, because those actions needs the volume
> > in
> > > "available" status.
> > >
> > > Now there are couple of patches proposed in nova trying to fix this
> > problem:
> > > [1] https://review.openstack.org/#/c/201458/
> > > [2] https://review.openstack.org/#/c/221732/
> > > [3] https://review.openstack.org/#/c/223887/
> > >
> > > [1] and [2] are trying to expose the API of detaching root devices, [3]
> > is
> > > trying to fix it in the current Rebuild API. But yet none of them got
> > much
> > > attention.
> > >
> > > As we now have strong demand on performing the "rebuilding" action for
> > > volume-backed instances, and yet there is not any clear information about
> > >  it. I wonder is there any plans of how to support it in Nova and Cinder?
> > >
> >
> > This seems entirely misguided by the users.
> >
> > Why not just boot a new instance on a new volume with the same image?
> > Names can be the same.. UUID's should never be anything except a physical
> > handle.
> >
> > __
> > OpenStack Development Mailing List (not for usage questions)
> > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> >

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Manila] Design summit notes

2015-11-10 Thread Silvan Kaiser
Ben, thanks a lot for this wrap-up, much appreciated!


2015-11-10 22:25 GMT+01:00 Ben Swartzlander :

> I wasn't going to write a wrap-up email this time around since so many
> people were able to attend in person, but enough people missed it that I
> changed my mind and decided to write down my own impressions of the
> sessions.
>
> Wednesday working session: Migration Improvements
> -
> During this session we covered the status of the migration feature so far
> (it's merged but experimental) and the existing gaps:
> 1) Will fail on shares on most backends with share servers
> 2) Controller node in the data path -- needs data copy service
> 3) No implementations of optimized migration yet
> 4) Confusion around task_state vs state
> 5) Need to disable most existing operations during a migration
> 6) Possibly need to change driver interface for getting mount info
>
> Basically there is a lot of work left to be done on migration but we're
> happy with the direction it's going. If we can address the gaps we could
> make the APIs supported in Mitaka. We're eager to get to building the
> valuable APIs on top of migration, but we can't do that until migration
> itself is solid.
>
> I also suggested that migration might benefit from an API change to allow
> a 2-phase migration which would allow the user (the admin in this case) to
> control when the final cutover happens instead of letting it happen by
> surprise.
>
> Wednesday working session: Access Allow/Deny Driver Interface
> -
> During this session I proposed a new driver interface for allowing/denying
> access to shares which is a single "sync access" API that the manager would
> call and pass all of the rules to. The main benefits of the change would be:
> 1) More reliable cleanup of errors
> 2) Support for atomically updating multiple rules
> 3) Simpler/more efficient implementation on some backends
>
> Most vendors agreed that the new interface would be superior, and
> generally speaking most vendors said that the new interface would be
> simpler and more efficient than the existing one.
>
> There were some who were unsure and one who specifically said an access
> sync would be inefficient compared to the current allow/deny semantics. We
> need to see if we can provide enough information in the new interface to
> allow them to be more efficient (such as providing the new rules AND the
> diff against the old rules).
>
> It was also pointed out that error reporting would be different using this
> new interface, because errors applying rules couldn't be associated with
> the specific rule that caused them. We need a solution to that problem.
>
> Thursday fishbowl session: Share Replication
> 
> There was a demo of the POC code from NetApp and general education on the
> new design. Since the idea was new to some and the implementation was new
> to everyone, there was not a lot of feedback.
>
> We did discuss a few issues, such as whether multiple shares should be
> allowed in a single AZ.
>
> We agreed that this replication feature will be exposed to the same kind
> of race conditions that exist in active/active HA, so there is additional
> pressure to solve the distributed locking problem. Fortunately the
> community seems to be converging on a solution to that problem -- the tooz
> library.
>
> We agreed that support for replication in a first party driver is
> essential for the feature to be accepted -- otherwise developers who don't
> have proprietary storage systems would be unable to develop/test on the
> feature.
>
> Thursday fishbowl session: Alternative Snapshot Semantics
> -
> During this session I proposed 2 new things you can do with snapshots:
> 1) Revert a share to a snapshot
> 2) Exporting snapshots directly as readonly shares
>
> For reverting snapshots, we agreed that the operation should preserve all
> existing snapshots. If a backend is unable to revert without deleting
> snapshots, it should not advertise the capability.
>
> For mounting snapshots, it was pointed out that we need to define the
> access rules for the share. I proposed simply inheriting the rules for the
> parent share with rw rules squashed to ro. That approach has downsides
> though because it links to the access on the snapshot and the share (which
> may no be desired) and also forces us to pass a list of snapshots into the
> access calls so the driver can update snapshot access when updating share
> access.
>
> Sage proposed creating a new concept of a readonly share and simply
> overloading the existing create-share-from-snapshot logic with a -readonly
> flag which gives us the semantics we want with much complexity. The
> downside to that approach is that we will have to add code to handle
> readonly shares.
>
> There was an additional proposal to allow the crea

Re: [openstack-dev] [nova][cinder] About rebuilding volume-backed instances.

2015-11-10 Thread John Griffith
On Mon, Nov 9, 2015 at 8:15 AM, Zhenyu Zheng 
wrote:

> Hi, thanks all for replying, sorry I might be a bit unclear.
>
> We have user demands that we only change the root device of an
> volume-backed instance for upper layer services. It's not cloudy but it is
> quite common. And changing OS is another demand that sort of related to
> this.
>

​I'm not sure I agree that it's "common", not to mention "common" in what
context/perspective?  Regardless I don't think the complexity and risk that
something like this introduces is worth the questionable value that it
gives.  No offense perhaps I'm still not quite getting the use case.​  It's
possible I don't know what you mean by "upper layer services".

By the way, when you say things like "it's not cloudy" I unfortunately
automatically think "isn't OpenStack a Cloud Platform"?


> Cinder supports live-backup volume, but not support live-restore a volume.
>

​Nope, no plans at all.  This would be a pretty involved task given that
the backup in cinder is just dd of blocks at the raw device level.  It is
however certainly possible to restore to a "new" volume and then swap
attachments, but then that goes against your desire of keeping UUID's, and
it doesn't work for Cinder backed Instances, so I guess that suggestion is
out as well. ​


>
> Are we planning to support this kind of action?
>

​Nope, no plans at all.  I'd also argue that we rather than compromising
and doing things that are "not cloudy" we should be pushing harder on being
cloudy and the strengths that come from it.​  We should be good at one
paradigm, not bad at all of them.
​

>
> Yours,
> Zheng
>
> On Mon, Nov 9, 2015 at 8:24 PM, Duncan Thomas 
> wrote:
>
>> On 9 November 2015 at 09:04, Zhenyu Zheng 
>> wrote:
>>
>>>  And Nova side also doesn't support detaching root device, that means we
>>> cannot performing volume backup/restore from cinder side, because those
>>> actions needs the volume in "available" status.
>>>
>>>
>>
>> It might be of interest to note that volume snapshots have always worked
>> on attached volumes, and as of liberty, the backup operation now supports a
>> --force=True option that does a backup of a live volume (via an internal
>> snapshot, so it should be crash consistent)
>>
>>
>> --
>> --
>> Duncan Thomas
>>
>> __
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe:
>> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>>
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] Help with getting keystone to migrate to Debian testing: fixing repoze.what and friends

2015-11-10 Thread Morgan Fainberg
On Nov 10, 2015 16:48, "Clint Byrum"  wrote:
>
> Excerpts from Morgan Fainberg's message of 2015-11-10 15:31:16 -0800:
> > On Tue, Nov 10, 2015 at 3:20 PM, Thomas Goirand  wrote:
> >
> > > Hi there!
> > >
> > > All of Liberty would be migrating from Sid to Testing (which is the
> > > pre-condition for an upload to offical Debian backports) if I didn't
> > > have a really annoying situation with the repoze.{what,who} packages.
I
> > > feel like I could get some help from the Python export folks here.
> > >
> > > What is it about?
> > > =
> > >
> > > Here's the dependency chain:
> > >
> > > - Keystone depends on pysaml2.
> > > - Pysaml2 depends on python-repoze.who >=2, which I uploaded to Sid.
> > > - python-repoze.what depends on python-repoze.who < 1.99
> > >
> > > Unfortunately, python-repoze.who doesn't migrate to Debian Testing
> > > because it would make python-repoze.what broken.
> > >
> > > To make the situation worse, python-repoze.what build-depends on
> > > python-repoze.who-testutil, which itself doesn't work with
> > > python-repoze.who >= 2.
> > >
> > > Note: repoze.who-testutil is within the package
> > > python-repoze.who-plugins who also contains 4 other plugins which are
> > > all broken with repoze.who >= 2, but the others could be dropped from
> > > Debian easily). We can't drop repoze.what completely, because there's
> > > turbogears2 and another package who needs it.
> > >
> > > There's no hope from upstream, as all of these seem to be abandoned
> > > projects.
> > >
> > > So I'm a bit stuck here, helpless, and I don't know how to fix the
> > > situation... :(
> > >
> > > What to fix?
> > > 
> > > Make repoze.what and repoze.who-testutil work with repoze.who >= 2.
> > >
> > > Call for help
> > > =
> > > I'm a fairly experienced package maintainer, but I still consider
myself
> > > a poor Python coder (probably because I spend all my time packaging
> > > rather than programming in Python: I know a way better other
programing
> > > languages).
> > >
> > > So I would enjoy a lot having some help here, also because my time is
> > > very limited and probably better invested working on packages to
assist
> > > the whole OpenStack project, rather than upstream code on some weirdo
> > > dependencies that I don't fully understand.
> > >
> > > So, would anyone be able to invest a bit of time, and help me fix the
> > > problems with repoze.what / repoze.who in Debian? If you can help,
> > > please ping me on IRC.
> > >
> > > Cheers,
> > >
> > > Thomas Goirand (zigo)
> > >
> > >
> > It looks like pysaml2 might be ok with < 1.99 of repoze.who here:
> > https://github.com/rohe/pysaml2/blob/master/setup.py#L30
> >
> > I admit I haven't tested it, but the requirements declaration doesn't
seem
> > to enforce the need for > 2. If that is in-fact the case that > 2 is
> > needed, we are a somewhat of an impass with dead/abandonware holding us
> > ransom. I'm not sure what the proper handling of that ends up being in
the
> > debian world.
>
> repoze.who doesn't look abandoned to me, so it is just repoze.what:
>
> https://github.com/repoze/repoze.who/commits/master
>
> who's just not being released (does anybody else smell a Laurel and
> Hardy skit coming on?)

Seriously!

>
> Also, this may have been something temporary, that then got left around
> because nobody bothered to try the released versions:
>
>
https://github.com/repoze/repoze.what/commit/b9fc014c0e174540679678af99f04b01756618de
>
> note, 2.0a1 wasn't compatible.. but perhaps 2.2 would work fine?
>
>

Def something to try out. If this is still an outstanding issue next week
(when I have a bit more time) I'll see what I can do to test out the
variations.

--Morgan
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


[openstack-dev] [Neutron][IPAM] ML2 don't call ipam driver remove_subnet function

2015-11-10 Thread thanh le giang
Dear folks

I have met a problem when implement IPAM driver, ML2 doesn't call
remove_subnet function of ipam driver because ML2 doesn't use delete_subnet
function of NeutronDbPluginV2

Now I workaround by using "SUBNET BEFORE_DELETE" event to notify external
IPAM that subnet will be deleted, but I think it's not a good solution.
According to my understanding this event should be used for checking subnet
is in-used or not. I think ML2 should call IPAM driver for this situation
or provide additional "SUBNET AFTER_DELETE" event

Thanks,
Thanh

Email: legiangt...@gmail.com 
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [fuel] Add Fuel to OpenStack projects: status update

2015-11-10 Thread Davanum Srinivas
Dima,

+1 to "additional scrutiny is there because they want to get this
right. Lets prove that their trust in us is not misplaced."

Thanks,
Dims

On Tue, Nov 10, 2015 at 10:10 PM, Dmitry Borodaenko
 wrote:
> As you may have guessed, many Fuel developers were holding their breath
> for the Technical Committee meeting today, where the decision on whether
> to accept Fuel into Big Tent as an OpenStack project [0] was on the
> agenda [1].
>
> [0] https://review.openstack.org/199232
> [1] 
> http://eavesdrop.openstack.org/meetings/tc/2015/tc.2015-11-10-20.02.log.html#l-115
>
> Unfortunately, we'll have to hold breath for another week: our proposal
> was not approved today, and the vote was postponed again. The good news
> is, most of the TC members present were in favor and have acknowledged
> that Fuel team has made significant progress in the right direction.
>
> The remaining objections are not new and not insurmountable: Jim Blair
> has pointed out that it's not enough to have _most_ of Fuel repositories
> covered by PTI compliant gate jobs, it has to be _all_ of them, and that
> we still have a few gaps. Thierry was willing to let us get away with a
> commitment that we complete this work by the end of the year, or be
> removed from the projects if we fail. However, Jim's concerns were
> seconded by Russel Bryant and Mark McClain who explicitly abstained
> until, in Russel's words, "the Infra team is happy". Without their votes
> and with 4 more TC members absent from the meeting, our proposal did not
> get enough votes to pass.
>
> I have documented the specific gaps in the gate jobs in my comment to
> the governance review linked above. To sum up, what's left to bring Fuel
> into full compliance with PTI is:
>
> 1) Enable the currently non-voting gate jobs for the new repositories
> extracted from fuel-web last week: fuel-menu, network-checker, shotgun.
>
> 2) Fix and enable the failing docs jobs in fuel-astute and fuel-docs.
>
> 3) Finish the unit test job for fuel-ostf.
>
> 4) Set up Ruby unit tests and syntax checks for fuel-astute and
> fuel-nailgun-agent.
>
> While figuring out some of the job failures here is tricky, I believe we
> should focus on remaining gaps and close all of them soon. It would be a
> shame to have come this far and have our proposal rejected because of a
> missing syntax check or a failure to compile HTML from RST.
>
> Jim's request to start work on running the more complex tests
> (specifically, multi-node deployment tests from fuel-qa) turned out to
> be more controversial, both because it is a new requirement that was
> explicitly excluded during the previous round of discussions in July,
> and because it's hard to objectively assess how much work, short of
> complete implementation and full conversion, would be enough to prove
> that there is a sufficient collaboration between Fuel and Infrastructure
> teams.
>
> We had a good opening discussion on #openstack-dev about this after the
> TC meeting [2]. Aleksandra Fedorova has mentioned that she actually
> proposed a talk for Tokyo about exactly this topic (which was
> unfortunately rejected), and promised to kick off a thread on
> openstack-dev ML based on the research she has done so far. It's a
> worthwhile long-term goal, I completely understand Infra team's desire
> to make sure Fuel project can pull its own weight on OpenStack Infra,
> and I will support efforts by Aleksandra and other Fuel Infra engineers
> to fully align our CI with OpenStack Infra.
>
> [2] 
> http://eavesdrop.openstack.org/irclogs/%23openstack-dev/%23openstack-dev.2015-11-10.log.html#t2015-11-10T21:03:34
>
> Still, I believe that making this a hard requirement for Fuel's
> acceptance into Big Tent would be one step too far down a slippery slope
> into a whole new vat of worms. Objective inclusion criteria such as
> Project Requirements and Project Testing Interface are there to protect
> OpenStack contributors from real and perceived favouritism. Declaring,
> especially selectively, that meeting these criteria may be insufficient,
> takes all the objectivity out of them. Fortunately, Jim did not insist
> on making progress with Fuel multi-node tests a hard requirement and
> confirmed that he will not block our proposal based on that. He still
> wants us to finish setting up gates, though, fair is fair.
>
> Finally, the odd one out was the objection from Dean Troyer: "re Fuel,
> I'm just not convinced it fits OpenStack's mission... we generally have
> stayed away from being a distro". It was quickly dismissed by other
> participants, but Dean still abstained, putting us one more vote short
> of approval. I think this serves to illustrate that many prominent
> members of OpenStack community still view Fuel as an OpenStack
> distribution, even after the two years we've spent establishing Fuel as
> a toolset for deployment and operation of OpenStack environments,
> decoupled from whatever Linux and OpenStack distributions you choose to
> deploy with i

[openstack-dev] [fuel] Add Fuel to OpenStack projects: status update

2015-11-10 Thread Dmitry Borodaenko
As you may have guessed, many Fuel developers were holding their breath
for the Technical Committee meeting today, where the decision on whether
to accept Fuel into Big Tent as an OpenStack project [0] was on the
agenda [1].

[0] https://review.openstack.org/199232
[1] 
http://eavesdrop.openstack.org/meetings/tc/2015/tc.2015-11-10-20.02.log.html#l-115

Unfortunately, we'll have to hold breath for another week: our proposal
was not approved today, and the vote was postponed again. The good news
is, most of the TC members present were in favor and have acknowledged
that Fuel team has made significant progress in the right direction.

The remaining objections are not new and not insurmountable: Jim Blair
has pointed out that it's not enough to have _most_ of Fuel repositories
covered by PTI compliant gate jobs, it has to be _all_ of them, and that
we still have a few gaps. Thierry was willing to let us get away with a
commitment that we complete this work by the end of the year, or be
removed from the projects if we fail. However, Jim's concerns were
seconded by Russel Bryant and Mark McClain who explicitly abstained
until, in Russel's words, "the Infra team is happy". Without their votes
and with 4 more TC members absent from the meeting, our proposal did not
get enough votes to pass.

I have documented the specific gaps in the gate jobs in my comment to
the governance review linked above. To sum up, what's left to bring Fuel
into full compliance with PTI is:

1) Enable the currently non-voting gate jobs for the new repositories
extracted from fuel-web last week: fuel-menu, network-checker, shotgun.

2) Fix and enable the failing docs jobs in fuel-astute and fuel-docs.

3) Finish the unit test job for fuel-ostf.

4) Set up Ruby unit tests and syntax checks for fuel-astute and
fuel-nailgun-agent.

While figuring out some of the job failures here is tricky, I believe we
should focus on remaining gaps and close all of them soon. It would be a
shame to have come this far and have our proposal rejected because of a
missing syntax check or a failure to compile HTML from RST.

Jim's request to start work on running the more complex tests
(specifically, multi-node deployment tests from fuel-qa) turned out to
be more controversial, both because it is a new requirement that was
explicitly excluded during the previous round of discussions in July,
and because it's hard to objectively assess how much work, short of
complete implementation and full conversion, would be enough to prove
that there is a sufficient collaboration between Fuel and Infrastructure
teams.

We had a good opening discussion on #openstack-dev about this after the
TC meeting [2]. Aleksandra Fedorova has mentioned that she actually
proposed a talk for Tokyo about exactly this topic (which was
unfortunately rejected), and promised to kick off a thread on
openstack-dev ML based on the research she has done so far. It's a
worthwhile long-term goal, I completely understand Infra team's desire
to make sure Fuel project can pull its own weight on OpenStack Infra,
and I will support efforts by Aleksandra and other Fuel Infra engineers
to fully align our CI with OpenStack Infra.

[2] 
http://eavesdrop.openstack.org/irclogs/%23openstack-dev/%23openstack-dev.2015-11-10.log.html#t2015-11-10T21:03:34

Still, I believe that making this a hard requirement for Fuel's
acceptance into Big Tent would be one step too far down a slippery slope
into a whole new vat of worms. Objective inclusion criteria such as
Project Requirements and Project Testing Interface are there to protect
OpenStack contributors from real and perceived favouritism. Declaring,
especially selectively, that meeting these criteria may be insufficient,
takes all the objectivity out of them. Fortunately, Jim did not insist
on making progress with Fuel multi-node tests a hard requirement and
confirmed that he will not block our proposal based on that. He still
wants us to finish setting up gates, though, fair is fair.

Finally, the odd one out was the objection from Dean Troyer: "re Fuel,
I'm just not convinced it fits OpenStack's mission... we generally have
stayed away from being a distro". It was quickly dismissed by other
participants, but Dean still abstained, putting us one more vote short
of approval. I think this serves to illustrate that many prominent
members of OpenStack community still view Fuel as an OpenStack
distribution, even after the two years we've spent establishing Fuel as
a toolset for deployment and operation of OpenStack environments,
decoupled from whatever Linux and OpenStack distributions you choose to
deploy with it. I can only hope that Fuel is accepted into Big Tent and
more distributions are encouraged to use it, so that this particular
confusion is finally laid to rest.

Some of you may be surprised by how much scrutiny Fuel is facing when
compared to smaller and younger projects. In a way, Fuel is a victim of
its own success: we've got so many components and such an exte

[openstack-dev] [neutron][tap-as-a-service] weekly meeting

2015-11-10 Thread Takashi Yamamoto
hi,

tap-as-a-service meeting will be held weekly, starting today.
http://eavesdrop.openstack.org/#Tap_as_a_Service_Meeting
anyone interested in the project is welcome.
sorry for immediate notice.

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [stable][infra][neutron] ZUUL_BRANCH not set for periodic stable jobs

2015-11-10 Thread Brandon Logan
+1 thanks for being on top of this Ihar

On Mon, 2015-11-09 at 20:10 -0500, Assaf Muller wrote:
> On Mon, Nov 9, 2015 at 5:30 PM, Ihar Hrachyshka  wrote:
> > Jeremy Stanley  wrote:
> >
> >> On 2015-11-09 17:31:00 +0100 (+0100), Ihar Hrachyshka wrote:
> >> [...]
> >>>
> >>> From the failure log, I determined that the tests fail because they
> >>> assume
> >>> neutron/liberty code, but actually run against neutron/master (that does
> >>> not
> >>> have that neutron.plugins.embrane.* namespace because the plugin was
> >>> removed
> >>> in Mitaka).
> >>>
> >>> I then compared how we fetch neutron in gate and in periodic jobs, and I
> >>> see
> >>> that ZUUL branch is not set in the latter jobs.
> >>
> >> [...]
> >>
> >> Short answer is that the periodic trigger in Zuul is changeless and
> >> thus branchless. It just wakes up at the specified time and starts a
> >> list of jobs associated with that pipeline for any projects. This is
> >> why the working periodic jobs have different names than their gerrit
> >> triggered pipeline equivalents... they need to hard-code a branch
> >> (usually as a JJB parameter).
> >> --
> >> Jeremy Stanley
> >
> >
> > UPD: we discussed the matter with Jeremy in irc in more details and came up
> > to agreement that the best way is actually modifying tools/tox_install.sh in
> > neutron-lbaas (and other projects affected) to set ZUUL_BRANCH if not passed
> > from Jenkins.
> >
> > For neutron-lbaas, I came up with the following patch:
> > https://review.openstack.org/#/c/24/
> >
> > Once it’s validated and reviewed, I will propose some more for other
> > projects (I believe at least vpnaas should be affected).
> >
> > Once it’s merged in master, I will propose backports for Liberty.
> 
> Great work Ihar, thanks for taking this on.
> 
> >
> > Ihar
> > __
> > OpenStack Development Mailing List (not for usage questions)
> > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> >
> 
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [kuryr] gerrit/git review problem error code 10061

2015-11-10 Thread Baohua Yang
Thanks to all!
It seems network connectivity problem. How sad again!  :(
i will try other ways.

On Tue, Nov 10, 2015 at 12:52 AM, Jeremy Stanley  wrote:

> On 2015-11-09 10:13:33 +0800 (+0800), Baohua Yang wrote:
> > Anyone recently meet such problem after cloning the latest code
> > from kuryr? Try proxy also, but not solved.
> [...]
> > The following command failed with exit code 1
> > "scp -P29418 yangbao...@review.openstack.org:hooks/commit-msg
> > .git\hooks\commit-msg"
> > ---
> > FATAL: Unable to connect to relay host, errno=10061
> > ssh_exchange_identification: Connection closed by remote host
> [...]
>
> I've checked our Gerrit SSH API authentication logs from the past 30
> days and find no record of any yangbaohua authenticating. Chances
> are this is a broken local proxy or some sort of intercepting
> firewall which is preventing your 29418/tcp connection from even
> reaching review.openstack.org.
>
> If you use Telnet or NetCat to connect to port 29418 on
> review.openstack.org directly, do you see an SSH banner starting
> with a string like "SSH-2.0-GerritCodeReview_2.8.4-19-g4548330
> (SSHD-CORE-0.9.0.201311081)" or something else?
> --
> Jeremy Stanley
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>


-- 
Best wishes!
Baohua
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


[openstack-dev] [NFV][Telco] Meeting Reminder - November 11th - 1400 UTC

2015-11-10 Thread Steve Gordon
Hi all,

Just a reminder this week's Telco Working Group meeting is scheduled for 
Wednesday the 11th at 1400 UTC. As always the agenda etherpad is here:

https://etherpad.openstack.org/p/nfv-meeting-agenda

I recognize that many folks will be at OPNFV summit this week but we will see 
how we go since last week we effectively skipped due to a number of people 
getting caught out by DST related changes.

Thanks,

Steve

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [networking-ovn] ovn-northd is a spof

2015-11-10 Thread Russell Bryant
On 11/10/2015 07:06 PM, gong_ys2004 wrote:
> Regard this, I read some architecture doc of it, SPOF in OVN way exists
> in two place:
> 1. ovn-northd, I don't know if we can run multiple ones, but the arch
> doc demos only one

The idea is that we will make it so you can run multiple.  We haven't
done it yet, though.  In the meantime, the expectation is that you can
easily run it in active/passive HA mode.  That will impact scale,
though.  A single ovn-northd is obviously not the longer term goal.

> 2. north db and south db, currently it is a OVSDB respectively, I don't
> think the distributed db layer can be easily and quickly solved.

Someone is actually already looking at distributing ovsdb-server.  If
that doesn't work out, the project has been pretty clear from the
beginning that the use of ovsdb-server was for convenience and if it
doesn't work, we'll switch.

I've been planning to write a FAQ in the networking-ovn documentation.
Some comments about our plans around HA are part of it.  I'll try to get
that written this week.

> curiously, why we adopt two dbs design at the beginning. why does not
> the ovn neutron plugin interact with south db?

We could write directly to the southbound db and cut ovn-northd and the
northbound db off, but we'd have to implement all of the logic from
ovn-northd in Neutron in that case.  The northbound db is a much simpler
model.

-- 
Russell Bryant

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] Help with getting keystone to migrate to Debian testing: fixing repoze.what and friends

2015-11-10 Thread Clint Byrum
Excerpts from Morgan Fainberg's message of 2015-11-10 15:31:16 -0800:
> On Tue, Nov 10, 2015 at 3:20 PM, Thomas Goirand  wrote:
> 
> > Hi there!
> >
> > All of Liberty would be migrating from Sid to Testing (which is the
> > pre-condition for an upload to offical Debian backports) if I didn't
> > have a really annoying situation with the repoze.{what,who} packages. I
> > feel like I could get some help from the Python export folks here.
> >
> > What is it about?
> > =
> >
> > Here's the dependency chain:
> >
> > - Keystone depends on pysaml2.
> > - Pysaml2 depends on python-repoze.who >=2, which I uploaded to Sid.
> > - python-repoze.what depends on python-repoze.who < 1.99
> >
> > Unfortunately, python-repoze.who doesn't migrate to Debian Testing
> > because it would make python-repoze.what broken.
> >
> > To make the situation worse, python-repoze.what build-depends on
> > python-repoze.who-testutil, which itself doesn't work with
> > python-repoze.who >= 2.
> >
> > Note: repoze.who-testutil is within the package
> > python-repoze.who-plugins who also contains 4 other plugins which are
> > all broken with repoze.who >= 2, but the others could be dropped from
> > Debian easily). We can't drop repoze.what completely, because there's
> > turbogears2 and another package who needs it.
> >
> > There's no hope from upstream, as all of these seem to be abandoned
> > projects.
> >
> > So I'm a bit stuck here, helpless, and I don't know how to fix the
> > situation... :(
> >
> > What to fix?
> > 
> > Make repoze.what and repoze.who-testutil work with repoze.who >= 2.
> >
> > Call for help
> > =
> > I'm a fairly experienced package maintainer, but I still consider myself
> > a poor Python coder (probably because I spend all my time packaging
> > rather than programming in Python: I know a way better other programing
> > languages).
> >
> > So I would enjoy a lot having some help here, also because my time is
> > very limited and probably better invested working on packages to assist
> > the whole OpenStack project, rather than upstream code on some weirdo
> > dependencies that I don't fully understand.
> >
> > So, would anyone be able to invest a bit of time, and help me fix the
> > problems with repoze.what / repoze.who in Debian? If you can help,
> > please ping me on IRC.
> >
> > Cheers,
> >
> > Thomas Goirand (zigo)
> >
> >
> It looks like pysaml2 might be ok with < 1.99 of repoze.who here:
> https://github.com/rohe/pysaml2/blob/master/setup.py#L30
> 
> I admit I haven't tested it, but the requirements declaration doesn't seem
> to enforce the need for > 2. If that is in-fact the case that > 2 is
> needed, we are a somewhat of an impass with dead/abandonware holding us
> ransom. I'm not sure what the proper handling of that ends up being in the
> debian world.

repoze.who doesn't look abandoned to me, so it is just repoze.what:

https://github.com/repoze/repoze.who/commits/master

who's just not being released (does anybody else smell a Laurel and
Hardy skit coming on?)

Also, this may have been something temporary, that then got left around
because nobody bothered to try the released versions:

https://github.com/repoze/repoze.what/commit/b9fc014c0e174540679678af99f04b01756618de

note, 2.0a1 wasn't compatible.. but perhaps 2.2 would work fine?

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


[openstack-dev] [networking-ovn] ovn-northd is a spof

2015-11-10 Thread gong_ys2004
hi,I know everyone want to provide there own neutron implementation, 
networking-ovn is one of themRegard this, I read some architecture doc of it, 
SPOF in OVN way exists in two place:1. ovn-northd, I don't know if we can run 
multiple ones, but the arch doc demos only one2. north db and south db, 
currently it is a OVSDB respectively, I don't think the distributed db layer 
can be easily and quickly solved.curiously, why we adopt two dbs design at the 
beginning. why does not the ovn neutron plugin interact with south db?
Regards,yong sheng gong__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


[openstack-dev] [telemetry][gnocchi] defining scope/domain

2015-11-10 Thread gord chung

hi,

i was doing some googling on the current state of time series databases 
to see if there was a worthwhile db to implement as a Gnocchi driver and 
it seems we're currently in the same state as before: there are flaws 
with all open source time series databases but if you have lots of 
money, Splunk is legit[1].


diving into each of the existing tsdb, there was a common theme that 
kept cropping up -- each of the tsdb claimed to do here> really, really well. by association, as Gnocchi captures time 
series data, it's assumed Gnocchi should do  
really, really well. this is obviously incorrect and setting Gnocchi up 
to fail.


as i get asked what Gnocchi is/does pretty often, i usually point them 
to docs[2]. that said, do we think it's necessary to define and list 
explicit use cases for Gnocchi? Gnocchi is 'a resource indexing and 
metric storage service' is a nice one-liner but would a comparison of 
Gnocchi make it easier to understand[3]. just wondering if there's a  
way we can make Gnocchi easier to understand and consumable.


[1] https://news.ycombinator.com/item?id=10530983
[2] http://docs.openstack.org/developer/gnocchi/
[3] http://prometheus.io/docs/introduction/comparison/

cheers,

--
gord


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [neutron][sfc] How could an L2 agent extension access agent methods ?

2015-11-10 Thread Henry Fourie
Ihar,
   Networking-sfc installs flows on br-int and br-tun for steering
traffic to the SFC port-pairs. On each bridge additional tables are used
for an egress forwarding pipeline (from the service VM port) and an
ingress pipeline (to the service VM port). Rpc operations between the
OVS driver and agents is used to initiate the flow installation.

We'd like to work with you on defining the L2 extensions.

 - Louis


- 
-Original Message-
From: Ihar Hrachyshka [mailto:ihrac...@redhat.com] 
Sent: Monday, November 09, 2015 4:44 AM
To: OpenStack Development Mailing List (not for usage questions)
Subject: Re: [openstack-dev] [neutron][sfc] How could an L2 agent extension 
access agent methods ?

Thanks Thomas, much appreciated.

I need to admit that we haven’t heard from SFC folks just yet. I will try to 
raise awareness that we wait for their feedback today on team meeting.  
Adding [sfc] tag to the topic to get more attention.

Ihar

Thomas Morin  wrote:

> Hi Ihar,
>
> Ihar Hrachyshka :
>> Reviving the thread.
>> [...] (I appreciate if someone checks me on the following though):
>
> This is an excellent recap.
>
>>  I set up a new etherpad to collect feedback from subprojects [2].
>
> I've filled in details for networking-bgpvpn.
> Please tell me if you need more information.
>
>> Once we collect use cases there and agree on agent API for extensions 
>> (even if per agent type), we will implement it and define as stable 
>> API, then pass objects that implement the API into extensions thru 
>> extension manager. If extensions support multiple agent types, they 
>> can still distinguish between which API to use based on agent type 
>> string passed into extension manager.
>>
>> I really hope we start to collect use cases early so that we have 
>> time to polish agent API and make it part of l2 extensions earlier in 
>> Mitaka cycle.
>
> We'll be happy to validate the applicability of this approach as soon 
> as something is ready.
>
> Thanks for taking up this work!
>
> -Thomas
>
>
>
>> Ihar Hrachyshka  wrote:
>>
 On 30 Sep 2015, at 12:53, Miguel Angel Ajo  wrote:



 Ihar Hrachyshka wrote:
>> On 30 Sep 2015, at 12:08, thomas.mo...@orange.com wrote:
>>
>> Hi Ihar,
>>
>> Ihar Hrachyshka :
 Miguel Angel Ajo :
> Do you have a rough idea of what operations you may need to do?
 Right now, what bagpipe driver for networking-bgpvpn needs to 
 interact with is:
 - int_br OVSBridge (read-only)
 - tun_br OVSBridge (add patch port, add flows)
 - patch_int_ofport port number (read-only)
 - local_vlan_map dict (read-only)
 - setup_entry_for_arp_reply method (called to add static ARP
 entries)
>>> Sounds very tightly coupled to OVS agent.
> Please bear in mind, the extension interface will be available 
> from different agent types (OVS, SR-IOV, [eventually LB]), so 
> this interface you're talking about could also serve as a 
> translation driver for the agents (where the translation is 
> possible), I totally understand that most extensions are 
> specific agent bound, and we must be able to identify the 
> agent we're serving back exactly.
 Yes, I do have this in mind, but what we've identified for now 
 seems to be OVS specific.
>>> Indeed it does. Maybe you can try to define the needed pieces in 
>>> high level actions, not internal objects you need to access to.
>>> Like ‘- connect endpoint X to Y’, ‘determine segmentation id for 
>>> a network’ etc.
>> I've been thinking about this, but would tend to reach the 
>> conclusion that the things we need to interact with are pretty 
>> hard to abstract out into something that would be generic across 
>> different agents.  Everything we need to do in our case relates 
>> to how the agents use bridges and represent networks internally:
>> linuxbridge has one bridge per Network, while OVS has a limited 
>> number of bridges playing different roles for all networks with 
>> internal segmentation.
>>
>> To look at the two things you  mention:
>> - "connect endpoint X to Y" : what we need to do is redirect the 
>> traffic destinated to the gateway of a Neutron network, to the 
>> thing that will do the MPLS forwarding for the right BGP VPN 
>> context (called VRF), in our case br-mpls (that could be done 
>> with an OVS table too) ; that action might be abstracted out to 
>> hide the details specific to OVS, but I'm not sure on how to  
>> name the destination in a way that would be agnostic to these 
>> details, and this is not really relevant to do until we have a 
>> relevant context in which the linuxbridge would pass packets to 
>> something doing MPLS forwarding (OVS is currently the only option 
>> we support for MPLS forwarding, and it does not really make sense 
>> t

Re: [openstack-dev] Help with getting keystone to migrate to Debian testing: fixing repoze.what and friends

2015-11-10 Thread Morgan Fainberg
On Tue, Nov 10, 2015 at 3:20 PM, Thomas Goirand  wrote:

> Hi there!
>
> All of Liberty would be migrating from Sid to Testing (which is the
> pre-condition for an upload to offical Debian backports) if I didn't
> have a really annoying situation with the repoze.{what,who} packages. I
> feel like I could get some help from the Python export folks here.
>
> What is it about?
> =
>
> Here's the dependency chain:
>
> - Keystone depends on pysaml2.
> - Pysaml2 depends on python-repoze.who >=2, which I uploaded to Sid.
> - python-repoze.what depends on python-repoze.who < 1.99
>
> Unfortunately, python-repoze.who doesn't migrate to Debian Testing
> because it would make python-repoze.what broken.
>
> To make the situation worse, python-repoze.what build-depends on
> python-repoze.who-testutil, which itself doesn't work with
> python-repoze.who >= 2.
>
> Note: repoze.who-testutil is within the package
> python-repoze.who-plugins who also contains 4 other plugins which are
> all broken with repoze.who >= 2, but the others could be dropped from
> Debian easily). We can't drop repoze.what completely, because there's
> turbogears2 and another package who needs it.
>
> There's no hope from upstream, as all of these seem to be abandoned
> projects.
>
> So I'm a bit stuck here, helpless, and I don't know how to fix the
> situation... :(
>
> What to fix?
> 
> Make repoze.what and repoze.who-testutil work with repoze.who >= 2.
>
> Call for help
> =
> I'm a fairly experienced package maintainer, but I still consider myself
> a poor Python coder (probably because I spend all my time packaging
> rather than programming in Python: I know a way better other programing
> languages).
>
> So I would enjoy a lot having some help here, also because my time is
> very limited and probably better invested working on packages to assist
> the whole OpenStack project, rather than upstream code on some weirdo
> dependencies that I don't fully understand.
>
> So, would anyone be able to invest a bit of time, and help me fix the
> problems with repoze.what / repoze.who in Debian? If you can help,
> please ping me on IRC.
>
> Cheers,
>
> Thomas Goirand (zigo)
>
>
It looks like pysaml2 might be ok with < 1.99 of repoze.who here:
https://github.com/rohe/pysaml2/blob/master/setup.py#L30

I admit I haven't tested it, but the requirements declaration doesn't seem
to enforce the need for > 2. If that is in-fact the case that > 2 is
needed, we are a somewhat of an impass with dead/abandonware holding us
ransom. I'm not sure what the proper handling of that ends up being in the
debian world.

--Morgan
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


[openstack-dev] Help with getting keystone to migrate to Debian testing: fixing repoze.what and friends

2015-11-10 Thread Thomas Goirand
Hi there!

All of Liberty would be migrating from Sid to Testing (which is the
pre-condition for an upload to offical Debian backports) if I didn't
have a really annoying situation with the repoze.{what,who} packages. I
feel like I could get some help from the Python export folks here.

What is it about?
=

Here's the dependency chain:

- Keystone depends on pysaml2.
- Pysaml2 depends on python-repoze.who >=2, which I uploaded to Sid.
- python-repoze.what depends on python-repoze.who < 1.99

Unfortunately, python-repoze.who doesn't migrate to Debian Testing
because it would make python-repoze.what broken.

To make the situation worse, python-repoze.what build-depends on
python-repoze.who-testutil, which itself doesn't work with
python-repoze.who >= 2.

Note: repoze.who-testutil is within the package
python-repoze.who-plugins who also contains 4 other plugins which are
all broken with repoze.who >= 2, but the others could be dropped from
Debian easily). We can't drop repoze.what completely, because there's
turbogears2 and another package who needs it.

There's no hope from upstream, as all of these seem to be abandoned
projects.

So I'm a bit stuck here, helpless, and I don't know how to fix the
situation... :(

What to fix?

Make repoze.what and repoze.who-testutil work with repoze.who >= 2.

Call for help
=
I'm a fairly experienced package maintainer, but I still consider myself
a poor Python coder (probably because I spend all my time packaging
rather than programming in Python: I know a way better other programing
languages).

So I would enjoy a lot having some help here, also because my time is
very limited and probably better invested working on packages to assist
the whole OpenStack project, rather than upstream code on some weirdo
dependencies that I don't fully understand.

So, would anyone be able to invest a bit of time, and help me fix the
problems with repoze.what / repoze.who in Debian? If you can help,
please ping me on IRC.

Cheers,

Thomas Goirand (zigo)

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [neutron][sfc] How could an L2 agent extension access agent methods ?

2015-11-10 Thread Cathy Zhang
Hi Vikram,

Thanks for the heads-up. Take care of yourself and your family.
We will provide the feedback on L2 agent.

Thanks,
Cathy

From: Vikram Choudhary [mailto:viks...@gmail.com]
Sent: Monday, November 09, 2015 6:59 PM
To: OpenStack Development Mailing List (not for usage questions); Cathy Zhang
Subject: Re: [openstack-dev] [neutron][sfc] How could an L2 agent extension 
access agent methods ?


Hi Cathy,

Could you please check on this. My mother passed away yesterday and I will be 
on leave for couple of weeks.

Thanks
Vikram
On 09-Nov-2015 6:15 pm, "Ihar Hrachyshka" 
mailto:ihrac...@redhat.com>> wrote:
Thanks Thomas, much appreciated.

I need to admit that we haven’t heard from SFC folks just yet. I will try to 
raise awareness that we wait for their feedback today on team meeting. Adding 
[sfc] tag to the topic to get more attention.

Ihar

Thomas Morin mailto:thomas.mo...@orange.com>> wrote:
Hi Ihar,

Ihar Hrachyshka :
Reviving the thread.
[...] (I appreciate if someone checks me on the following though):

This is an excellent recap.
 I set up a new etherpad to collect feedback from subprojects [2].

I've filled in details for networking-bgpvpn.
Please tell me if you need more information.
Once we collect use cases there and agree on agent API for extensions (even if 
per agent type), we will implement it and define as stable API, then pass 
objects that implement the API into extensions thru extension manager. If 
extensions support multiple agent types, they can still distinguish between 
which API to use based on agent type string passed into extension manager.

I really hope we start to collect use cases early so that we have time to 
polish agent API and make it part of l2 extensions earlier in Mitaka cycle.

We'll be happy to validate the applicability of this approach as soon as 
something is ready.

Thanks for taking up this work!

-Thomas


Ihar Hrachyshka mailto:ihrac...@redhat.com>> wrote:
On 30 Sep 2015, at 12:53, Miguel Angel Ajo 
mailto:mangel...@redhat.com>> wrote:



Ihar Hrachyshka wrote:
On 30 Sep 2015, at 12:08, 
thomas.mo...@orange.com wrote:

Hi Ihar,

Ihar Hrachyshka :
Miguel Angel Ajo :
Do you have a rough idea of what operations you may need to do?
Right now, what bagpipe driver for networking-bgpvpn needs to interact with is:
- int_br OVSBridge (read-only)
- tun_br OVSBridge (add patch port, add flows)
- patch_int_ofport port number (read-only)
- local_vlan_map dict (read-only)
- setup_entry_for_arp_reply method (called to add static ARP entries)
Sounds very tightly coupled to OVS agent.
Please bear in mind, the extension interface will be available from different 
agent types
(OVS, SR-IOV, [eventually LB]), so this interface you're talking about could 
also serve as
a translation driver for the agents (where the translation is possible), I 
totally understand
that most extensions are specific agent bound, and we must be able to identify
the agent we're serving back exactly.
Yes, I do have this in mind, but what we've identified for now seems to be OVS 
specific.
Indeed it does. Maybe you can try to define the needed pieces in high level 
actions, not internal objects you need to access to. Like ‘- connect endpoint X 
to Y’, ‘determine segmentation id for a network’ etc.
I've been thinking about this, but would tend to reach the conclusion that the 
things we need to interact with are pretty hard to abstract out into something 
that would be generic across different agents.  Everything we need to do in our 
case relates to how the agents use bridges and represent networks internally: 
linuxbridge has one bridge per Network, while OVS has a limited number of 
bridges playing different roles for all networks with internal segmentation.

To look at the two things you  mention:
- "connect endpoint X to Y" : what we need to do is redirect the traffic 
destinated to the gateway of a Neutron network, to the thing that will do the 
MPLS forwarding for the right BGP VPN context (called VRF), in our case br-mpls 
(that could be done with an OVS table too) ; that action might be abstracted 
out to hide the details specific to OVS, but I'm not sure on how to  name the 
destination in a way that would be agnostic to these details, and this is not 
really relevant to do until we have a relevant context in which the linuxbridge 
would pass packets to something doing MPLS forwarding (OVS is currently the 
only option we support for MPLS forwarding, and it does not really make sense 
to mix linuxbridge for Neutron L2/L3 and OVS for MPLS)
- "determine segmentation id for a network": this is something really 
OVS-agent-specific, the linuxbridge agent uses multiple linux bridges, and does 
not rely on internal segmentation

Completely abstracting out packet forwarding pipelines in OVS and linuxbridge 
agents would possibly allow defining an interface that agent extension could 
use without to know about anything specific to OVS or the linuxbridge, but I 
belie

Re: [openstack-dev] [neutron][sfc] How could an L2 agent extension access agent methods ?

2015-11-10 Thread Cathy Zhang
Hi Ihar,

Thanks for initiating this discussion. I am in OPNFV Summit. Henry Fourie of 
SFC project team will reply with our feedback. 

Cathy

-Original Message-
From: Ihar Hrachyshka [mailto:ihrac...@redhat.com] 
Sent: Monday, November 09, 2015 4:44 AM
To: OpenStack Development Mailing List (not for usage questions)
Subject: Re: [openstack-dev] [neutron][sfc] How could an L2 agent extension 
access agent methods ?

Thanks Thomas, much appreciated.

I need to admit that we haven’t heard from SFC folks just yet. I will try to 
raise awareness that we wait for their feedback today on team meeting.  
Adding [sfc] tag to the topic to get more attention.

Ihar

Thomas Morin  wrote:

> Hi Ihar,
>
> Ihar Hrachyshka :
>> Reviving the thread.
>> [...] (I appreciate if someone checks me on the following though):
>
> This is an excellent recap.
>
>>  I set up a new etherpad to collect feedback from subprojects [2].
>
> I've filled in details for networking-bgpvpn.
> Please tell me if you need more information.
>
>> Once we collect use cases there and agree on agent API for extensions 
>> (even if per agent type), we will implement it and define as stable 
>> API, then pass objects that implement the API into extensions thru 
>> extension manager. If extensions support multiple agent types, they 
>> can still distinguish between which API to use based on agent type 
>> string passed into extension manager.
>>
>> I really hope we start to collect use cases early so that we have 
>> time to polish agent API and make it part of l2 extensions earlier in 
>> Mitaka cycle.
>
> We'll be happy to validate the applicability of this approach as soon 
> as something is ready.
>
> Thanks for taking up this work!
>
> -Thomas
>
>
>
>> Ihar Hrachyshka  wrote:
>>
 On 30 Sep 2015, at 12:53, Miguel Angel Ajo  wrote:



 Ihar Hrachyshka wrote:
>> On 30 Sep 2015, at 12:08, thomas.mo...@orange.com wrote:
>>
>> Hi Ihar,
>>
>> Ihar Hrachyshka :
 Miguel Angel Ajo :
> Do you have a rough idea of what operations you may need to do?
 Right now, what bagpipe driver for networking-bgpvpn needs to 
 interact with is:
 - int_br OVSBridge (read-only)
 - tun_br OVSBridge (add patch port, add flows)
 - patch_int_ofport port number (read-only)
 - local_vlan_map dict (read-only)
 - setup_entry_for_arp_reply method (called to add static ARP
 entries)
>>> Sounds very tightly coupled to OVS agent.
> Please bear in mind, the extension interface will be available 
> from different agent types (OVS, SR-IOV, [eventually LB]), so 
> this interface you're talking about could also serve as a 
> translation driver for the agents (where the translation is 
> possible), I totally understand that most extensions are 
> specific agent bound, and we must be able to identify the 
> agent we're serving back exactly.
 Yes, I do have this in mind, but what we've identified for now 
 seems to be OVS specific.
>>> Indeed it does. Maybe you can try to define the needed pieces in 
>>> high level actions, not internal objects you need to access to.
>>> Like ‘- connect endpoint X to Y’, ‘determine segmentation id for 
>>> a network’ etc.
>> I've been thinking about this, but would tend to reach the 
>> conclusion that the things we need to interact with are pretty 
>> hard to abstract out into something that would be generic across 
>> different agents.  Everything we need to do in our case relates 
>> to how the agents use bridges and represent networks internally:
>> linuxbridge has one bridge per Network, while OVS has a limited 
>> number of bridges playing different roles for all networks with 
>> internal segmentation.
>>
>> To look at the two things you  mention:
>> - "connect endpoint X to Y" : what we need to do is redirect the 
>> traffic destinated to the gateway of a Neutron network, to the 
>> thing that will do the MPLS forwarding for the right BGP VPN 
>> context (called VRF), in our case br-mpls (that could be done 
>> with an OVS table too) ; that action might be abstracted out to 
>> hide the details specific to OVS, but I'm not sure on how to  
>> name the destination in a way that would be agnostic to these 
>> details, and this is not really relevant to do until we have a 
>> relevant context in which the linuxbridge would pass packets to 
>> something doing MPLS forwarding (OVS is currently the only option 
>> we support for MPLS forwarding, and it does not really make sense 
>> to mix linuxbridge for Neutron
>> L2/L3 and OVS for MPLS)
>> - "determine segmentation id for a network": this is something 
>> really OVS-agent-specific, the linuxbridge agent uses multiple 
>> linux bridges, and does not rely on internal segmentation
>>

Re: [openstack-dev] [neutron][qos][fwaas] service groups vs. traffic classifiers

2015-11-10 Thread Eichberger, German
Sean and Mickey +1

As much as I like us using the same API to create classifiers (users only need 
to learn one way) I think for the short term we should work with our own 
constructs and rely on a common data model. That will allow us to iterate 
faster on the REST API level and not have premature constraints (as Mickey 
mentions).

Midterm we should create some common API which is a superset of the 
functionality of all projects using it.

German



On 11/10/15, 5:30 AM, "Sean M. Collins"  wrote:

>On Mon, Nov 09, 2015 at 07:58:34AM EST, Jay Pipes wrote:
>> In short, my preference, in order, would be:
>> 
>> 1) Enhance/evolve the existing security-groups and security-group-rules API
>> in Neutron to support more generic classification of traffic from L2 to L7,
>> using mostly the modeling that Sean has put together in his PoC library.
>
>
>
>> 2) Keep the security-group API as-is to keep outward compatibility with AWS.
>> Create a single, new service-groups and service-group-rules API for L2 to L7
>> traffic classification using mostly the modeling that Sean has put together.
>> Remove the networking-sfc repo and obselete the classifier spec. Not sure
>> what should/would happen to the FWaaS API, frankly.
>> 
>
>I'd prefer that since we're already redesigning the Firewall API that we
>go ahead and make the Firewall API more expressive, so users can
>classify L2 to L7 traffic and then make filtering decisions. Let's not
>complicate the Security Group API with more advanced features that we
>just bolt on. So my vote is for #2 - with slight adjustments. I still
>think the networking-sfc repo should stay around, and that collaboration
>on the common classifier framework should happen, so that we can start
>both projects (sfc and fwaas) with a common datamodel for the classifier
>piece.
>
>As to the REST-ful API for creating classifiers, I don't know if it
>should reside in the networking-sfc project. It's a big enough piece
>that it will most likely need to be its own endpoint and repo, and have
>stakeholders from other projects, not just networking-sfc. That will
>take time and quite a bit of wrangling, so I'd like to defer that for a
>bit and just work on all the services having the same data model, where
>we can make changes quickly, since they are not visible to API
>consumers.
>
>-- 
>Sean M. Collins
>
>__
>OpenStack Development Mailing List (not for usage questions)
>Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
>http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [heat][keystone] How to handle request for global admin in policy.json?

2015-11-10 Thread Steven Hardy
On Tue, Nov 10, 2015 at 10:53:46AM -0500, Adam Young wrote:
> On 11/10/2015 05:08 AM, Henry Nash wrote:
> >Steve,
> >
> >Currently, your best option is to use something similar to the 
> >policy.v3cloudsample.json, where you basically “bless” a project (or domain) 
> >as being the “cloud admin project/domain”.  Having a role on that gives you 
> >super-powers.  The only trouble with this right now is that you have to 
> >paste the ID of your blessed project/domain into the policy file (you only 
> >have to do that once, of course) - basically you replace the 
> >“admin_domain_id” with the ID of your blessed project/domain.
> >
> >What we are considering for Mitaka is make this a bit more friendly, so you 
> >don’t have to modify the policy file - rather you define your “blessed 
> >project” in your config file, and tokens that are issue on this blessed 
> >project will have an extra attribute (e.g. “is_admin_project”), which your 
> >policy file can check for.
> 
> Henry is using a bitof the British tendency toward understatement here.  Let
> me make this more explicit:
> 
> We are going to add a value to the Keystone token validation response that
> will indicate that the proejct is an admin project. Use that.  Don't develop
> something for Mitaka that does not use that.

Henry and Adam, many thanks for the information.

I'll follow the spec referenced by Adam and hopefully we can look to make
use of the new scheme when it's implemented - happy to help out with some
testing when you think it's ready for us to try.

Thanks!

Steve

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


[openstack-dev] [Manila] Design summit notes

2015-11-10 Thread Ben Swartzlander
I wasn't going to write a wrap-up email this time around since so many 
people were able to attend in person, but enough people missed it that I 
changed my mind and decided to write down my own impressions of the 
sessions.


Wednesday working session: Migration Improvements
-
During this session we covered the status of the migration feature so 
far (it's merged but experimental) and the existing gaps:

1) Will fail on shares on most backends with share servers
2) Controller node in the data path -- needs data copy service
3) No implementations of optimized migration yet
4) Confusion around task_state vs state
5) Need to disable most existing operations during a migration
6) Possibly need to change driver interface for getting mount info

Basically there is a lot of work left to be done on migration but we're 
happy with the direction it's going. If we can address the gaps we could 
make the APIs supported in Mitaka. We're eager to get to building the 
valuable APIs on top of migration, but we can't do that until migration 
itself is solid.


I also suggested that migration might benefit from an API change to 
allow a 2-phase migration which would allow the user (the admin in this 
case) to control when the final cutover happens instead of letting it 
happen by surprise.


Wednesday working session: Access Allow/Deny Driver Interface
-
During this session I proposed a new driver interface for 
allowing/denying access to shares which is a single "sync access" API 
that the manager would call and pass all of the rules to. The main 
benefits of the change would be:

1) More reliable cleanup of errors
2) Support for atomically updating multiple rules
3) Simpler/more efficient implementation on some backends

Most vendors agreed that the new interface would be superior, and 
generally speaking most vendors said that the new interface would be 
simpler and more efficient than the existing one.


There were some who were unsure and one who specifically said an access 
sync would be inefficient compared to the current allow/deny semantics. 
We need to see if we can provide enough information in the new interface 
to allow them to be more efficient (such as providing the new rules AND 
the diff against the old rules).


It was also pointed out that error reporting would be different using 
this new interface, because errors applying rules couldn't be associated 
with the specific rule that caused them. We need a solution to that problem.


Thursday fishbowl session: Share Replication

There was a demo of the POC code from NetApp and general education on 
the new design. Since the idea was new to some and the implementation 
was new to everyone, there was not a lot of feedback.


We did discuss a few issues, such as whether multiple shares should be 
allowed in a single AZ.


We agreed that this replication feature will be exposed to the same kind 
of race conditions that exist in active/active HA, so there is 
additional pressure to solve the distributed locking problem. 
Fortunately the community seems to be converging on a solution to that 
problem -- the tooz library.


We agreed that support for replication in a first party driver is 
essential for the feature to be accepted -- otherwise developers who 
don't have proprietary storage systems would be unable to develop/test 
on the feature.


Thursday fishbowl session: Alternative Snapshot Semantics
-
During this session I proposed 2 new things you can do with snapshots:
1) Revert a share to a snapshot
2) Exporting snapshots directly as readonly shares

For reverting snapshots, we agreed that the operation should preserve 
all existing snapshots. If a backend is unable to revert without 
deleting snapshots, it should not advertise the capability.


For mounting snapshots, it was pointed out that we need to define the 
access rules for the share. I proposed simply inheriting the rules for 
the parent share with rw rules squashed to ro. That approach has 
downsides though because it links to the access on the snapshot and the 
share (which may no be desired) and also forces us to pass a list of 
snapshots into the access calls so the driver can update snapshot access 
when updating share access.


Sage proposed creating a new concept of a readonly share and simply 
overloading the existing create-share-from-snapshot logic with a 
-readonly flag which gives us the semantics we want with much 
complexity. The downside to that approach is that we will have to add 
code to handle readonly shares.


There was an additional proposal to allow the create/delete snapshot 
calls without any other snapshot-related calls because some backends 
have in-band snapshot semantics. This is probably unnecessary because 
every backend that has snapshots is likely to support at 

Re: [openstack-dev] [neutron][upgrade] new 'all things upgrade' subteam

2015-11-10 Thread Martin Hickey
I am interested too and will be available to the subteam.

On Tue, Nov 10, 2015 at 9:03 PM, Sean M. Collins  wrote:

> I'm excited. I plan on attending and being part of the subteam. I think
> the tags that Dan Smith recently introduced could be our deliverables,
> where this subteam focuses on working towards Neutron being tagged with
> these tags.
>
> https://review.openstack.org/239771 - Introduce assert:supports-upgrade
> tag
>
> https://review.openstack.org/239778 - Introduce
> assert:supports-rolling-upgrade tag
> --
> Sean M. Collins
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [neutron][upgrade] new 'all things upgrade' subteam

2015-11-10 Thread Sean M. Collins
I'm excited. I plan on attending and being part of the subteam. I think
the tags that Dan Smith recently introduced could be our deliverables,
where this subteam focuses on working towards Neutron being tagged with
these tags.

https://review.openstack.org/239771 - Introduce assert:supports-upgrade tag

https://review.openstack.org/239778 - Introduce assert:supports-rolling-upgrade 
tag
-- 
Sean M. Collins

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


[openstack-dev] [Solum] Hack day / Bug triage day

2015-11-10 Thread Devdatta Kulkarni
Hi Solum team,

In last two IRC meetings we have discussed about spending a day triaging Solum 
bugs.
In today's meeting, participants agreed to do this on November 18, 2015.
You can find today's meeting logs here:
http://eavesdrop.openstack.org/meetings/solum_team_meeting/2015/solum_team_meeting.2015-11-10-17.00.log.html

This is a virtual event.

We will spend a day exercising Solum. We will triage bugs, try out new cli and 
api functionality, 
push outstanding patches forward, discuss design and implementation of new 
features.

I have setup an etherpad with relevant details.
https://etherpad.openstack.org/p/solum-hackday-nov18-2015

Please add yourself to the etherpad if you plan on participating in the event.

This will also be a good opportunity for anyone interested in Solum to try it 
out.
We can get you started with your Solum development environment.

Thanks,
Devdatta

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [rpm-packaging] core reviewers nomination

2015-11-10 Thread Dirk Müller
Haikel,


> I'd like to propose new candidates for RPM packaging core reviewers:
> Alan Pevec
> Jakub Ruzicka

+1

Greetings,
Dirk

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [OpenStack-Infra] Report from Gerrit User Summit

2015-11-10 Thread Sean Dague
On 11/10/2015 02:24 PM, David Pursehouse wrote:
> On Tue, Nov 10, 2015 at 4:53 AM Sean Dague  > wrote:
> 
> <...>
> 
> Is there any update on label:Code-Review<=-1,group:nova-core ? The group
> search support is documented, but as far as I can tell doesn't work
> except under very specific ldap configs.
> https://code.google.com/p/gerrit/issues/detail?id=3018
> 
> That would be hugely helpful.
> 
> 
> Khai started work on that but it's not working properly yet:
> 
> https://gerrit-review.googlesource.com/#/c/62880/
> 
> Let's see if we can get some eyes on it during this week's hackathon...

Thanks much!

-Sean

-- 
Sean Dague
http://dague.net

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


[openstack-dev] [Neutron] Stop pushing stuff to the gate queue

2015-11-10 Thread Armando M.
Neutron Cores,

We have high failure rate (see [1] for more context). We have an initial
bug report [2] filed, and more triaged is happening.

Let's hold on before we push stuff to the gate queue. Once [2] is solved
and the fire is put out, we'll resume the merge frenzy.

As a general reminder, please be conscious of failure rate of [3], and pay
attention to the Neutron dashboards [4], it helps us detect issues sooner
rather than later.

Cheers,
Armando

[1]
http://lists.openstack.org/pipermail/openstack-dev/2015-November/079133.html
[2] https://bugs.launchpad.net/neutron/+bug/1514935
[3] http://tinyurl.com/ne3ex4v
[4] http://docs.openstack.org/developer/neutron/#dashboards
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Fuel] Running Fuel node as non-superuser

2015-11-10 Thread Dmitry Nikishov
Stanislaw,

I've been experimenting with 'capsh' on the 6.1 master node and it doesn't
seem to preserve any capabilities when setting SECURE_NOROOT bit, even if
explicitely told to do so (via either --keep=1 or "SECURE_KEEP_CAPS" bit).

On Tue, Nov 10, 2015 at 11:20 AM, Dmitry Nikishov 
wrote:

> Bartolomiej, Adam,
> Stanislaw is correct. And this is going to be ported to master. The goal
> currently is to reach an agreement on the implementation so that there's
> going to be a some kinf of compatibility during upgrades.
>
> Stanislaw,
> Do I understand correctly that you propose using something like sucap to
> launch from root, switch to a different user and then drop capabilities
> which are not required?
>
> On Tue, Nov 10, 2015 at 3:11 AM, Stanislaw Bogatkin <
> sbogat...@mirantis.com> wrote:
>
>> Bartolomiej, it's customer-related patches, they, I think, have to be
>> done for 6.1 prior to 8+ release.
>>
>> Dmitry, it's nice to hear about it. Did you consider to use linux
>> capabilities on fuel-related processes instead of just using non-extended
>> POSIX privileged/non-privileged permission checks?
>>
>> On Tue, Nov 10, 2015 at 10:11 AM, Bartlomiej Piotrowski <
>> bpiotrow...@mirantis.com> wrote:
>>
>>> We don't develop features for already released versions… It should be
>>> done for master instead.
>>>
>>> BP
>>>
>>> On Tue, Nov 10, 2015 at 7:02 AM, Adam Heczko 
>>> wrote:
>>>
 Dmitry,
 +1

 Do you plan to port your patchset to future Fuel releases?

 A.

 On Tue, Nov 10, 2015 at 12:14 AM, Dmitry Nikishov <
 dnikis...@mirantis.com> wrote:

> Hey guys.
>
> I've been working on making Fuel not to rely on superuser privileges
> at least for day-to-day operations. These include:
> a) running Fuel services (nailgun, astute etc)
> b) user operations (create env, deploy, update, log in)
>
> The reason for this is that many security policies simply do not
> allow root access (especially remote) to servers/environments.
>
> This feature/enhancement means that anything that currently is being
> run under root, will be evaluated and, if possible, put under a
> non-privileged
> user. This also means that remote root access will be disabled.
> Instead, users will have to log in with "fueladmin" user.
>
> Together with Omar  we've put together a blueprint[0] and a
> spec[1] for this feature. I've been developing this for Fuel 6.1, so
> there
> are two patches into fuel-main[2] and fuel-library[3] that can give
> you an
> impression of current approach.
>
> These patches do following:
> - Add fuel-admin-user package, which creates 'fueladmin'
> - Make all other fuel-* packages depend on fuel-admin-user
> - Put supervisord under 'fueladmin' user.
>
> Please review the spec/patches and let's have a discussion on the
> approach to
> this feature.
>
> Thank you.
>
> [0] https://blueprints.launchpad.net/fuel/+spec/fuel-nonsuperuser
> [1] https://review.openstack.org/243340
> [2] https://review.openstack.org/243337
> [3] https://review.openstack.org/243313
>
> --
> Dmitry Nikishov,
> Deployment Engineer,
> Mirantis, Inc.
>
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe:
> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>


 --
 Adam Heczko
 Security Engineer @ Mirantis Inc.


 __
 OpenStack Development Mailing List (not for usage questions)
 Unsubscribe:
 openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


>>>
>>>
>>> __
>>> OpenStack Development Mailing List (not for usage questions)
>>> Unsubscribe:
>>> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>
>>>
>>
>> __
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe:
>> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>>
>
>
> --
> Dmitry Nikishov,
> Deployment Engineer,
> Mirantis, Inc.
>



-- 
Dmitry Nikishov,
Deployment Engineer,
Mirantis, Inc.
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailm

Re: [openstack-dev] [neutron] [gate] 35% failure rate for neutron tempest jobs

2015-11-10 Thread Armando M.
On 10 November 2015 at 11:11, Sean Dague  wrote:

> On 11/10/2015 01:37 PM, Armando M. wrote:
> >
> >
> > On 10 November 2015 at 09:49, Sean Dague  > > wrote:
> >
> > The neutron tempest jobs are now at a 35% failure rate:
> > http://tinyurl.com/ne3ex4v (note, 35% is basically the worst
> possible
> > fail rate, because it's just passing enough to land patches that
> cause
> > that kind of fail on two test runs check/gate with a coin flip).
> >
> >
> > Sean, thanks for the heads-up.
> >
> >
> >
> > The failure is currently seen here -
> >
> http://logstash.openstack.org/#/dashboard/file/logstash.json?query=message:%5C%22No%20IPv4%20addresses%20found%20in:%20%5B%5D%5C%22
> >
> > That is a new assert that was added in Tempest. However it was added
> in
> > a path that expects there should be an IPv4 address. The fact that
> port
> > is sometimes not returning one is problematic.
> > https://review.openstack.org/#/c/241800/
> >
> > The server via nova is returning an address here -
> >
> http://logs.openstack.org/76/243676/1/check/gate-tempest-dsvm-neutron-full/291e1d7/logs/tempest.txt.gz#_2015-11-10_17_14_35_465
> >
> > But then when the port is polled here:
> >
> http://logs.openstack.org/76/243676/1/check/gate-tempest-dsvm-neutron-full/291e1d7/logs/tempest.txt.gz#_2015-11-10_17_14_35_527
> > it comes back with {"ports": []}
> >
> >
> > This can be contrasted with a working path where we do the similar
> > action on the Server is active here -
> >
> http://logs.openstack.org/76/243676/1/check/gate-tempest-dsvm-neutron-full/291e1d7/logs/tempest.txt.gz#_2015-11-10_17_13_48_193
> >
> > Then we verify the port -
> >
> http://logs.openstack.org/76/243676/1/check/gate-tempest-dsvm-neutron-full/291e1d7/logs/tempest.txt.gz#_2015-11-10_17_13_48_230
> >
> > Which returns:
> >
> >   Body: {"ports": [{"status": "ACTIVE", "binding:host_id":
> > "devstack-trusty-rax-dfw-5784820", "allowed_address_pairs": [],
> > "extra_dhcp_opts": [], "dns_assignment": [{"hostname":
> > "host-10-100-0-3", "ip_address": "10.100.0.3", "fqdn":
> > "host-10-100-0-3.openstacklocal."}], "device_owner": "compute:None",
> > "port_security_enabled": true, "binding:profile": {}, "fixed_ips":
> > [{"subnet_id": "147b1e65-3463-4965-8461-11b76a00dd99", "ip_address":
> > "10.100.0.3"}], "id": "65c11c76-42fc-4010-bbb8-58996911803e",
> > "security_groups": ["f2d48dcf-ea8d-4a7c-bf09-da37d3c2ee37"],
> > "device_id": "b03bec85-fe69-4c0d-94e8-51753a8bebd5", "name": "",
> > "admin_state_up": true, "network_id":
> > "eb72d3af-f1a0-410b-8085-76cbe19ace90", "dns_name": "",
> > "binding:vif_details": {"port_filter": true, "ovs_hybrid_plug":
> true},
> > "binding:vnic_type": "normal", "binding:vif_type": "ovs",
> "tenant_id":
> > "eab50a3d331c4db3a68f71d1ebdc41bf", "mac_address":
> > "fa:16:3e:02:e4:ee"}]}
> >
> >
> > HenryG suggested this might be related to the ERROR of "No more IP
> > addresses available on network". However that ERROR is thrown a lot
> in
> > neutron, and 60% of the times the tempest run is successful.
> >
> >
> > This issue is currently stuck and needs neutron folks to engage to
> get
> > us somewhere. Reverting the tempest patch which does the early
> > verification might make this class of fail go away, but I think what
> > it's done is surface a more fundamental bit where ports aren't active
> > when the server is active, which may explain deeper races we've had
> over
> > the years. So actually getting folks to dive in here would be really
> > great.
> >
> >
> > We'll dig into this more deeply. AFAIK, Nova servers won't go ACTIVE if
> > the port isn't, so we might have a regression. That said, it's been on
> > our radar to better synchronize actions that need to happen on port
> > setup. Right now for instance, DHCP and L2 setup is uncoordinated and
> > Kevin Benton has been looking into it.
> >
> > That said, I wonder if reverting the tempest patch is the best course of
> > action: we can then use Depends-on to test a Neutron fix and the revert
> > of the revert together without causing the gate too much grief.
> >
> > Thoughts?
>
> So, I just stared at the Tempest patch again, and honestly, reverting it
> isn't going to help anything.
>
> https://github.com/openstack/tempest/blob/a1edb75d7901a9e338ab397d208a40c99c5fd9a1/tempest/scenario/manager.py#L760-L765
>
> Because a revert just removes the assert len != 0
>
> The next line is an assert len == 1 (which has been there for a long time)
>
> So a len 0 will fail there as well. Which probably points to this being
> a neutron regression entirely, we'd still be failing with the empty
> ports list, it would just be an incredibly cryptic error of "Found
> multiple IPv4 addresses: []" (which actually means found 0 port addresses).
>
> The reason the change was pushed in Tempest was to make the fail
> con

Re: [openstack-dev] Can we get some sanity in the Neutron logs please?

2015-11-10 Thread Armando M.
On 10 November 2015 at 11:12, Matt Riedemann 
wrote:

>
>
> On 11/10/2015 1:10 PM, Matt Riedemann wrote:
>
>>
>>
>> On 11/10/2015 12:51 PM, Armando M. wrote:
>>
>>>
>>>
>>> On 10 November 2015 at 10:33, Matt Riedemann >> > wrote:
>>>
>>> Let me qualify by saying I'm not a Neutron person.
>>>
>>> We know that gate-tempest-dsvm-neutron-full is failing hard as of
>>> the last 24 hours [1].
>>>
>>> An error that's been showing up in tempest runs with neutron a lot
>>> is:
>>>
>>> "AssertionError: 0 == 0 : No IPv4 addresses found in: []"
>>>
>>> So checking logstash [2] it's hitting a lot. It's only recent
>>> because that failure message is new to Tempest in the last day or
>>> so, but it has a lot of hits, so whatever it is, it's failing a lot.
>>>
>>> So the next step is usually digging into service logs looking for
>>> errors. I check the q-svc logs first. Not many errors but a
>>> bazillion warnings for things not found (networks and devices). [3]
>>>
>>> For example:
>>>
>>> 2015-11-10 17:13:02.542 WARNING neutron.plugins.ml2.rpc
>>> [req-15a73753-1512-4689-9404-9658a0cd0c09 None None] Device
>>> aaa525be-14eb-44a5-beb0-ed722896be93 requested by agent
>>> ovs-agent-devstack-trusty-rax-iad-5785199 not found in database
>>>
>>> 2015-11-10 17:14:17.754 WARNING neutron.api.rpc.handlers.dhcp_rpc
>>> [req-3d7e9848-6151-4780-907f-43f11a2a8545 None None] Network
>>> b07ad9b2-e63e-4459-879d-3721074704e5 could not be found, it might
>>> have been deleted concurrently.
>>>
>>> Are several hundred of these warnings useful to an operator trying
>>> to debug a problem? The point of the CI gate testing is to try and
>>> simulate a production cloud environment. When something goes wrong,
>>> you check the logs. With the amount of warning/error level logging
>>> that is in the neutron logs, finding a real problem is like looking
>>> for a needle in a haystack. Since everything is async, 404s are
>>> expected when racing to delete a resource and they should be handled
>>> gracefully.
>>>
>>> Anyway, the server log isn't useful so I go digging in the agent
>>> logs and stacktraces there are aplenty. [4]
>>>
>>> Particularly this:
>>>
>>> "Exception: Port tapcea51630-e1 is not ready, resync needed"
>>>
>>> That's due to a new change landing in the last 24 hours [5]. But the
>>> trace shows up over 16K times since it landed [6].
>>>
>>> Checking the code, it's basically a loop processing events and when
>>> it hits an event it can't handle, it punts (breaking the loop so you
>>> don't process the other events after it - which is a bug), and the
>>> code that eventually handles it is just catching all Exception and
>>> tracing them out assuming they are really bad.
>>>
>>> At this point, as a non-neutron person, i.e. not well versed in the
>>> operations of neutron or how to debug it in great detail, I assume
>>> something is bad here but I don't really know - and the logs are so
>>> full of noise that I can't distinguish real failures.
>>>
>>> I don't mean to pick on this particular change, but it's a good
>>> example of a recent thing.
>>>
>>> I'd like to know if this is all known issue or WIP type stuff. I've
>>> complained about excessively noisey neutron logs in channel before
>>> and I'm usually told that they are either necessary (for whatever
>>> reason) or that rather than complain about the verbosity, I should
>>> fix the race that is causing it - which is not likely to happen
>>> since I don't have the async rpc happy nature of everything in
>>> neutron in my head to debug it (I doubt many do).
>>>
>>>
>>> I am not sure that's a fair statement: we usually pinpoint that just
>>> lowering log levels is not really solving the underlying issue
>>> (whichever it may be), and that comment really should apply to any
>>> project, not just Neutron. That said, we had examples where we took your
>>> input and drove the right fix ourselves.
>>>
>>> We have a 'logging' tag for Neutron bugs that we use to identify these
>>> type of cleanups. We'd need your attention to details to alert us of
>>> issues like these; we'll take care of the right fixes. Currently, the
>>> queue is pretty dry. If you can top it up, that'll be great. Going off
>>> on a log cleanup rampage doesn't seem like the best course of action;
>>> I'd rather knock issues one by one as they come, like the one you just
>>> mentioned.
>>>
>>> [1] https://bugs.launchpad.net/neutron/+bugs?field.tag=logging
>>>
>>
>> Tagged:
>>
>> https://bugs.launchpad.net/nova/+bug/1514935
>>
>> Although it must not be an official tag since it didn't auto-complete
>> for me, that should be fixed.
>>
>
> Crap, nevermind that, my bad - I had created the bug against nova (force
> of habit) rather than neutron. Confirmed that 'logging' is a neutron 

Re: [openstack-dev] Can we get some sanity in the Neutron logs please?

2015-11-10 Thread Armando M.
On 10 November 2015 at 11:10, Matt Riedemann 
wrote:

>
>
> On 11/10/2015 12:51 PM, Armando M. wrote:
>
>>
>>
>> On 10 November 2015 at 10:33, Matt Riedemann > > wrote:
>>
>> Let me qualify by saying I'm not a Neutron person.
>>
>> We know that gate-tempest-dsvm-neutron-full is failing hard as of
>> the last 24 hours [1].
>>
>> An error that's been showing up in tempest runs with neutron a lot is:
>>
>> "AssertionError: 0 == 0 : No IPv4 addresses found in: []"
>>
>> So checking logstash [2] it's hitting a lot. It's only recent
>> because that failure message is new to Tempest in the last day or
>> so, but it has a lot of hits, so whatever it is, it's failing a lot.
>>
>> So the next step is usually digging into service logs looking for
>> errors. I check the q-svc logs first. Not many errors but a
>> bazillion warnings for things not found (networks and devices). [3]
>>
>> For example:
>>
>> 2015-11-10 17:13:02.542 WARNING neutron.plugins.ml2.rpc
>> [req-15a73753-1512-4689-9404-9658a0cd0c09 None None] Device
>> aaa525be-14eb-44a5-beb0-ed722896be93 requested by agent
>> ovs-agent-devstack-trusty-rax-iad-5785199 not found in database
>>
>> 2015-11-10 17:14:17.754 WARNING neutron.api.rpc.handlers.dhcp_rpc
>> [req-3d7e9848-6151-4780-907f-43f11a2a8545 None None] Network
>> b07ad9b2-e63e-4459-879d-3721074704e5 could not be found, it might
>> have been deleted concurrently.
>>
>> Are several hundred of these warnings useful to an operator trying
>> to debug a problem? The point of the CI gate testing is to try and
>> simulate a production cloud environment. When something goes wrong,
>> you check the logs. With the amount of warning/error level logging
>> that is in the neutron logs, finding a real problem is like looking
>> for a needle in a haystack. Since everything is async, 404s are
>> expected when racing to delete a resource and they should be handled
>> gracefully.
>>
>> Anyway, the server log isn't useful so I go digging in the agent
>> logs and stacktraces there are aplenty. [4]
>>
>> Particularly this:
>>
>> "Exception: Port tapcea51630-e1 is not ready, resync needed"
>>
>> That's due to a new change landing in the last 24 hours [5]. But the
>> trace shows up over 16K times since it landed [6].
>>
>> Checking the code, it's basically a loop processing events and when
>> it hits an event it can't handle, it punts (breaking the loop so you
>> don't process the other events after it - which is a bug), and the
>> code that eventually handles it is just catching all Exception and
>> tracing them out assuming they are really bad.
>>
>> At this point, as a non-neutron person, i.e. not well versed in the
>> operations of neutron or how to debug it in great detail, I assume
>> something is bad here but I don't really know - and the logs are so
>> full of noise that I can't distinguish real failures.
>>
>> I don't mean to pick on this particular change, but it's a good
>> example of a recent thing.
>>
>> I'd like to know if this is all known issue or WIP type stuff. I've
>> complained about excessively noisey neutron logs in channel before
>> and I'm usually told that they are either necessary (for whatever
>> reason) or that rather than complain about the verbosity, I should
>> fix the race that is causing it - which is not likely to happen
>> since I don't have the async rpc happy nature of everything in
>> neutron in my head to debug it (I doubt many do).
>>
>>
>> I am not sure that's a fair statement: we usually pinpoint that just
>> lowering log levels is not really solving the underlying issue
>> (whichever it may be), and that comment really should apply to any
>> project, not just Neutron. That said, we had examples where we took your
>> input and drove the right fix ourselves.
>>
>> We have a 'logging' tag for Neutron bugs that we use to identify these
>> type of cleanups. We'd need your attention to details to alert us of
>> issues like these; we'll take care of the right fixes. Currently, the
>> queue is pretty dry. If you can top it up, that'll be great. Going off
>> on a log cleanup rampage doesn't seem like the best course of action;
>> I'd rather knock issues one by one as they come, like the one you just
>> mentioned.
>>
>> [1] https://bugs.launchpad.net/neutron/+bugs?field.tag=logging
>>
>
> Tagged:
>
> https://bugs.launchpad.net/nova/+bug/1514935
>
> Although it must not be an official tag since it didn't auto-complete for
> me, that should be fixed.
>

It is...that's perhaps because you tried to tag whilst the bug was
targeting nova? btw thanks!


>
>
>>
>> Anyway, this is a plea for sanity in the logs. There are logging
>> guidelines for openstack [7]. Let's please abide by them. Let's keep
>> operators in mind when we're looking at logs and 

Re: [openstack-dev] [OpenStack-Infra] Report from Gerrit User Summit

2015-11-10 Thread David Pursehouse
On Tue, Nov 10, 2015 at 4:53 AM Sean Dague  wrote:

<...>

Is there any update on label:Code-Review<=-1,group:nova-core ? The group
> search support is documented, but as far as I can tell doesn't work
> except under very specific ldap configs.
> https://code.google.com/p/gerrit/issues/detail?id=3018
>
> That would be hugely helpful.
>
>
Khai started work on that but it's not working properly yet:

https://gerrit-review.googlesource.com/#/c/62880/

Let's see if we can get some eyes on it during this week's hackathon...
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Group-based-policy]

2015-11-10 Thread Sumit Naiksatam
On Tue, Nov 10, 2015 at 8:41 AM, Duarte Cardoso, Igor <
igor.duarte.card...@intel.com> wrote:

> Hi Ernesto,
>
>
>
> Let me answer the first question for you.
>
>
>
> You can use GBP without OpenDaylight.
>
>
>
> OpenDaylight has a separate Group-based Policy project, which might make
> you wonder if you need it to run OpenStack GBP. Don’t worry, you can deploy
> OpenStack GBP right away without OpenDaylight.
>
>
>
> Best regards,
>
>
>
> [image: intel]
>
>
>
> *Igor Duarte Cardoso*
>
> Software Engineer
>
> +353 61 777 858
>
> SIE1-2-170
>
> Intel Shannon Ltd.
>
> Dromore House, East Park
>
> Shannon, Co. Clare
>
> IRELAND
>
>
>
> *From:* Ernesto Valentino [mailto:ern.valent...@gmail.com]
> *Sent:* Tuesday, November 10, 2015 3:05 PM
> *To:* openstack-dev@lists.openstack.org
> *Subject:* [openstack-dev] [Group-based-policy]
>
>
>
> Dear sirs,
>
> I'm a student trying to testing Group Based Policy functionalities. I have
> some questions about it, because from the documentation is not clear to me
> what role assume opendaylight in the plug-in.
>
> I can use gbp only with openstack or is mandatory to use it with
> opendaylight? And next, if I want to add my personal nfv to gbp is there a
> possibility to do that or not ?
>


​Can you elaborate​ a little as to what exactly you mean by a “personal
NFV”?


Thanks so much for answering.
>
> Waiting for your kind reply.
>
>
>
> Best regards,
>
> Ernesto Valentino
>
> --
> Intel Research and Development Ireland Limited
> Registered in Ireland
> Registered Office: Collinstown Industrial Park, Leixlip, County Kildare
> Registered Number: 308263
>
> This e-mail and any attachments may contain confidential material for the
> sole use of the intended recipient(s). Any review or distribution by others
> is strictly prohibited. If you are not the intended recipient, please
> contact the sender and delete all copies.
>
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [release][stackalytics] possibly breaking bug closing statistics?

2015-11-10 Thread Ilya Shakhat
Doug,

You are right, there should not be any changes in the stats. Bugs are
mapped to release only by date, target milestone is not taken into account.
For resolved bugs metric Stackalytics uses 'date_fix_commited' field, for
filed bugs metric - 'date_created'.

Thanks,
Ilya

2015-11-10 0:53 GMT+03:00 Doug Hellmann :

> Stackalytics experts,
>
> When we roll out the launchpad process changes described in my earlier
> email [1], we will no longer be targeting bugs as part of closing them.
>
> Thierry raised a concern that this might break the way stackalytics
> calculates closed bug statistics for a cycle. Looking at the code
> in [2] and [3], I see it querying by date range and looking at the
> status, but not looking at the milestone to which the bug is targeted.
>
> Am I right in believing that we won't have any change in our
> statistics gathering if we go ahead with the current plan?
>
> Thanks,
> Doug
>
> [1]
> http://lists.openstack.org/pipermail/openstack-dev/2015-November/078301.html
> [2]
> http://git.openstack.org/cgit/openstack/stackalytics/tree/stackalytics/processor/main.py#n99
> [3]
> http://git.openstack.org/cgit/openstack/stackalytics/tree/stackalytics/processor/record_processor.py#n511
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] Can we get some sanity in the Neutron logs please?

2015-11-10 Thread Matt Riedemann



On 11/10/2015 1:10 PM, Matt Riedemann wrote:



On 11/10/2015 12:51 PM, Armando M. wrote:



On 10 November 2015 at 10:33, Matt Riedemann mailto:mrie...@linux.vnet.ibm.com>> wrote:

Let me qualify by saying I'm not a Neutron person.

We know that gate-tempest-dsvm-neutron-full is failing hard as of
the last 24 hours [1].

An error that's been showing up in tempest runs with neutron a lot
is:

"AssertionError: 0 == 0 : No IPv4 addresses found in: []"

So checking logstash [2] it's hitting a lot. It's only recent
because that failure message is new to Tempest in the last day or
so, but it has a lot of hits, so whatever it is, it's failing a lot.

So the next step is usually digging into service logs looking for
errors. I check the q-svc logs first. Not many errors but a
bazillion warnings for things not found (networks and devices). [3]

For example:

2015-11-10 17:13:02.542 WARNING neutron.plugins.ml2.rpc
[req-15a73753-1512-4689-9404-9658a0cd0c09 None None] Device
aaa525be-14eb-44a5-beb0-ed722896be93 requested by agent
ovs-agent-devstack-trusty-rax-iad-5785199 not found in database

2015-11-10 17:14:17.754 WARNING neutron.api.rpc.handlers.dhcp_rpc
[req-3d7e9848-6151-4780-907f-43f11a2a8545 None None] Network
b07ad9b2-e63e-4459-879d-3721074704e5 could not be found, it might
have been deleted concurrently.

Are several hundred of these warnings useful to an operator trying
to debug a problem? The point of the CI gate testing is to try and
simulate a production cloud environment. When something goes wrong,
you check the logs. With the amount of warning/error level logging
that is in the neutron logs, finding a real problem is like looking
for a needle in a haystack. Since everything is async, 404s are
expected when racing to delete a resource and they should be handled
gracefully.

Anyway, the server log isn't useful so I go digging in the agent
logs and stacktraces there are aplenty. [4]

Particularly this:

"Exception: Port tapcea51630-e1 is not ready, resync needed"

That's due to a new change landing in the last 24 hours [5]. But the
trace shows up over 16K times since it landed [6].

Checking the code, it's basically a loop processing events and when
it hits an event it can't handle, it punts (breaking the loop so you
don't process the other events after it - which is a bug), and the
code that eventually handles it is just catching all Exception and
tracing them out assuming they are really bad.

At this point, as a non-neutron person, i.e. not well versed in the
operations of neutron or how to debug it in great detail, I assume
something is bad here but I don't really know - and the logs are so
full of noise that I can't distinguish real failures.

I don't mean to pick on this particular change, but it's a good
example of a recent thing.

I'd like to know if this is all known issue or WIP type stuff. I've
complained about excessively noisey neutron logs in channel before
and I'm usually told that they are either necessary (for whatever
reason) or that rather than complain about the verbosity, I should
fix the race that is causing it - which is not likely to happen
since I don't have the async rpc happy nature of everything in
neutron in my head to debug it (I doubt many do).


I am not sure that's a fair statement: we usually pinpoint that just
lowering log levels is not really solving the underlying issue
(whichever it may be), and that comment really should apply to any
project, not just Neutron. That said, we had examples where we took your
input and drove the right fix ourselves.

We have a 'logging' tag for Neutron bugs that we use to identify these
type of cleanups. We'd need your attention to details to alert us of
issues like these; we'll take care of the right fixes. Currently, the
queue is pretty dry. If you can top it up, that'll be great. Going off
on a log cleanup rampage doesn't seem like the best course of action;
I'd rather knock issues one by one as they come, like the one you just
mentioned.

[1] https://bugs.launchpad.net/neutron/+bugs?field.tag=logging


Tagged:

https://bugs.launchpad.net/nova/+bug/1514935

Although it must not be an official tag since it didn't auto-complete
for me, that should be fixed.


Crap, nevermind that, my bad - I had created the bug against nova (force 
of habit) rather than neutron. Confirmed that 'logging' is a neutron tag 
in LP.







Anyway, this is a plea for sanity in the logs. There are logging
guidelines for openstack [7]. Let's please abide by them. Let's keep
operators in mind when we're looking at logs and be proactive about
making them useful (which includes more granular error handling and
less global try/except Exception: LOG.exception constructs).

Your point is duly noted. We have documented this, and we are being

Re: [openstack-dev] Regarding Designate Issues

2015-11-10 Thread Hayes, Graham
On 10/11/15 11:02, Sharma Swati6 wrote:
> Hi All,
> 
> I am trying to install Designate using Devstack-Kilo but facing the
> following issues while starting its services-
> 
>  1. WARNING keystone.common.wsgi [-] Could not find service: dns
>  2. WARNING keystone.common.wsgi [-] Could not find project: service
>  3. DEBUG keystone.middleware.core [-] Auth token not in the request
> header. Will not build auth context. process_request
> /opt/stack/keystone/keystone/middleware/core.py:223
> 
> I guess all the dependencies are installed for this now (oslo.service,
> Babel, etc) and PowerDNS is starting properly, but the designate
> services are still not starting.
> 
> My query is the same as has been posted here -
> https://ask.openstack.org/en/question/82412/configuring-designate-in-devstack/?answer=84419#post-id-84419
> 
> Please help me out with the next steps.
> 
> Thanks & Regards
> Swati Sharma
> System Engineer
> Tata Consultancy Services
> Ground to 8th Floors, Building No. 1 & 2,
> Skyview Corporate Park, Sector 74A,NH 8
> Gurgaon - 122 004,Haryana
> India
> Cell:- +91-9717238784
> Mailto: sharma.swa...@tcs.com
> Website: http://www.tcs.com
> 
> Experience certainty. IT Services
> Business Solutions
> Consulting
> 
> 
> =-=-=
> Notice: The information contained in this e-mail
> message and/or attachments to it may contain
> confidential or privileged information. If you are
> not the intended recipient, any dissemination, use,
> review, distribution, printing or copying of the
> information contained in this e-mail message
> and/or attachments to it are strictly prohibited. If
> you have received this communication in error,
> please notify us by reply e-mail or telephone and
> immediately and permanently delete the message
> and any attachments. Thank you
> 

It seems we have an issue with our devstack plugin -

I filed https://bugs.launchpad.net/designate/+bug/1514969
and we will start looking at it as soon as we can.

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [neutron] [gate] 35% failure rate for neutron tempest jobs

2015-11-10 Thread Sean Dague
On 11/10/2015 01:37 PM, Armando M. wrote:
> 
> 
> On 10 November 2015 at 09:49, Sean Dague  > wrote:
> 
> The neutron tempest jobs are now at a 35% failure rate:
> http://tinyurl.com/ne3ex4v (note, 35% is basically the worst possible
> fail rate, because it's just passing enough to land patches that cause
> that kind of fail on two test runs check/gate with a coin flip).
> 
> 
> Sean, thanks for the heads-up.
>  
> 
> 
> The failure is currently seen here -
> 
> http://logstash.openstack.org/#/dashboard/file/logstash.json?query=message:%5C%22No%20IPv4%20addresses%20found%20in:%20%5B%5D%5C%22
> 
> That is a new assert that was added in Tempest. However it was added in
> a path that expects there should be an IPv4 address. The fact that port
> is sometimes not returning one is problematic.
> https://review.openstack.org/#/c/241800/
> 
> The server via nova is returning an address here -
> 
> http://logs.openstack.org/76/243676/1/check/gate-tempest-dsvm-neutron-full/291e1d7/logs/tempest.txt.gz#_2015-11-10_17_14_35_465
> 
> But then when the port is polled here:
> 
> http://logs.openstack.org/76/243676/1/check/gate-tempest-dsvm-neutron-full/291e1d7/logs/tempest.txt.gz#_2015-11-10_17_14_35_527
> it comes back with {"ports": []}
> 
> 
> This can be contrasted with a working path where we do the similar
> action on the Server is active here -
> 
> http://logs.openstack.org/76/243676/1/check/gate-tempest-dsvm-neutron-full/291e1d7/logs/tempest.txt.gz#_2015-11-10_17_13_48_193
> 
> Then we verify the port -
> 
> http://logs.openstack.org/76/243676/1/check/gate-tempest-dsvm-neutron-full/291e1d7/logs/tempest.txt.gz#_2015-11-10_17_13_48_230
> 
> Which returns:
> 
>   Body: {"ports": [{"status": "ACTIVE", "binding:host_id":
> "devstack-trusty-rax-dfw-5784820", "allowed_address_pairs": [],
> "extra_dhcp_opts": [], "dns_assignment": [{"hostname":
> "host-10-100-0-3", "ip_address": "10.100.0.3", "fqdn":
> "host-10-100-0-3.openstacklocal."}], "device_owner": "compute:None",
> "port_security_enabled": true, "binding:profile": {}, "fixed_ips":
> [{"subnet_id": "147b1e65-3463-4965-8461-11b76a00dd99", "ip_address":
> "10.100.0.3"}], "id": "65c11c76-42fc-4010-bbb8-58996911803e",
> "security_groups": ["f2d48dcf-ea8d-4a7c-bf09-da37d3c2ee37"],
> "device_id": "b03bec85-fe69-4c0d-94e8-51753a8bebd5", "name": "",
> "admin_state_up": true, "network_id":
> "eb72d3af-f1a0-410b-8085-76cbe19ace90", "dns_name": "",
> "binding:vif_details": {"port_filter": true, "ovs_hybrid_plug": true},
> "binding:vnic_type": "normal", "binding:vif_type": "ovs", "tenant_id":
> "eab50a3d331c4db3a68f71d1ebdc41bf", "mac_address":
> "fa:16:3e:02:e4:ee"}]}
> 
> 
> HenryG suggested this might be related to the ERROR of "No more IP
> addresses available on network". However that ERROR is thrown a lot in
> neutron, and 60% of the times the tempest run is successful.
> 
> 
> This issue is currently stuck and needs neutron folks to engage to get
> us somewhere. Reverting the tempest patch which does the early
> verification might make this class of fail go away, but I think what
> it's done is surface a more fundamental bit where ports aren't active
> when the server is active, which may explain deeper races we've had over
> the years. So actually getting folks to dive in here would be really
> great.
> 
> 
> We'll dig into this more deeply. AFAIK, Nova servers won't go ACTIVE if
> the port isn't, so we might have a regression. That said, it's been on
> our radar to better synchronize actions that need to happen on port
> setup. Right now for instance, DHCP and L2 setup is uncoordinated and
> Kevin Benton has been looking into it.
> 
> That said, I wonder if reverting the tempest patch is the best course of
> action: we can then use Depends-on to test a Neutron fix and the revert
> of the revert together without causing the gate too much grief.
> 
> Thoughts?

So, I just stared at the Tempest patch again, and honestly, reverting it
isn't going to help anything.
https://github.com/openstack/tempest/blob/a1edb75d7901a9e338ab397d208a40c99c5fd9a1/tempest/scenario/manager.py#L760-L765

Because a revert just removes the assert len != 0

The next line is an assert len == 1 (which has been there for a long time)

So a len 0 will fail there as well. Which probably points to this being
a neutron regression entirely, we'd still be failing with the empty
ports list, it would just be an incredibly cryptic error of "Found
multiple IPv4 addresses: []" (which actually means found 0 port addresses).

The reason the change was pushed in Tempest was to make the fail
condition more clear.

-Sean

-- 
Sean Dague
http://dague.net

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe

Re: [openstack-dev] Can we get some sanity in the Neutron logs please?

2015-11-10 Thread Matt Riedemann



On 11/10/2015 12:51 PM, Armando M. wrote:



On 10 November 2015 at 10:33, Matt Riedemann mailto:mrie...@linux.vnet.ibm.com>> wrote:

Let me qualify by saying I'm not a Neutron person.

We know that gate-tempest-dsvm-neutron-full is failing hard as of
the last 24 hours [1].

An error that's been showing up in tempest runs with neutron a lot is:

"AssertionError: 0 == 0 : No IPv4 addresses found in: []"

So checking logstash [2] it's hitting a lot. It's only recent
because that failure message is new to Tempest in the last day or
so, but it has a lot of hits, so whatever it is, it's failing a lot.

So the next step is usually digging into service logs looking for
errors. I check the q-svc logs first. Not many errors but a
bazillion warnings for things not found (networks and devices). [3]

For example:

2015-11-10 17:13:02.542 WARNING neutron.plugins.ml2.rpc
[req-15a73753-1512-4689-9404-9658a0cd0c09 None None] Device
aaa525be-14eb-44a5-beb0-ed722896be93 requested by agent
ovs-agent-devstack-trusty-rax-iad-5785199 not found in database

2015-11-10 17:14:17.754 WARNING neutron.api.rpc.handlers.dhcp_rpc
[req-3d7e9848-6151-4780-907f-43f11a2a8545 None None] Network
b07ad9b2-e63e-4459-879d-3721074704e5 could not be found, it might
have been deleted concurrently.

Are several hundred of these warnings useful to an operator trying
to debug a problem? The point of the CI gate testing is to try and
simulate a production cloud environment. When something goes wrong,
you check the logs. With the amount of warning/error level logging
that is in the neutron logs, finding a real problem is like looking
for a needle in a haystack. Since everything is async, 404s are
expected when racing to delete a resource and they should be handled
gracefully.

Anyway, the server log isn't useful so I go digging in the agent
logs and stacktraces there are aplenty. [4]

Particularly this:

"Exception: Port tapcea51630-e1 is not ready, resync needed"

That's due to a new change landing in the last 24 hours [5]. But the
trace shows up over 16K times since it landed [6].

Checking the code, it's basically a loop processing events and when
it hits an event it can't handle, it punts (breaking the loop so you
don't process the other events after it - which is a bug), and the
code that eventually handles it is just catching all Exception and
tracing them out assuming they are really bad.

At this point, as a non-neutron person, i.e. not well versed in the
operations of neutron or how to debug it in great detail, I assume
something is bad here but I don't really know - and the logs are so
full of noise that I can't distinguish real failures.

I don't mean to pick on this particular change, but it's a good
example of a recent thing.

I'd like to know if this is all known issue or WIP type stuff. I've
complained about excessively noisey neutron logs in channel before
and I'm usually told that they are either necessary (for whatever
reason) or that rather than complain about the verbosity, I should
fix the race that is causing it - which is not likely to happen
since I don't have the async rpc happy nature of everything in
neutron in my head to debug it (I doubt many do).


I am not sure that's a fair statement: we usually pinpoint that just
lowering log levels is not really solving the underlying issue
(whichever it may be), and that comment really should apply to any
project, not just Neutron. That said, we had examples where we took your
input and drove the right fix ourselves.

We have a 'logging' tag for Neutron bugs that we use to identify these
type of cleanups. We'd need your attention to details to alert us of
issues like these; we'll take care of the right fixes. Currently, the
queue is pretty dry. If you can top it up, that'll be great. Going off
on a log cleanup rampage doesn't seem like the best course of action;
I'd rather knock issues one by one as they come, like the one you just
mentioned.

[1] https://bugs.launchpad.net/neutron/+bugs?field.tag=logging


Tagged:

https://bugs.launchpad.net/nova/+bug/1514935

Although it must not be an official tag since it didn't auto-complete 
for me, that should be fixed.





Anyway, this is a plea for sanity in the logs. There are logging
guidelines for openstack [7]. Let's please abide by them. Let's keep
operators in mind when we're looking at logs and be proactive about
making them useful (which includes more granular error handling and
less global try/except Exception: LOG.exception constructs).

Your point is duly noted. We have documented this, and we are being more
meticulous during reviews.


[1] http://tinyurl.com/ne3ex4v
[2]

http://logstash.openstack.org/#dashboard/file/logstash.json?query=message:%5C%22AssertionError:%200%20==%200%20:%20No%20IPv

Re: [openstack-dev] [all] Outcome of distributed lock manager discussion @ the summit

2015-11-10 Thread Joshua Harlow

Sean Dague wrote:

On 11/10/2015 05:12 AM, Thierry Carrez wrote:

Kevin Carter wrote:

I believe Clint already linked to
https://aphyr.com/posts/309-knossos-redis-and-linearizability or
similar - but 'known for general ease of use and reliability' is uhm,
a bold claim. Its worth comparing that (and the other redis writeups)
to this one: https://aphyr.com/posts/291-call-me-maybe-zookeeper. "Use
zookeeper, its mature".

Those write ups are from 2013 and with general improvements in Redis over the 
last two years I'd find it hard to believe that they're still relevant, however 
its worth testing to confirm if Redis is deemed as a viable option.


The openjdk is present on the same linux distributions, and has been
used in both open source and proprietary programs for decades. *what*
license implications are you speaking of?

The license issues would be related to deployers using Oracle java

which may or may not be needed by certain deployers for scale and
performance requirements. While I do not have specific performance
numbers at my fingertips to illustrate general performance issues using
zookeeper at scale with OpenJDK, I have, in the past, compared OpenJDK
to Oracle Java and found that Oracle java was quite a bit more stable
and packed far more performance capabilities.   I did find [
http://blog.cloud-benchmarks.org/2015/07/17/cassandra-write-performance-on-gce-and-aws.html
] which claims a 32% performance improvement with casandra using Oracle
Java 8 over OpenJDK on top of the fact that it was less prone to
crashes but that may not be entirely relevant to this case. Also
there's no denying that Oracle has a questionable history dealing with
Opensource projects regarding Java in the past and because performance
/ stability concerns may require the use of Oracle Java which will
undoubtedly

   come wit
h questionable license requirements.

I can't be suspected of JVM sympathies, and I find that a bit unfair.
I'll try to summarize:

1- ZooKeeper is a very good DLM

2- ZooKeeper is totally supported under OpenJDK and is run under this
configuration by large users (see Josh's other post for data)

3- /Some/ large Java stacks run faster / better under Oracle's non-free
JVM, but (1) there is no evidence that ZooKeeper is one of them and (2)
this is less and less true with modern JDKs (which are all built on top
of OpenJDK)

4- Still, some shops will prefer to stay out of Java stacks for various
reasons and need an alternative

I don't think anything in this discussion invalidates the compromise
(using tooz) we came up during the session. ZooKeeper can totally be the
tooz default in devstack (something has to be). If other tooz drivers
reach the same level of maturity one day, they could run in specific
tests and/or become the new default ?


And another good datapoint is Nova's had optional Zookeeper support for
service groups (landed in Folsom/Grizzly), which provides instantaneous
reporting of compute workers coming / going. That's been used to build a
lot of HA orchestration bits outside of Nova by a bunch of folks in the
NFV space.


Are they still using the nova zookeeper support (for service groups), 
from the analysis done by a coworker I'm not sure they can even use it 
anymore.


The following outline the reasons...

https://review.openstack.org/#/c/190322/ (the uberspec)

https://review.openstack.org/#/c/138607 (tooz for service groups) that 
one uncovered the following:


'For example, the Zookeeper driver uses evzookeeper which is no longer 
actively maintained and doesn't work with eventlet >= 0.17.1.'


The following has more details of this investigation:

http://lists.openstack.org/pipermail/openstack-dev/2015-May/063602.html



So it's also not just theory that Zookeeper is keeping up here, many
OpenStack deploys already are using it quite heavily.


Agreed that people do use it, just they might not be using it for 
service groups (unless they use an older release of nova) due to the 
above issues (which is why those specs were created, to resolve that and 
to fix the service group layer as a whole).




-Sean



__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] Can we get some sanity in the Neutron logs please?

2015-11-10 Thread Armando M.
On 10 November 2015 at 10:33, Matt Riedemann 
wrote:

> Let me qualify by saying I'm not a Neutron person.
>
> We know that gate-tempest-dsvm-neutron-full is failing hard as of the last
> 24 hours [1].
>
> An error that's been showing up in tempest runs with neutron a lot is:
>
> "AssertionError: 0 == 0 : No IPv4 addresses found in: []"
>
> So checking logstash [2] it's hitting a lot. It's only recent because that
> failure message is new to Tempest in the last day or so, but it has a lot
> of hits, so whatever it is, it's failing a lot.
>
> So the next step is usually digging into service logs looking for errors.
> I check the q-svc logs first. Not many errors but a bazillion warnings for
> things not found (networks and devices). [3]
>
> For example:
>
> 2015-11-10 17:13:02.542 WARNING neutron.plugins.ml2.rpc
> [req-15a73753-1512-4689-9404-9658a0cd0c09 None None] Device
> aaa525be-14eb-44a5-beb0-ed722896be93 requested by agent
> ovs-agent-devstack-trusty-rax-iad-5785199 not found in database
>
> 2015-11-10 17:14:17.754 WARNING neutron.api.rpc.handlers.dhcp_rpc
> [req-3d7e9848-6151-4780-907f-43f11a2a8545 None None] Network
> b07ad9b2-e63e-4459-879d-3721074704e5 could not be found, it might have been
> deleted concurrently.
>
> Are several hundred of these warnings useful to an operator trying to
> debug a problem? The point of the CI gate testing is to try and simulate a
> production cloud environment. When something goes wrong, you check the
> logs. With the amount of warning/error level logging that is in the neutron
> logs, finding a real problem is like looking for a needle in a haystack.
> Since everything is async, 404s are expected when racing to delete a
> resource and they should be handled gracefully.
>
> Anyway, the server log isn't useful so I go digging in the agent logs and
> stacktraces there are aplenty. [4]
>
> Particularly this:
>
> "Exception: Port tapcea51630-e1 is not ready, resync needed"
>
> That's due to a new change landing in the last 24 hours [5]. But the trace
> shows up over 16K times since it landed [6].
>
> Checking the code, it's basically a loop processing events and when it
> hits an event it can't handle, it punts (breaking the loop so you don't
> process the other events after it - which is a bug), and the code that
> eventually handles it is just catching all Exception and tracing them out
> assuming they are really bad.
>
> At this point, as a non-neutron person, i.e. not well versed in the
> operations of neutron or how to debug it in great detail, I assume
> something is bad here but I don't really know - and the logs are so full of
> noise that I can't distinguish real failures.
>
> I don't mean to pick on this particular change, but it's a good example of
> a recent thing.
>
> I'd like to know if this is all known issue or WIP type stuff. I've
> complained about excessively noisey neutron logs in channel before and I'm
> usually told that they are either necessary (for whatever reason) or that
> rather than complain about the verbosity, I should fix the race that is
> causing it - which is not likely to happen since I don't have the async rpc
> happy nature of everything in neutron in my head to debug it (I doubt many
> do).
>

I am not sure that's a fair statement: we usually pinpoint that just
lowering log levels is not really solving the underlying issue (whichever
it may be), and that comment really should apply to any project, not just
Neutron. That said, we had examples where we took your input and drove the
right fix ourselves.

We have a 'logging' tag for Neutron bugs that we use to identify these type
of cleanups. We'd need your attention to details to alert us of issues like
these; we'll take care of the right fixes. Currently, the queue is pretty
dry. If you can top it up, that'll be great. Going off on a log cleanup
rampage doesn't seem like the best course of action; I'd rather knock
issues one by one as they come, like the one you just mentioned.

[1] https://bugs.launchpad.net/neutron/+bugs?field.tag=logging


>
> Anyway, this is a plea for sanity in the logs. There are logging
> guidelines for openstack [7]. Let's please abide by them. Let's keep
> operators in mind when we're looking at logs and be proactive about making
> them useful (which includes more granular error handling and less global
> try/except Exception: LOG.exception constructs).
>

Your point is duly noted. We have documented this, and we are being more
meticulous during reviews.


> [1] http://tinyurl.com/ne3ex4v
> [2]
> http://logstash.openstack.org/#dashboard/file/logstash.json?query=message:%5C%22AssertionError:%200%20==%200%20:%20No%20IPv4%20addresses%20found%20in:%20%5B%5D%5C%22%20AND%20tags:%5C%22console%5C%22
> [3]
> http://logs.openstack.org/85/239885/2/gate/gate-tempest-dsvm-neutron-full/602d864/logs/screen-q-svc.txt.gz?level=TRACE
> [4]
> http://logs.openstack.org/85/239885/2/gate/gate-tempest-dsvm-neutron-full/602d864/logs/screen-q-agt.txt.gz?level=TRACE
> [5] https://r

[openstack-dev] [searchlight] Searching admin-only fields

2015-11-10 Thread McLellan, Steven
An issue discovered at the end of the Liberty cycle [1] meant that while still 
present in results (for admins) we had to disable searching of admin-only 
fields (TL;DR – a non-admin can fish for results to discover values for 
admin-only fields by searching e.g. "hostId": "a*"). We'd like to fix this, 
though I've only come up with three ideas. Any feedback would be very welcome.

1. There is a plugin for Elasticsearch called Shield (unfortunately only 
available commercially) that provides field level access by allowing roles to 
specify a list of fields they can search and see in results. The list is 
inclusive and can use wildcards. Shield has access to the parsed query and so 
can exclude any terms that refer to blocked fields (it treats them as having no 
results). It also disables the _all field for roles where a field list is 
specified. Even were we able to use it, Shield is more restrictive that we 
would like.

2. Create multiple indices (searchlight-admin, searchlight-user) and index some 
fields only in the admin index. This has several things going for it:
 * it's free
 * it's reasonably easy to understand
 * the code isn't complicated
 * it's secure
 * allows full text searching of _all

The strikes:
 * more indices (especially where someone configures indices specific to a 
plugin, though we could also allow plugins to not require an admin index)
 * double the data stored
 * greater risk of inconsistency (Elasticsearch doesn't have transactions)
 * complicates the effort for zero-downtime reindexing

3. Implement something similar to Shield's field control ourselves. We'd need 
to exclude fields from _all (because there's no good way to intercept queries 
against it), and scrub incoming queries against the admin-only field list.

Naively, it's not too hard to conceive of doing this, but I envisage a trickle 
of edge cases that work around the protection. For instance, to protect 
'hostId' one might take the incoming dictionary and look for all instances of 
'hostId', returning a 403 if it's found. This will find false positives (e.g. 
another type has a non-admin field called hostId), and (worse) false negatives; 
a query such as {"query_string": {"query": "hostId:a*"}} would escape it. Even 
scrubbing the entire input string would have holes ({"multimatch": {"fields": 
["hostI*"], "query": "aaabbccc"}}). We would probably be able to determine many 
of the issues, but I'd always worry about finding more holes. Shield has the 
advantage of being post-query parser.

4. ???

Conclusion:
My view is that a separate index is the only sensible way to do this, but I am 
willing to be swayed.

[1] https://bugs.launchpad.net/searchlight/+bug/1504399


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [oslo.messaging] State wrapping in the MessageHandlingServer

2015-11-10 Thread Joshua Harlow

Matthew Booth wrote:

My patch to MessageHandlingServer is currently being reverted because it
broke Nova tests:

https://review.openstack.org/#/c/235347/

Specifically it causes a number of tests to take a very long time to
execute, which ultimately results in the total build time limit being
exceeded. This is very easy to replicate. The
test 
nova.tests.functional.test_server_group.ServerGroupTest.test_boot_servers_with_affinity
is an example test which will always hit this issue. The problem is
that ServerGroupTest.setUp() does:

 self.compute2 = self.start_service('compute', host='host2')
 self.addCleanup(self.compute2.kill)

The problem with this is that start_service() adds a fixture which also
adds kill as a cleanup method. kill does stop(), wait(). This means that
the resulting call order is: start, stop, wait, stop, wait. The
redundant call to kill is obviously a wart, but I feel we should have
handled it anyway.

The problem is that we decided it should be possible to restart a
server. There are some unit tests in oslo.messaging that do this. It's
not clear to me that there are any projects which do this, but after
this experience I feel like it would be good to check before changing it :)

The implication of that is that after wait() the state wraps, and we're
now waiting on start() again. Consequently, when the second cleanup call
hangs.

We could fix Nova (at least the usage we have seen) by removing the
wrapping. After wait() if you want to start a server again you need to
create a new one.

So, to be specific, lets consider the following 2 call sequences:

1. start stop wait stop wait
2. start stop wait start stop wait

What should they do? The behaviours with and without wrapping are:

1. start stop wait stop wait
WRAP: start stop wait HANG HANG
NO WRAP: start stop wait NO-OP NO-OP

2. start stop wait start stop wait
WRAP: start stop wait start stop wait
NO WRAP: start stop wait NO-OP NO-OP NO-OP

I'll refresh my memory on what they did before my change in the morning.
Perhaps it might be simpler to codify the current behaviour, but iirc I
proposed this because it was previously undefined due to races.


I personally prefer not allowing restarting, its needless code 
complexity imho and a feature that people imho probably aren't using 
anyway (just create a new server object if u are doing this), so I'd be 
fine with doing the above NO WRAP and turning those into NO-OPs (and for 
example raising a runtime error in the case of start stop wait start ... 
to denote that restarting isn't recommended/possible). If we have a 
strong enough reason to really to start stop wait start ...


I might be convinced the code complexity is worth it but for now I'm not 
convinced...




Matt

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [neutron] [gate] 35% failure rate for neutron tempest jobs

2015-11-10 Thread Armando M.
On 10 November 2015 at 09:49, Sean Dague  wrote:

> The neutron tempest jobs are now at a 35% failure rate:
> http://tinyurl.com/ne3ex4v (note, 35% is basically the worst possible
> fail rate, because it's just passing enough to land patches that cause
> that kind of fail on two test runs check/gate with a coin flip).
>

Sean, thanks for the heads-up.


>
> The failure is currently seen here -
>
> http://logstash.openstack.org/#/dashboard/file/logstash.json?query=message:%5C%22No%20IPv4%20addresses%20found%20in:%20%5B%5D%5C%22
>
> That is a new assert that was added in Tempest. However it was added in
> a path that expects there should be an IPv4 address. The fact that port
> is sometimes not returning one is problematic.
> https://review.openstack.org/#/c/241800/
>
> The server via nova is returning an address here -
>
> http://logs.openstack.org/76/243676/1/check/gate-tempest-dsvm-neutron-full/291e1d7/logs/tempest.txt.gz#_2015-11-10_17_14_35_465
>
> But then when the port is polled here:
>
> http://logs.openstack.org/76/243676/1/check/gate-tempest-dsvm-neutron-full/291e1d7/logs/tempest.txt.gz#_2015-11-10_17_14_35_527
> it comes back with {"ports": []}
>
>
> This can be contrasted with a working path where we do the similar
> action on the Server is active here -
>
> http://logs.openstack.org/76/243676/1/check/gate-tempest-dsvm-neutron-full/291e1d7/logs/tempest.txt.gz#_2015-11-10_17_13_48_193
>
> Then we verify the port -
>
> http://logs.openstack.org/76/243676/1/check/gate-tempest-dsvm-neutron-full/291e1d7/logs/tempest.txt.gz#_2015-11-10_17_13_48_230
>
> Which returns:
>
>   Body: {"ports": [{"status": "ACTIVE", "binding:host_id":
> "devstack-trusty-rax-dfw-5784820", "allowed_address_pairs": [],
> "extra_dhcp_opts": [], "dns_assignment": [{"hostname":
> "host-10-100-0-3", "ip_address": "10.100.0.3", "fqdn":
> "host-10-100-0-3.openstacklocal."}], "device_owner": "compute:None",
> "port_security_enabled": true, "binding:profile": {}, "fixed_ips":
> [{"subnet_id": "147b1e65-3463-4965-8461-11b76a00dd99", "ip_address":
> "10.100.0.3"}], "id": "65c11c76-42fc-4010-bbb8-58996911803e",
> "security_groups": ["f2d48dcf-ea8d-4a7c-bf09-da37d3c2ee37"],
> "device_id": "b03bec85-fe69-4c0d-94e8-51753a8bebd5", "name": "",
> "admin_state_up": true, "network_id":
> "eb72d3af-f1a0-410b-8085-76cbe19ace90", "dns_name": "",
> "binding:vif_details": {"port_filter": true, "ovs_hybrid_plug": true},
> "binding:vnic_type": "normal", "binding:vif_type": "ovs", "tenant_id":
> "eab50a3d331c4db3a68f71d1ebdc41bf", "mac_address": "fa:16:3e:02:e4:ee"}]}
>
>
> HenryG suggested this might be related to the ERROR of "No more IP
> addresses available on network". However that ERROR is thrown a lot in
> neutron, and 60% of the times the tempest run is successful.
>
>
> This issue is currently stuck and needs neutron folks to engage to get
> us somewhere. Reverting the tempest patch which does the early
> verification might make this class of fail go away, but I think what
> it's done is surface a more fundamental bit where ports aren't active
> when the server is active, which may explain deeper races we've had over
> the years. So actually getting folks to dive in here would be really great.
>

We'll dig into this more deeply. AFAIK, Nova servers won't go ACTIVE if the
port isn't, so we might have a regression. That said, it's been on our
radar to better synchronize actions that need to happen on port setup.
Right now for instance, DHCP and L2 setup is uncoordinated and Kevin Benton
has been looking into it.

That said, I wonder if reverting the tempest patch is the best course of
action: we can then use Depends-on to test a Neutron fix and the revert of
the revert together without causing the gate too much grief.

Thoughts?

Armando


>
> -Sean
>
> --
> Sean Dague
> http://dague.net
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


[openstack-dev] Can we get some sanity in the Neutron logs please?

2015-11-10 Thread Matt Riedemann

Let me qualify by saying I'm not a Neutron person.

We know that gate-tempest-dsvm-neutron-full is failing hard as of the 
last 24 hours [1].


An error that's been showing up in tempest runs with neutron a lot is:

"AssertionError: 0 == 0 : No IPv4 addresses found in: []"

So checking logstash [2] it's hitting a lot. It's only recent because 
that failure message is new to Tempest in the last day or so, but it has 
a lot of hits, so whatever it is, it's failing a lot.


So the next step is usually digging into service logs looking for 
errors. I check the q-svc logs first. Not many errors but a bazillion 
warnings for things not found (networks and devices). [3]


For example:

2015-11-10 17:13:02.542 WARNING neutron.plugins.ml2.rpc 
[req-15a73753-1512-4689-9404-9658a0cd0c09 None None] Device 
aaa525be-14eb-44a5-beb0-ed722896be93 requested by agent 
ovs-agent-devstack-trusty-rax-iad-5785199 not found in database


2015-11-10 17:14:17.754 WARNING neutron.api.rpc.handlers.dhcp_rpc 
[req-3d7e9848-6151-4780-907f-43f11a2a8545 None None] Network 
b07ad9b2-e63e-4459-879d-3721074704e5 could not be found, it might have 
been deleted concurrently.


Are several hundred of these warnings useful to an operator trying to 
debug a problem? The point of the CI gate testing is to try and simulate 
a production cloud environment. When something goes wrong, you check the 
logs. With the amount of warning/error level logging that is in the 
neutron logs, finding a real problem is like looking for a needle in a 
haystack. Since everything is async, 404s are expected when racing to 
delete a resource and they should be handled gracefully.


Anyway, the server log isn't useful so I go digging in the agent logs 
and stacktraces there are aplenty. [4]


Particularly this:

"Exception: Port tapcea51630-e1 is not ready, resync needed"

That's due to a new change landing in the last 24 hours [5]. But the 
trace shows up over 16K times since it landed [6].


Checking the code, it's basically a loop processing events and when it 
hits an event it can't handle, it punts (breaking the loop so you don't 
process the other events after it - which is a bug), and the code that 
eventually handles it is just catching all Exception and tracing them 
out assuming they are really bad.


At this point, as a non-neutron person, i.e. not well versed in the 
operations of neutron or how to debug it in great detail, I assume 
something is bad here but I don't really know - and the logs are so full 
of noise that I can't distinguish real failures.


I don't mean to pick on this particular change, but it's a good example 
of a recent thing.


I'd like to know if this is all known issue or WIP type stuff. I've 
complained about excessively noisey neutron logs in channel before and 
I'm usually told that they are either necessary (for whatever reason) or 
that rather than complain about the verbosity, I should fix the race 
that is causing it - which is not likely to happen since I don't have 
the async rpc happy nature of everything in neutron in my head to debug 
it (I doubt many do).


Anyway, this is a plea for sanity in the logs. There are logging 
guidelines for openstack [7]. Let's please abide by them. Let's keep 
operators in mind when we're looking at logs and be proactive about 
making them useful (which includes more granular error handling and less 
global try/except Exception: LOG.exception constructs).


[1] http://tinyurl.com/ne3ex4v
[2] 
http://logstash.openstack.org/#dashboard/file/logstash.json?query=message:%5C%22AssertionError:%200%20==%200%20:%20No%20IPv4%20addresses%20found%20in:%20%5B%5D%5C%22%20AND%20tags:%5C%22console%5C%22
[3] 
http://logs.openstack.org/85/239885/2/gate/gate-tempest-dsvm-neutron-full/602d864/logs/screen-q-svc.txt.gz?level=TRACE
[4] 
http://logs.openstack.org/85/239885/2/gate/gate-tempest-dsvm-neutron-full/602d864/logs/screen-q-agt.txt.gz?level=TRACE

[5] https://review.openstack.org/#/c/164880/
[6] 
http://logstash.openstack.org/#dashboard/file/logstash.json?query=message:%5C%22Exception:%20Port%5C%22%20AND%20message:%5C%22is%20not%20ready,%20resync%20needed%5C%22%20AND%20tags:%5C%22screen-q-agt.txt%5C%22
[7] 
http://specs.openstack.org/openstack/openstack-specs/specs/log-guidelines.html


--

Thanks,

Matt Riedemann


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


[openstack-dev] [oslo.messaging] State wrapping in the MessageHandlingServer

2015-11-10 Thread Matthew Booth
My patch to MessageHandlingServer is currently being reverted because it
broke Nova tests:

https://review.openstack.org/#/c/235347/

Specifically it causes a number of tests to take a very long time to
execute, which ultimately results in the total build time limit being
exceeded. This is very easy to replicate. The
test 
nova.tests.functional.test_server_group.ServerGroupTest.test_boot_servers_with_affinity
is an example test which will always hit this issue. The problem is
that ServerGroupTest.setUp() does:

self.compute2 = self.start_service('compute', host='host2')
self.addCleanup(self.compute2.kill)

The problem with this is that start_service() adds a fixture which also
adds kill as a cleanup method. kill does stop(), wait(). This means that
the resulting call order is: start, stop, wait, stop, wait. The redundant
call to kill is obviously a wart, but I feel we should have handled it
anyway.

The problem is that we decided it should be possible to restart a server.
There are some unit tests in oslo.messaging that do this. It's not clear to
me that there are any projects which do this, but after this experience I
feel like it would be good to check before changing it :)

The implication of that is that after wait() the state wraps, and we're now
waiting on start() again. Consequently, when the second cleanup call hangs.

We could fix Nova (at least the usage we have seen) by removing the
wrapping. After wait() if you want to start a server again you need to
create a new one.

So, to be specific, lets consider the following 2 call sequences:

1. start stop wait stop wait
2. start stop wait start stop wait

What should they do? The behaviours with and without wrapping are:

1. start stop wait stop wait
WRAP: start stop wait HANG HANG
NO WRAP: start stop wait NO-OP NO-OP

2. start stop wait start stop wait
WRAP: start stop wait start stop wait
NO WRAP: start stop wait NO-OP NO-OP NO-OP

I'll refresh my memory on what they did before my change in the morning.
Perhaps it might be simpler to codify the current behaviour, but iirc I
proposed this because it was previously undefined due to races.

Matt
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [OpenStack-Infra] Report from Gerrit User Summit

2015-11-10 Thread Spencer Krum
Thanks for the report Jim. I got excited about polygerrit and deployed
it sortof... pointing at review-dev.o.o in case anyone wants to kick the
tires.

http://polygerrit.nibalizer.com/q/status:open

After the gerrit upgrade next week we can point it at review.o.o. I
think it would be good for infra to maintain a deploy of this software,
even though it is immature right now.

-- 
  Spencer Krum
  n...@spencerkrum.com

On Tue, Nov 10, 2015, at 05:18 AM, Markus Zoeller wrote:
> David Pursehouse  wrote on 11/10/2015
> 07:39:32 
> AM:
> 
> > From: David Pursehouse 
> > To: OpenStack Development Mailing List 
> 
> > Cc: openstack-in...@lists.openstack.org
> > Date: 11/10/2015 07:40 AM
> > Subject: Re: [openstack-dev] [OpenStack-Infra] Report from Gerrit User 
> Summit
> > [...]
> > We're looking into the possibility of enabling only enough of the 
> > notedb to make hashtags work in 2.12.
> > [...]
> 
> +1 
> 
> In case someone is wondering what these hashtags are, it's basically
> Gerrits term for a folksonomy which usually uses the term "labels" or
> "tags" (not git tags). This was previously discussed on the ML with [1].
> 
> [1] [openstack-dev] [infra][all] Reviews with a prio label?:
> http://lists.openstack.org/pipermail/openstack-dev/2015-October/077532.html
> 
> Regards, Markus Zoeller (markus_z)
> 
> 
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe:
> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [TripleO] [Ironic] Let's stop hijacking other projects' OSC namespaces

2015-11-10 Thread Ben Nemec
On 11/10/2015 11:18 AM, Dmitry Tantsur wrote:
> On 11/10/2015 06:08 PM, Ben Nemec wrote:
>> On 11/10/2015 10:28 AM, John Trowbridge wrote:
>>>
>>>
>>> On 11/10/2015 10:43 AM, Ben Nemec wrote:
 On 11/10/2015 05:26 AM, John Trowbridge wrote:
>
>
> On 11/09/2015 07:44 AM, Dmitry Tantsur wrote:
>> Hi OOO'ers, hopefully the subject caught your attentions :)
>>
>> Currently, tripleoclient exposes several commands in "openstack
>> baremetal" and "openstack baremetal introspection" namespaces belonging
>> to ironic and ironic-inspector accordingly. TL;DR of this email is to
>> deprecate them and move to TripleO-specific namespaces. Read on to know
>> why.
>>
>> Problem
>> ===
>>
>> I realized that we're doing a wrong thing when people started asking me
>> why "baremetal introspection start" and "baremetal introspection bulk
>> start" behave so differently (the former is from ironic-inspector, the
>> latter is from tripleoclient). The problem with TripleO commands is that
>> they're highly opinionated workflows commands, but there's no way a user
>> can distinguish them from general-purpose ironic/ironic-inspector
>> commands. The way some of them work is not generic enough ("baremetal
>> import"), or uses different defaults from an upstream project
>> ("configure boot"), or does something completely unacceptable upstream
>> (e.g. the way "introspection bulk start" deals with node states).
>>
>> So, here are commands that tripleoclient exposes with my comments:
>>
>> 1. baremetal instackenv validate
>>
>>   This command assumes there's an "baremetal instackenv" object, while
>> instackenv is a tripleo-specific file format.
>>
>> 2. baremetal import
>>
>>   This command supports a limited subset of ironic drivers and driver
>> properties, only those known to os-cloud-config.
>>
>> 3. baremetal introspection bulk start
>>
>>   This command does several bad (IMO) things:
>>   a. Messes with ironic node states
>>   b. Operates implicitly on all nodes (in a wrong state)
>>   c. Defaults to polling
>>
>
> I have considered this whole command as a bug for a while now. I
> understand what we were trying to do and why, but it is pretty bad to
> hijack another project's namespace with a command that would get a firm
> -2 there.
>
>> 4. baremetal show capabilities
>>
>>   This is the only commands that is generic enough and could actually
>> make it to ironicclient itself.
>>
>> 5. baremetal introspection bulk status
>>
>>   See "bulk start" above.
>>
>> 6. baremetal configure ready state
>>
>>   First of all, this and the next command use "baremetal configure"
>> prefix. I would not promise we'll never start using it in ironic,
>> breaking the whole TripleO.
>>
>>   Seconds, it's actually DELL-specific.
>>
>> 7. baremetal configure boot
>>
>>   This one is nearly ok, but it defaults to local boot, which is not an
>> upstream default. Default values for images may not work outside of
>> TripleO as well.
>>
>> Proposal
>> 
>>
>> As we already have "openstack undercloud" and "openstack overcloud"
>> prefixes for TripleO, I suggest we move these commands under "openstack
>> overcloud nodes" namespace. So we end up with:
>>
>>   overcloud nodes import
>>   overcloud nodes configure ready state --drac
>>   overcloud nodes configure boot
>>
>> As you see, I require an explicit --drac argument for "ready state"
>> command. As to the remaining commands:
>>
>> 1. baremetal introspection status --all
>>
>>This is fine to move to inspector-client, as inspector knows which
>> nodes are/were on introspection. We'll need a new API though.
>>
>> 2. baremetal show capabilities
>>
>>We'll have this or similar command in ironic, hopefully this cycle.
>>
>> 3. overcloud nodes introspect --poll --allow-available
>>
>>I believe that we need to make 2 things explicit in this replacement
>> for "introspection bulk status": polling and operating on "available"
>> nodes.
>
> I am not totally convinced that we gain a huge amount by hiding the
> state manipulation in this command. We need to move that logic to
> tripleo-common anyways, so I think it is worth considering splitting it
> from the introspect command.
>
> Dmitry and I discussed briefly at summit having the ability to pass a
> list of nodes to the inspector client for introspection as well. So if
> we separated out the bulk state manipulation bit, we could just use that.
>
> I get that this is going in the opposite direction of the original
> intention of lowering the amount of commands needed to get a functional
> d

Re: [openstack-dev] [nova] Proposal to add Sylvain Bauza to nova-core

2015-11-10 Thread melanie witt
On Nov 6, 2015, at 7:32, John Garbutt  wrote:

> Hi,
> 
> I propose we add Sylvain Bauza[1] to nova-core.
> 
> Over the last few cycles he has consistently been doing great work,
> including some quality reviews, particularly around the Scheduler.
> 
> Please respond with comments, +1s, or objections within one week.
> 
> Many thanks,
> John
> 
> [1] 
> http://stackalytics.com/?module=nova-group&user_id=sylvain-bauza&release=all
> 
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

+1

-melanie (irc: melwitt)







signature.asc
Description: Message signed with OpenPGP using GPGMail
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [nova] Proposal to add Alex Xu to nova-core

2015-11-10 Thread melanie witt
On Nov 6, 2015, at 7:32, John Garbutt  wrote:

> Hi,
> 
> I propose we add Alex Xu[1] to nova-core.
> 
> Over the last few cycles he has consistently been doing great work,
> including some quality reviews, particularly around the API.
> 
> Please respond with comments, +1s, or objections within one week.
> 
> Many thanks,
> John
> 
> [1]http://stackalytics.com/?module=nova-group&user_id=xuhj&release=all
> 
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

+1

-melanie (irc: melwitt)







signature.asc
Description: Message signed with OpenPGP using GPGMail
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


[openstack-dev] [all] devstack-gate jobs that will break at Mitaka-1

2015-11-10 Thread Sean Dague
Here is the weekly list of jobs that will no longer work at Mitaka-1
when we remove extras.d support in devstack (and the number of runs in
the last 48 hours):

gate-congress-dsvm-api  652
gate-murano-congress-devstack-dsvm  284
gate-cue-integration-dsvm-rabbitmq  176
gate-designate-dsvm-powerdns168
gate-designate-dsvm-bind9   120
gate-rally-dsvm-cue-rabbitmq112
gate-solum-devstack-dsvm96
gate-solum-devstack-dsvm-centos748
gate-rally-dsvm-designate-designate 48
gate-tempest-dsvm-docker-centos724

Owners of those jobs are encouraged to fix them ASAP. There is less than
a month now before these will no longer work.

-Sean

-- 
Sean Dague
http://dague.net

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [TripleO] [Ironic] Let's stop hijacking other projects' OSC namespaces

2015-11-10 Thread Zane Bitter

On 09/11/15 07:44, Dmitry Tantsur wrote:


As we already have "openstack undercloud" and "openstack overcloud"
prefixes for TripleO, I suggest we move these commands under "openstack
overcloud nodes" namespace. So we end up with:

  overcloud nodes import
  overcloud nodes configure ready state --drac
  overcloud nodes configure boot


Terminology around these machines is always confusing, because they are 
related to both the undercloud (in the sense of being resources managed 
the undercloud's Ironic) and the overcloud (in the sense of being the 
substrate on which the overcloud is deployed).


I have probably been using the terminology wrongly, but FWIW it was 
extremely surprising to me to see these fall into the overcloud bucket, 
given that at the time they're used the overcloud doesn't even exist. (I 
guess I would have gone with "undercloud baremetal", although I can see 
how that could potentially be equally confusing.)


cheers,
Zane.

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [stable] Making stable maintenance its own OpenStack project team

2015-11-10 Thread Ihar Hrachyshka

Thierry Carrez  wrote:


Hi everyone,

A few cycles ago we set up the Release Cycle Management team which was a
bit of a frankenteam of the things I happened to be leading: release
management, stable branch maintenance and vulnerability management.
While you could argue that there was some overlap between those
functions (as in, "all these things need to be released") logic was not
the primary reason they were put together.

When the Security Team was created, the VMT was spinned out of the
Release Cycle Management team and joined there. Now I think we should
spin out stable branch maintenance as well:

* A good chunk of the stable team work used to be stable point release
management, but as of stable/liberty this is now done by the release
management team and triggered by the project-specific stable maintenance
teams, so there is no more overlap in tooling used there

* Following the kilo reform, the stable team is now focused on defining
and enforcing a common stable branch policy[1], rather than approving
every patch. Being more visible and having more dedicated members can
only help in that very specific mission

* The release team is now headed by Doug Hellmann, who is focused on
release management and does not have the history I had with stable
branch policy. So it might be the right moment to refocus release
management solely on release management and get the stable team its own
leadership

* Empowering that team to make its own decisions, giving it more
visibility and recognition will hopefully lead to more resources being
dedicated to it

* If the team expands, it could finally own stable branch health and
gate fixing. If that ends up all falling under the same roof, that team
could make decisions on support timeframes as well, since it will be the
primary resource to make that work

So.. good idea ? bad idea ? What do current stable-maint-core[2] members
think of that ? Who thinks they could step up to lead that team ?

[1] http://docs.openstack.org/project-team-guide/stable-branches.html
[2] https://review.openstack.org/#/admin/groups/530,members

--
Thierry Carrez (ttx)


Lots of great thoughts in the thread, trying to catch up with it.

I lean towards supporting the split; if anything, to make responsibilities  
more clear. Currently, [especially since we disintegrated the monolithic  
stable-maint team into per-project teams] I struggle to reach the whole  
stable team and get some final decisions on unclear matters set. F.e. I  
sent several [stable] emails to this ML recently that would require some  
more input from outside neutron-stable-maint that I am part of, and I don’t  
see enough responses, neither a way to understand where the discussion  
leaned and what my next steps are.


I believe if we have a dedicated team for the effort, we would at least be  
able to get more attention to such discussions, and hopefully would be able  
to make final calls on policies common to the whole project.


I am not sure we actually *require* PTL for that to happen, though I also  
don’t see it a bad thing if we do have the role. [And if you wonder, I  
won’t be able to run for PTL for the project but I am happy to help the  
cause.]


I think the main responsibility of the team would be enabling other,  
per-project teams to do their job. Meaning, handling common policy  
questions; providing tools where needed; spearheading and monitoring  
initiatives that would make stable branches less broken; providing guidance  
to project teams on how to manage stable branches properly, thru docs and  
irc/email support; making calls on support phases and their length; etc.


Obviously it is not a full blown software project; separating concerns  
seems justified here nevertheless.


Ihar

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


[openstack-dev] [neutron] [gate] 35% failure rate for neutron tempest jobs

2015-11-10 Thread Sean Dague
The neutron tempest jobs are now at a 35% failure rate:
http://tinyurl.com/ne3ex4v (note, 35% is basically the worst possible
fail rate, because it's just passing enough to land patches that cause
that kind of fail on two test runs check/gate with a coin flip).

The failure is currently seen here -
http://logstash.openstack.org/#/dashboard/file/logstash.json?query=message:%5C%22No%20IPv4%20addresses%20found%20in:%20%5B%5D%5C%22

That is a new assert that was added in Tempest. However it was added in
a path that expects there should be an IPv4 address. The fact that port
is sometimes not returning one is problematic.
https://review.openstack.org/#/c/241800/

The server via nova is returning an address here -
http://logs.openstack.org/76/243676/1/check/gate-tempest-dsvm-neutron-full/291e1d7/logs/tempest.txt.gz#_2015-11-10_17_14_35_465

But then when the port is polled here:
http://logs.openstack.org/76/243676/1/check/gate-tempest-dsvm-neutron-full/291e1d7/logs/tempest.txt.gz#_2015-11-10_17_14_35_527
it comes back with {"ports": []}


This can be contrasted with a working path where we do the similar
action on the Server is active here -
http://logs.openstack.org/76/243676/1/check/gate-tempest-dsvm-neutron-full/291e1d7/logs/tempest.txt.gz#_2015-11-10_17_13_48_193

Then we verify the port -
http://logs.openstack.org/76/243676/1/check/gate-tempest-dsvm-neutron-full/291e1d7/logs/tempest.txt.gz#_2015-11-10_17_13_48_230

Which returns:

  Body: {"ports": [{"status": "ACTIVE", "binding:host_id":
"devstack-trusty-rax-dfw-5784820", "allowed_address_pairs": [],
"extra_dhcp_opts": [], "dns_assignment": [{"hostname":
"host-10-100-0-3", "ip_address": "10.100.0.3", "fqdn":
"host-10-100-0-3.openstacklocal."}], "device_owner": "compute:None",
"port_security_enabled": true, "binding:profile": {}, "fixed_ips":
[{"subnet_id": "147b1e65-3463-4965-8461-11b76a00dd99", "ip_address":
"10.100.0.3"}], "id": "65c11c76-42fc-4010-bbb8-58996911803e",
"security_groups": ["f2d48dcf-ea8d-4a7c-bf09-da37d3c2ee37"],
"device_id": "b03bec85-fe69-4c0d-94e8-51753a8bebd5", "name": "",
"admin_state_up": true, "network_id":
"eb72d3af-f1a0-410b-8085-76cbe19ace90", "dns_name": "",
"binding:vif_details": {"port_filter": true, "ovs_hybrid_plug": true},
"binding:vnic_type": "normal", "binding:vif_type": "ovs", "tenant_id":
"eab50a3d331c4db3a68f71d1ebdc41bf", "mac_address": "fa:16:3e:02:e4:ee"}]}


HenryG suggested this might be related to the ERROR of "No more IP
addresses available on network". However that ERROR is thrown a lot in
neutron, and 60% of the times the tempest run is successful.


This issue is currently stuck and needs neutron folks to engage to get
us somewhere. Reverting the tempest patch which does the early
verification might make this class of fail go away, but I think what
it's done is surface a more fundamental bit where ports aren't active
when the server is active, which may explain deeper races we've had over
the years. So actually getting folks to dive in here would be really great.


-Sean

-- 
Sean Dague
http://dague.net

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [stable] Making stable maintenance its own OpenStack project team

2015-11-10 Thread Matthew Treinish
On Tue, Nov 10, 2015 at 10:13:09AM +0100, Thierry Carrez wrote:
> Matthew Treinish wrote:
> > [...]
> > The discussion here is about the cross project effort around stable 
> > branches,
> > which by design is a more limited scope now. Right now the cross project 
> > effort
> > around stable branch policy is really 2 things (both of which ttx already
> > mentioned):
> > 
> > 1. Keeping the gates working on the stable branches
> > 2. Defining and enforcing stable branch policy.
> 
> Right. My argument is that we need to expand the work on point (2), and
> then expand the work on point (1), and that empowering them by giving
> them their own team can only help.

Well you know my position on this well, but without people working on #1 policy
changes don't actually do a whole lot. You can look at our "experiment" in a
longer support window last year for proof of that. I don't think coming about
it from the other direction will actually change who looks at gate issues and
how we fail to scale there.

> 
> For point (2), currently the stable branch policy is not really enforced
> as much as it should. I think it's important to have a standard
> definition across OpenStack of what a "stable branch" means in terms of
> acceptable backports, support periods, frequency of point releases, etc.
> With the big tent we see a lot of variance in the stable branch policy,
> and what used to be mostly self-policed is likely to now require a lot
> more attention from policy overlords.
> 
> I'm not seeing that happening with the current team structure, which is
> why I proposed the change. Once empowered with their own team, it will
> be easier to justify dedicating resources to it, and I hope that will go
> all the way up to owning gate fixing when stable branch breaks (point
> (1) which is currently mostly done off-team).
> 
> > The only lever on #2 is that the global stable-maint-core is the only group
> > which has add permissions to the per project stable core groups. (also the
> > stable branch policy wiki, but that rarely changes)
> 
> Actually we'll likely create a "follows-stable-policy" tag that would be
> granted by this team. I expect that to be a good lever as well.
> 
> > [...] 
> > This is my whole argument that creating a new team doesn't do anything. 
> > Living
> > under rel-mgt, some other project, or creating a new one isn't going to 
> > change
> > the fact that cross-project stable maint is the same 2 tasks which basically
> > nobody cares about, which TBH your email is just an indication of. Even if 
> > we
> > wanted to grow to something beyond these 2 tasks they would still be the 
> > core of
> > whatever it becomes and a lack of people caring about them will just block 
> > any
> > potential scope growth.
> 
> I have evidence that making that group its own project team will result
> in additional resources, and increase of focus from existing resources.
> The alternative is to kill stable branches altogether. But as the other
> thread shows, there is still interest in them, so providing the right
> vessel for resources to be dedicated to it is still worth a try.
> 
> I'm arguing that one of the reasons "nobody cares" is because it's
> always been the 5th wheel of release management, and making it a
> first-class citizen could unlock the situation. I don't see harm in
> trying. Do you ?
> 

No, there is no harm in trying. Maybe I'm just more jaded and bitter, but I just
don't see how making it a separate thing is going to change anything. I don't
expect all the stable unicorns and fairies to come out of the forest and solve
all our problems once they finally have a separate project team. The work has
always been there a dedicated project team doesn't magically create interest
from my POV. If people were truly interested in helping they'd be contributing
already.

-Matt Treinish


signature.asc
Description: PGP signature
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


[openstack-dev] Cross-Project Meeting SKIPPED, Tue Nov 10th, 21:00 UTC

2015-11-10 Thread Mike Perez
Hi all!

As discussed in the previous cross-project meeting [1], we will be skipping
meetings if there is no agenda items to discuss. There are no items to discuss
this time around, but someone can still call for a meeting by adding an agenda
item for November 17th [2].

To keep up with the cross-project items we still recommend the OpenStack Dev
list [3].

If you're unable to keep up with the Dev list, there is also the Dev summary
which has been posted in the OpenStack newsletter [4], but will soon be
separated from the newsletter to make it more visible.

[1] - 
http://eavesdrop.openstack.org/meetings/crossproject/2015/crossproject.2015-11-03-21.01.log.html#l-57
[2] - 
https://wiki.openstack.org/wiki/Meetings/CrossProjectMeeting#Proposed_agenda
[3] - http://lists.openstack.org/pipermail/openstack-dev/
[4] - 
http://www.openstack.org/blog/2015/11/openstack-weekly-community-newsletter-oct-31-nov-6/#dev-digest

-- 
Mike Perez

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [TripleO] [Ironic] Let's stop hijacking other projects' OSC namespaces

2015-11-10 Thread Ben Nemec
On 11/10/2015 10:40 AM, Dmitry Tantsur wrote:
> On 11/10/2015 05:18 PM, Ben Nemec wrote:
>> +1 to moving anything we can into Ironic itself.
>>
>> I do want to note that if we rename anything, we can't just rip and
>> replace.  We have users of the current commands, and we need to
>> deprecate those to give people a chance to move to the new ones.
> 
> Definitely. I think I used word deprecation somewhere in this long letter :)

Cool, just wanted to make sure we were all on the same page. :-)

> 
>>
>> A few more thoughts inline.
>>
>> On 11/09/2015 06:44 AM, Dmitry Tantsur wrote:
>>> Hi OOO'ers, hopefully the subject caught your attentions :)
>>>
>>> Currently, tripleoclient exposes several commands in "openstack
>>> baremetal" and "openstack baremetal introspection" namespaces belonging
>>> to ironic and ironic-inspector accordingly. TL;DR of this email is to
>>> deprecate them and move to TripleO-specific namespaces. Read on to know why.
>>>
>>> Problem
>>> ===
>>>
>>> I realized that we're doing a wrong thing when people started asking me
>>> why "baremetal introspection start" and "baremetal introspection bulk
>>> start" behave so differently (the former is from ironic-inspector, the
>>> latter is from tripleoclient). The problem with TripleO commands is that
>>> they're highly opinionated workflows commands, but there's no way a user
>>> can distinguish them from general-purpose ironic/ironic-inspector
>>> commands. The way some of them work is not generic enough ("baremetal
>>> import"), or uses different defaults from an upstream project
>>> ("configure boot"), or does something completely unacceptable upstream
>>> (e.g. the way "introspection bulk start" deals with node states).
>>>
>>> So, here are commands that tripleoclient exposes with my comments:
>>>
>>> 1. baremetal instackenv validate
>>>
>>>This command assumes there's an "baremetal instackenv" object, while
>>> instackenv is a tripleo-specific file format.
>>>
>>> 2. baremetal import
>>>
>>>This command supports a limited subset of ironic drivers and driver
>>> properties, only those known to os-cloud-config.
>>
>> True, although I feel like an "import from JSON" feature would not be
>> inappropriate for inclusion in Ironic.  I can't believe that we're the
>> only ones who would be interested in mass importing nodes from a very
>> common format like this.
> 
> I would warmly welcome such command, but it should not require patching 
> os-cloud-config every time we add a new driver or driver property.

Yeah, I wonder if we should just drop the os-cloud-config abstraction
and have the JSON contain the actual node parameters.  A bunch of the
drivers have driver-specific configuration options anyway, so it's not
like you can just change the driver name without altering the other bits
of the configuration anyway.

The only drawback to that is if a driver changed the name of a
parameter, but if this lived in Ironic you could just add a name mapping
to the import code at the same time, whereas having it in
os-cloud-config you'd still break, but there's an extra layer of
indirection to getting it fixed.

The more I think about it the more I like it.  Not sure I'm going to
have time to follow up though. :-/

> 
>>
>>>
>>> 3. baremetal introspection bulk start
>>>
>>>This command does several bad (IMO) things:
>>>a. Messes with ironic node states
>>>b. Operates implicitly on all nodes (in a wrong state)
>>
>> I thought that was fixed?  It used to try to introspect nodes that were
>> in an invalid state (like active), but it shouldn't anymore.
> 
> It introspects nodes in "available" state, which is a rude violation of 
> the ironic state machine ;)
> 
>>
>> Unless your objection is that it introspects things in an available
>> state, which I think has to do with the state Ironic puts (or used to
>> put) nodes in after registration.  In any case, this one likely requires
>> some more discussion over how it should work.
> 
> Well, if we upgrade baremetal API version we used from 1.6 (essentially 
> Kilo) to Liberty one, nodes would appear in a new ENROLL state. I've 
> started a patch for it, but I don't have time to finish. Anyone is free 
> to overtake: https://review.openstack.org/#/c/235158/

This seems like the right thing to do all around.  It allows us to
introspect newly registered nodes without breaking the Ironic state
machine.  I've subscribed to the review at least, will see if I have
time to do anything with it.

> 
>>
>>>c. Defaults to polling
>>>
>>> 4. baremetal show capabilities
>>>
>>>This is the only commands that is generic enough and could actually
>>> make it to ironicclient itself.
>>>
>>> 5. baremetal introspection bulk status
>>>
>>>See "bulk start" above.
>>>
>>> 6. baremetal configure ready state
>>>
>>>First of all, this and the next command use "baremetal configure"
>>> prefix. I would not promise we'll never start using it in ironic,
>>> breaking the whole TripleO.
>>>
>>>Secon

Re: [openstack-dev] [Fuel] Running Fuel node as non-superuser

2015-11-10 Thread Dmitry Nikishov
Bartolomiej, Adam,
Stanislaw is correct. And this is going to be ported to master. The goal
currently is to reach an agreement on the implementation so that there's
going to be a some kinf of compatibility during upgrades.

Stanislaw,
Do I understand correctly that you propose using something like sucap to
launch from root, switch to a different user and then drop capabilities
which are not required?

On Tue, Nov 10, 2015 at 3:11 AM, Stanislaw Bogatkin 
wrote:

> Bartolomiej, it's customer-related patches, they, I think, have to be done
> for 6.1 prior to 8+ release.
>
> Dmitry, it's nice to hear about it. Did you consider to use linux
> capabilities on fuel-related processes instead of just using non-extended
> POSIX privileged/non-privileged permission checks?
>
> On Tue, Nov 10, 2015 at 10:11 AM, Bartlomiej Piotrowski <
> bpiotrow...@mirantis.com> wrote:
>
>> We don't develop features for already released versions… It should be
>> done for master instead.
>>
>> BP
>>
>> On Tue, Nov 10, 2015 at 7:02 AM, Adam Heczko 
>> wrote:
>>
>>> Dmitry,
>>> +1
>>>
>>> Do you plan to port your patchset to future Fuel releases?
>>>
>>> A.
>>>
>>> On Tue, Nov 10, 2015 at 12:14 AM, Dmitry Nikishov <
>>> dnikis...@mirantis.com> wrote:
>>>
 Hey guys.

 I've been working on making Fuel not to rely on superuser privileges
 at least for day-to-day operations. These include:
 a) running Fuel services (nailgun, astute etc)
 b) user operations (create env, deploy, update, log in)

 The reason for this is that many security policies simply do not
 allow root access (especially remote) to servers/environments.

 This feature/enhancement means that anything that currently is being
 run under root, will be evaluated and, if possible, put under a
 non-privileged
 user. This also means that remote root access will be disabled.
 Instead, users will have to log in with "fueladmin" user.

 Together with Omar  we've put together a blueprint[0] and a
 spec[1] for this feature. I've been developing this for Fuel 6.1, so
 there
 are two patches into fuel-main[2] and fuel-library[3] that can give you
 an
 impression of current approach.

 These patches do following:
 - Add fuel-admin-user package, which creates 'fueladmin'
 - Make all other fuel-* packages depend on fuel-admin-user
 - Put supervisord under 'fueladmin' user.

 Please review the spec/patches and let's have a discussion on the
 approach to
 this feature.

 Thank you.

 [0] https://blueprints.launchpad.net/fuel/+spec/fuel-nonsuperuser
 [1] https://review.openstack.org/243340
 [2] https://review.openstack.org/243337
 [3] https://review.openstack.org/243313

 --
 Dmitry Nikishov,
 Deployment Engineer,
 Mirantis, Inc.


 __
 OpenStack Development Mailing List (not for usage questions)
 Unsubscribe:
 openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


>>>
>>>
>>> --
>>> Adam Heczko
>>> Security Engineer @ Mirantis Inc.
>>>
>>>
>>> __
>>> OpenStack Development Mailing List (not for usage questions)
>>> Unsubscribe:
>>> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>
>>>
>>
>> __
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe:
>> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>>
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>


-- 
Dmitry Nikishov,
Deployment Engineer,
Mirantis, Inc.
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


[openstack-dev] [HA] weekly High Availability meetings on IRC start next Monday

2015-11-10 Thread Adam Spiers
Hi all,

After some discussion in Tokyo by stakeholders in OpenStack High
Availability, I'm pleased to announce that from next Monday we're
starting a series of weekly meetings on IRC.  Details are here:

  https://wiki.openstack.org/wiki/Meetings/HATeamMeeting
  http://eavesdrop.openstack.org/#High_Availability_Meeting

The agenda for the first meeting is set and will focus on

  1. the pros and cons of the existing approaches to hypervisor HA
 which rely on automatic resurrection[0] of VMs, and

  2. how we might be able to converge on a best-of-breed solution.

All are welcome to join!

On a related note, even if you can't attend the meeting, you can still
use the new FreeNode IRC channel #openstack-ha for all HA-related
discussion.

Cheers,
Adam

[0] In the OpenStack community resurrection is commonly referred to
as "evacuation", which is a slightly unfortunate misnomer.

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [TripleO] [Ironic] Let's stop hijacking other projects' OSC namespaces

2015-11-10 Thread Dmitry Tantsur

On 11/10/2015 06:08 PM, Ben Nemec wrote:

On 11/10/2015 10:28 AM, John Trowbridge wrote:



On 11/10/2015 10:43 AM, Ben Nemec wrote:

On 11/10/2015 05:26 AM, John Trowbridge wrote:



On 11/09/2015 07:44 AM, Dmitry Tantsur wrote:

Hi OOO'ers, hopefully the subject caught your attentions :)

Currently, tripleoclient exposes several commands in "openstack
baremetal" and "openstack baremetal introspection" namespaces belonging
to ironic and ironic-inspector accordingly. TL;DR of this email is to
deprecate them and move to TripleO-specific namespaces. Read on to know
why.

Problem
===

I realized that we're doing a wrong thing when people started asking me
why "baremetal introspection start" and "baremetal introspection bulk
start" behave so differently (the former is from ironic-inspector, the
latter is from tripleoclient). The problem with TripleO commands is that
they're highly opinionated workflows commands, but there's no way a user
can distinguish them from general-purpose ironic/ironic-inspector
commands. The way some of them work is not generic enough ("baremetal
import"), or uses different defaults from an upstream project
("configure boot"), or does something completely unacceptable upstream
(e.g. the way "introspection bulk start" deals with node states).

So, here are commands that tripleoclient exposes with my comments:

1. baremetal instackenv validate

  This command assumes there's an "baremetal instackenv" object, while
instackenv is a tripleo-specific file format.

2. baremetal import

  This command supports a limited subset of ironic drivers and driver
properties, only those known to os-cloud-config.

3. baremetal introspection bulk start

  This command does several bad (IMO) things:
  a. Messes with ironic node states
  b. Operates implicitly on all nodes (in a wrong state)
  c. Defaults to polling



I have considered this whole command as a bug for a while now. I
understand what we were trying to do and why, but it is pretty bad to
hijack another project's namespace with a command that would get a firm
-2 there.


4. baremetal show capabilities

  This is the only commands that is generic enough and could actually
make it to ironicclient itself.

5. baremetal introspection bulk status

  See "bulk start" above.

6. baremetal configure ready state

  First of all, this and the next command use "baremetal configure"
prefix. I would not promise we'll never start using it in ironic,
breaking the whole TripleO.

  Seconds, it's actually DELL-specific.

7. baremetal configure boot

  This one is nearly ok, but it defaults to local boot, which is not an
upstream default. Default values for images may not work outside of
TripleO as well.

Proposal


As we already have "openstack undercloud" and "openstack overcloud"
prefixes for TripleO, I suggest we move these commands under "openstack
overcloud nodes" namespace. So we end up with:

  overcloud nodes import
  overcloud nodes configure ready state --drac
  overcloud nodes configure boot

As you see, I require an explicit --drac argument for "ready state"
command. As to the remaining commands:

1. baremetal introspection status --all

   This is fine to move to inspector-client, as inspector knows which
nodes are/were on introspection. We'll need a new API though.

2. baremetal show capabilities

   We'll have this or similar command in ironic, hopefully this cycle.

3. overcloud nodes introspect --poll --allow-available

   I believe that we need to make 2 things explicit in this replacement
for "introspection bulk status": polling and operating on "available"
nodes.


I am not totally convinced that we gain a huge amount by hiding the
state manipulation in this command. We need to move that logic to
tripleo-common anyways, so I think it is worth considering splitting it
from the introspect command.

Dmitry and I discussed briefly at summit having the ability to pass a
list of nodes to the inspector client for introspection as well. So if
we separated out the bulk state manipulation bit, we could just use that.

I get that this is going in the opposite direction of the original
intention of lowering the amount of commands needed to get a functional
deployment. However, I think that goal is better solved elsewhere
(tripleo.sh, some ansible playbooks, etc.). Instead it would be nice if
the tripleoclient was more transparent.


-2.  This is exactly the thing that got us to a place where our GUI was
unusable.  Business logic (and state management around Ironic node
inspection is just that) has to live in the API so all consumers can
take advantage of it.  Otherwise everyone has to reimplement it
themselves and anything but the developer-used CLI interfaces (like
tripleo.sh) fall behind, and we end up right back where we are today.

It's not simply about reducing the number of commands to deploy a cloud,
it's also (and more importantly) about codifying the process behind an
API that can be used by the CLI, GUI, and any other future clie

Re: [openstack-dev] [fuel] Using upstream packages & modules

2015-11-10 Thread Vladimir Kuklin
Alex

Thanks for your very detailed answer - it clarified things a bit. So, if
you would allow me to rephrase it - you are actually conducting a research
on what is the actual gap between our downstream/fork and upstream
UCA/puppet-openstack. This seems to be a very promising initiative. Are you
going to come up with some external-user readable report soon?

Also, regarding multiple distros support.  I think, we need to come up with
an approach of making 'release manager' piece of Nailgun data driven and
just allow a user to run any distribution he or she wants. Just register a
set of repos with packages and run it. Actually, we already have it - we
need to make base-image generation for RPM more flexible and any RPM/DEB
based distro should work ok with it.

The remaining piece is to actually support distro-specific things, e.g.
CentOS/RHEL networking configuration, e.g. l23 network stored config puppet
providers. But this is a distro-supporter/community burden.



On Tue, Nov 10, 2015 at 6:25 PM, Alex Schultz  wrote:

> Hey Vladimir,
>
> On Tue, Nov 10, 2015 at 5:56 AM, Vladimir Kuklin 
> wrote:
> > Alex
> >
> > That's great to hear that. But be aware - making all of the components
> work
> > exactly the way they work within MOS is actually identical to upstreaming
> > MOS. We are using some components of different versions to satisfy many
> > requirements for our Reference Architecutre implementation. It will not
> be
> > so easy to base them upon existing 3rd party distributions. For example,
> > `read timeout` for SQL requests is crucial for our HA as it handles cases
> > when an SQL node dies while serving the request. And you will find
> myriads
> > of them. And as we do not control things in upstream, we will always have
> > some downstream divergence.
> >
>
> Yes, I'm aware that it'll be an effort to make it work identically to
> MOS.  Currently that's not my goal. My goal is to get it working at
> all and be able to document the deficiencies when using upstream
> packages/modules vs MOS provided ones.  Once we have documented these
> differences we will be able to make decisions as to what efforts
> should be made if we choose to address the differences.  The read
> timeout thing seems to be an issue with what mysql python driver is
> used so that could easily be configurable based on a packages or a
> configuration option.
>
> > I guess, the optimal solution is to figure out the actual divergence
> between
> > upstream and downstream and try to push things upstream as hard as we
> can,
> > while retaining overrides for some packages and manifests on top of
> upstream
> > versions. Do not get me wrong, but it seems there is exactly 0 (zero)
> ways
> > you can get Fuel working with upstream packages unless they support
> exactly
> > the same feature set and fix the same bugs in various components that
> Fuel
> > expect them to support or fix. By 'working' I mean passing the same set
> of
> > at least smoke and destructive tests, let alone passing scale tests.
> >
>
> So I think this is where we are currently backwards in the way we're
> doings. As we hope to push Fuel as a community project, we need to be
> more open to supporting the distributions versions of these packages
> as being supported.  If we want to continue with specific versions of
> things we need to be able to modularize them and apply them to a
> downstream version of Fuel that we can promote with the MOS package
> set.  I agree that right now it's highly unlikely that one would be
> able to use Fuel without any MOS packages. Because I don't think it's
> possible right now, what I'm attempting to do is be able to deploy an
> alternate OpenStack package set but not all of the other pieces as
> they relate to our reference architecture.  That seems to be a more
> achievable goal right now and allows us to document the gaps where
> Fuel/MOS are a head (or behind depending on your views) with upstream.
>
> > Simon
> >
> > AFAIK, this patch is really simple and can be easily applied to any
> version
> > of HAProxy. So it is not too hard handle it while it provides sufficient
> > benefits to our deployment engineers. If you have any better solution,
> > please feel free to share the code with us.
> >
>
> Unfortunately the haproxy fork that we're currently running with also
> includes our forked version of the puppet haproxy module. I'm not sure
> the includes configs patch is really worth carrying the additional
> work/forks but I think that is a conversation for another day.
> Personally I think it would better if fuel-library supported the
> currently available upstream versions of haproxy & haproxy puppet
> module and if we want to continue using our copies of haproxy & puppet
> haproxy module that we turn that into a downstream that is applied for
> the MOS specific version of Fuel.  That's the advantage of the
> Puppetfile we've been working on. We can change out the modules with
> our own version downstream.
>
> Just to add some ad

Re: [openstack-dev] [tacker] Proposing Sripriya Seetharam to Tacker core

2015-11-10 Thread Stephen Wong
+1

On Mon, Nov 9, 2015 at 6:22 PM, Sridhar Ramaswamy  wrote:

> I'd like to propose Sripriya Seetharam to join the Tacker core team.
> Sripriya
> ramped up quickly in early Liberty cycle and had become an expert in the
> Tacker
> code base. Her major contributions include landing MANO API blueprint,
> introducing unit test framework along with the initial unit-tests and
> tirelessly
> squashing hard to resolve bugs (including chasing the recent nova-neutron
> goose
> hunt). Her reviews are solid fine tooth comb and constructive [1].
>
> I'm glad to welcome Sripriya to the core team. Current cores members,
> please vote
> with your +1 / -1.
>
> [1]
> http://stackalytics.com/?release=liberty&user_id=sseetha&project_type=openstack-others
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [TripleO] [Ironic] Let's stop hijacking other projects' OSC namespaces

2015-11-10 Thread Ben Nemec
On 11/10/2015 10:28 AM, John Trowbridge wrote:
> 
> 
> On 11/10/2015 10:43 AM, Ben Nemec wrote:
>> On 11/10/2015 05:26 AM, John Trowbridge wrote:
>>>
>>>
>>> On 11/09/2015 07:44 AM, Dmitry Tantsur wrote:
 Hi OOO'ers, hopefully the subject caught your attentions :)

 Currently, tripleoclient exposes several commands in "openstack
 baremetal" and "openstack baremetal introspection" namespaces belonging
 to ironic and ironic-inspector accordingly. TL;DR of this email is to
 deprecate them and move to TripleO-specific namespaces. Read on to know
 why.

 Problem
 ===

 I realized that we're doing a wrong thing when people started asking me
 why "baremetal introspection start" and "baremetal introspection bulk
 start" behave so differently (the former is from ironic-inspector, the
 latter is from tripleoclient). The problem with TripleO commands is that
 they're highly opinionated workflows commands, but there's no way a user
 can distinguish them from general-purpose ironic/ironic-inspector
 commands. The way some of them work is not generic enough ("baremetal
 import"), or uses different defaults from an upstream project
 ("configure boot"), or does something completely unacceptable upstream
 (e.g. the way "introspection bulk start" deals with node states).

 So, here are commands that tripleoclient exposes with my comments:

 1. baremetal instackenv validate

  This command assumes there's an "baremetal instackenv" object, while
 instackenv is a tripleo-specific file format.

 2. baremetal import

  This command supports a limited subset of ironic drivers and driver
 properties, only those known to os-cloud-config.

 3. baremetal introspection bulk start

  This command does several bad (IMO) things:
  a. Messes with ironic node states
  b. Operates implicitly on all nodes (in a wrong state)
  c. Defaults to polling

>>>
>>> I have considered this whole command as a bug for a while now. I
>>> understand what we were trying to do and why, but it is pretty bad to
>>> hijack another project's namespace with a command that would get a firm
>>> -2 there.
>>>
 4. baremetal show capabilities

  This is the only commands that is generic enough and could actually
 make it to ironicclient itself.

 5. baremetal introspection bulk status

  See "bulk start" above.

 6. baremetal configure ready state

  First of all, this and the next command use "baremetal configure"
 prefix. I would not promise we'll never start using it in ironic,
 breaking the whole TripleO.

  Seconds, it's actually DELL-specific.

 7. baremetal configure boot

  This one is nearly ok, but it defaults to local boot, which is not an
 upstream default. Default values for images may not work outside of
 TripleO as well.

 Proposal
 

 As we already have "openstack undercloud" and "openstack overcloud"
 prefixes for TripleO, I suggest we move these commands under "openstack
 overcloud nodes" namespace. So we end up with:

  overcloud nodes import
  overcloud nodes configure ready state --drac
  overcloud nodes configure boot

 As you see, I require an explicit --drac argument for "ready state"
 command. As to the remaining commands:

 1. baremetal introspection status --all

   This is fine to move to inspector-client, as inspector knows which
 nodes are/were on introspection. We'll need a new API though.

 2. baremetal show capabilities

   We'll have this or similar command in ironic, hopefully this cycle.

 3. overcloud nodes introspect --poll --allow-available

   I believe that we need to make 2 things explicit in this replacement
 for "introspection bulk status": polling and operating on "available"
 nodes.
>>>
>>> I am not totally convinced that we gain a huge amount by hiding the
>>> state manipulation in this command. We need to move that logic to
>>> tripleo-common anyways, so I think it is worth considering splitting it
>>> from the introspect command.
>>>
>>> Dmitry and I discussed briefly at summit having the ability to pass a
>>> list of nodes to the inspector client for introspection as well. So if
>>> we separated out the bulk state manipulation bit, we could just use that.
>>>
>>> I get that this is going in the opposite direction of the original
>>> intention of lowering the amount of commands needed to get a functional
>>> deployment. However, I think that goal is better solved elsewhere
>>> (tripleo.sh, some ansible playbooks, etc.). Instead it would be nice if
>>> the tripleoclient was more transparent.
>>
>> -2.  This is exactly the thing that got us to a place where our GUI was
>> unusable.  Business logic (and state management around Ironic node
>> inspect

Re: [openstack-dev] [Ironic] Do we need to have a mid-cycle?

2015-11-10 Thread Dmitry Tantsur

On 11/10/2015 05:45 PM, Lucas Alvares Gomes wrote:

Hi,

In the last Ironic meeting [1] we started a discussion about whether
we need to have a mid-cycle meeting for the Mitaka cycle or not. Some
ideas about the format of the midcycle were presented in that
conversation and this email is just a follow up on that conversation.

The ideas presented were:

1. Normal mid-cycle

Same format as the previous ones, the meetup will happen in a specific
venue somewhere in the world.


I would really want to see you all as often as possible. However, I 
don't see much value in proper face-to-face mid-cycles as compared to 
improving our day-to-day online communications.




2. Virtual mid-cycle

People doing a virtual hack session on IRC / google hangout /
others... Something like virtual sprints [2].


Actually we could do it more often that mid-cycles and with less 
planning. Say, when we feel a need to. Face-to-face communication is the 
most important part of a mid-cycle, so this choice is not an 
alternative, just one more good thing we could do from time to time.




3. Coordinated regional mid-cycles

Having more than one meetup happening in different parts of the world
with a preferable time overlap between them so we could use video
conference for some hours each day to sync up what was done/discussed
on each of the meetups.


This sounds like a good compromise between not having a midcycle at all 
(I include virtual sprints in this category too) and spending a big 
budget on traveling overseas. I would try something like that.




4. Not having a mid-cycle at all


So, what people think about it? Should we have a mid-cycle for the
Mitaka release or not? If so, what format should we use?

Other ideas are also welcome.

[1] 
http://eavesdrop.openstack.org/meetings/ironic/2015/ironic.2015-11-09-17.00.log.html
[2] https://wiki.openstack.org/wiki/VirtualSprints

Cheers,
Lucas

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev




__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [tacker] Proposing Sripriya Seetharam to Tacker core

2015-11-10 Thread Karthik Natarajan
+1

From: Sridhar Ramaswamy [mailto:sric...@gmail.com]
Sent: Monday, November 09, 2015 6:22 PM
To: OpenStack Development Mailing List (not for usage questions)
Subject: [openstack-dev] [tacker] Proposing Sripriya Seetharam to Tacker core

I'd like to propose Sripriya Seetharam to join the Tacker core team. Sripriya
ramped up quickly in early Liberty cycle and had become an expert in the Tacker
code base. Her major contributions include landing MANO API blueprint,
introducing unit test framework along with the initial unit-tests and tirelessly
squashing hard to resolve bugs (including chasing the recent nova-neutron goose
hunt). Her reviews are solid fine tooth comb and constructive [1].

I'm glad to welcome Sripriya to the core team. Current cores members, please 
vote
with your +1 / -1.

[1] 
http://stackalytics.com/?release=liberty&user_id=sseetha&project_type=openstack-others
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


[openstack-dev] [Ironic] Do we need to have a mid-cycle?

2015-11-10 Thread Lucas Alvares Gomes
Hi,

In the last Ironic meeting [1] we started a discussion about whether
we need to have a mid-cycle meeting for the Mitaka cycle or not. Some
ideas about the format of the midcycle were presented in that
conversation and this email is just a follow up on that conversation.

The ideas presented were:

1. Normal mid-cycle

Same format as the previous ones, the meetup will happen in a specific
venue somewhere in the world.

2. Virtual mid-cycle

People doing a virtual hack session on IRC / google hangout /
others... Something like virtual sprints [2].

3. Coordinated regional mid-cycles

Having more than one meetup happening in different parts of the world
with a preferable time overlap between them so we could use video
conference for some hours each day to sync up what was done/discussed
on each of the meetups.

4. Not having a mid-cycle at all


So, what people think about it? Should we have a mid-cycle for the
Mitaka release or not? If so, what format should we use?

Other ideas are also welcome.

[1] 
http://eavesdrop.openstack.org/meetings/ironic/2015/ironic.2015-11-09-17.00.log.html
[2] https://wiki.openstack.org/wiki/VirtualSprints

Cheers,
Lucas

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Fuel] Master node upgrade

2015-11-10 Thread Vladimir Kuklin
Evgeniy

I am not sure you addressed me, but, anyway, - yes, we will have a
situation with old containers on new host node. This will be identical to
old host node from database migration point of view.

On Tue, Nov 10, 2015 at 7:38 PM, Evgeniy L  wrote:

> Hi Vladimir,
>
> Just to make sure that we are on the same page. We'll have to use upgrade
> script anyway, since you will need to run database migration and register
> new releases.
>
> Thanks,
>
> On Monday, 9 November 2015, Vladimir Kozhukalov 
> wrote:
>
>> Looks like most people thing that building backup/re-install approach is
>> more viable. So, we certainly need to invent completely new upgrade from
>> and thus my suggestion is disable building/testing upgrade tarball right
>> now, because anyway it makes no sense.
>>
>>
>> Vladimir Kozhukalov
>>
>> On Fri, Nov 6, 2015 at 8:21 PM, Vladimir Kuklin 
>> wrote:
>>
>>> Just my 2 cents here - let's do docker backup and roll it up onto brand
>>> new Fuel 8 node.
>>>
>>> On Fri, Nov 6, 2015 at 7:54 PM, Oleg Gelbukh 
>>> wrote:
>>>
 Matt,

 You are talking about this part of Operations guide [1], or you mean
 something else?

 If yes, then we still need to extract data from backup containers. I'd
 prefer backup of DB in simple plain text file, since our DBs are not that
 big.

 [1]
 https://docs.mirantis.com/openstack/fuel/fuel-7.0/operations.html#howto-backup-and-restore-fuel-master

 --
 Best regards,
 Oleg Gelbukh

 On Fri, Nov 6, 2015 at 6:03 PM, Matthew Mosesohn <
 mmoses...@mirantis.com> wrote:

> Oleg,
>
> All the volatile information, including a DB dump, are contained in
> the small Fuel Master backup. There should be no information lost unless
> there was manual customization done inside the containers (such as puppet
> manifest changes). There shouldn't be a need to back up the entire
> containers.
>
> The information we would lose would include the IP configuration
> interfaces besides the one used for the Fuel PXE network and any custom
> configuration done on the Fuel Master.
>
> I want #1 to work smoothly, but #2 should also be a safe route.
>
> On Fri, Nov 6, 2015 at 5:39 PM, Oleg Gelbukh 
> wrote:
>
>> Evgeniy,
>>
>> On Fri, Nov 6, 2015 at 3:27 PM, Evgeniy L  wrote:
>>
>>> Also we should decide when to run containers
>>> upgrade + host upgrade? Before or after new CentOS is installed?
>>> Probably
>>> it should be done before we run backup, in order to get the latest
>>> scripts for
>>> backup/restore actions.
>>>
>>
>> We're working to determine if we need to backup/upgrade containers at
>> all. My expectation is that we should be OK with just backup of DB, IP
>> addresses settings from astute.yaml for the master node, and credentials
>> from configuration files for the services.
>>
>> --
>> Best regards,
>> Oleg Gelbukh
>>
>>
>>>
>>> Thanks,
>>>
>>> On Fri, Nov 6, 2015 at 1:29 PM, Vladimir Kozhukalov <
>>> vkozhuka...@mirantis.com> wrote:
>>>
 Dear colleagues,

 At the moment I'm working on deprecating Fuel upgrade tarball.
 Currently, it includes the following:

 * RPM repository (upstream + mos)
 * DEB repository (mos)
 * openstack.yaml
 * version.yaml
 * upgrade script itself (+ virtualenv)

 Apart from upgrading docker containers this upgrade script makes
 copies of the RPM/DEB repositories and puts them on the master node 
 naming
 these repository directories depending on what is written in 
 openstack.yaml
 and version.yaml. My plan was something like:

 1) deprecate version.yaml (move all fields from there to various
 places)
 2) deliver openstack.yaml with fuel-openstack-metadata package
 3) do not put new repos on the master node (instead we should use
 online repos or use fuel-createmirror to make local mirrors)
 4) deliver fuel-upgrade package (throw away upgrade virtualenv)

 Then UX was supposed to be roughly like:

 1) configure /etc/yum.repos.d/nailgun.repo (add new RPM MOS repo)
 2) yum install fuel-upgrade
 3) /usr/bin/fuel-upgrade (script was going to become lighter,
 because there should have not be parts coping RPM/DEB repos)

 However, it turned out that Fuel 8.0 is going to be run on Centos 7
 and it is not enough to just do things which we usually did during
 upgrades. Now there are two ways to upgrade:
 1) to use the official Centos upgrade script for upgrading from 6
 to 7
 2) to backup the master node, then reinstall it from scratch and
 then apply backup

 Upgrade team is trying to und

Re: [openstack-dev] [Fuel] shotgun code freeze

2015-11-10 Thread Evgeniy L
+1 to Dmitry, thanks for pushing this activity Vladimir.

On Friday, 6 November 2015, Dmitry Pyzhov  wrote:

> Great job! We are much closer to removal of fuel-web repo.
>
> On Tue, Oct 27, 2015 at 7:35 PM, Vladimir Kozhukalov <
> vkozhuka...@mirantis.com
> > wrote:
>
>> Dear colleagues,
>>
>> I am glad to announce that since now shotgun is a separate project.
>> fuel-web/shotgun directory has been deprecated. There is yet another patch
>> that has not been merged https://review.openstack.org/238525 (adds
>> .gitignore file to the new shotgun repo). Please review it.
>>
>> Shotgun
>>
>>- Launchpad bug https://bugs.launchpad.net/fuel/+bug/1506894
>>- project-config patch https://review.openstack.org/235355 (DONE)
>>- pypi (DONE)
>>- run_tests.sh https://review.openstack.org/235368 (DONE)
>>- rpm/deb specs https://review.openstack.org/#/c/235382/ (DONE)
>>- fuel-ci verification jobs https://review.fuel-infra.org/12872 (DONE)
>>- label jenkins slaves for verification (DONE)
>>- directory freeze (DONE)
>>- prepare upstream (DONE)
>>- waiting for project-config patch to be merged (DONE)
>>- .gitreview https://review.openstack.org/238476 (DONE)
>>- .gitignore https://review.openstack.org/238525 (ON REVIEW)
>>- custom jobs parameters https://review.fuel-infra.org/13209 (DONE)
>>- fix core group (DONE)
>>- fuel-main https://review.openstack.org/#/c/238953/ (DONE)
>>- packaging-ci  https://review.fuel-infra.org/13181 (DONE)
>>- MAINTAINERS https://review.openstack.org/239410 (DONE)
>>- deprecate shotgun directory https://review.openstack.org/239407
>>(DONE)
>>- fix verify-fuel-web-docs job (it installs shotgun for some reason)
>>https://review.fuel-infra.org/#/c/13194/ (DONE)
>>- remove old shotgun package (DONE)
>>
>>
>>
>> Vladimir Kozhukalov
>>
>> On Wed, Oct 21, 2015 at 2:46 PM, Vladimir Kozhukalov <
>> vkozhuka...@mirantis.com
>> > wrote:
>>
>>> Dear colleagues,
>>>
>>> As you might know I'm working on splitting multiproject fuel-web
>>> repository into several sub-projects. Shotgun is one of directories that
>>> are to be moved to a separate git project.
>>>
>>> Checklist for this to happen is as follows:
>>>
>>>- Launchpad bug https://bugs.launchpad.net/fuel/+bug/1506894
>>>- project-config patch  https://review.openstack.org/#/c/235355 (ON
>>>REVIEW)
>>>- pypi project
>>>https://pypi.python.org/pypi?%3Aaction=pkg_edit&name=Shotgun (DONE)
>>>- run_tests.sh  https://review.openstack.org/235368 (DONE)
>>>- rpm/deb specs  https://review.openstack.org/#/c/235382 (DONE)
>>>- fuel-ci verification jobs https://review.fuel-infra.org/#/c/12872/ (ON
>>>REVIEW)
>>>- label jenkins slaves for verification jobs (ci team)
>>>- directory freeze (WE ARE HERE)
>>>- prepare upstream (TODO)
>>>- waiting for project-config patch to be merged (ON REVIEW)
>>>- fuel-main patch (TODO)
>>>- packaging-ci patch (TODO)
>>>- deprecate fuel-web/shotgun directory (TODO)
>>>
>>> Now we are at the point where we need to freeze fuel-web/shotgun
>>> directory. So, I'd like to announce code freeze for this directory and all
>>> patches that make changes in the directory and are currently on review will
>>> need to be backported to the new git repository.
>>>
>>> Vladimir Kozhukalov
>>>
>>
>>
>> __
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe:
>> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>>
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Group-based-policy]

2015-11-10 Thread Duarte Cardoso, Igor
Hi Ernesto,

Let me answer the first question for you.

You can use GBP without OpenDaylight.

OpenDaylight has a separate Group-based Policy project, which might make you 
wonder if you need it to run OpenStack GBP. Don’t worry, you can deploy 
OpenStack GBP right away without OpenDaylight.

Best regards,

[intel]

Igor Duarte Cardoso
Software Engineer
+353 61 777 858
SIE1-2-170
Intel Shannon Ltd.
Dromore House, East Park
Shannon, Co. Clare
IRELAND

From: Ernesto Valentino [mailto:ern.valent...@gmail.com]
Sent: Tuesday, November 10, 2015 3:05 PM
To: openstack-dev@lists.openstack.org
Subject: [openstack-dev] [Group-based-policy]

Dear sirs,
I'm a student trying to testing Group Based Policy functionalities. I have some 
questions about it, because from the documentation is not clear to me what role 
assume opendaylight in the plug-in.
I can use gbp only with openstack or is mandatory to use it with opendaylight? 
And next, if I want to add my personal nfv to gbp is there a possibility to do 
that or not ? Thanks so much for answering.
Waiting for your kind reply.

Best regards,
Ernesto Valentino
--
Intel Research and Development Ireland Limited
Registered in Ireland
Registered Office: Collinstown Industrial Park, Leixlip, County Kildare
Registered Number: 308263


This e-mail and any attachments may contain confidential material for the sole
use of the intended recipient(s). Any review or distribution by others is
strictly prohibited. If you are not the intended recipient, please contact the
sender and delete all copies.
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [tripleo] Location of TripleO REST API

2015-11-10 Thread John Trowbridge


On 11/10/2015 10:08 AM, Tzu-Mainn Chen wrote:
> Hi all,
> 
> At the last IRC meeting it was agreed that the new TripleO REST API
> should forgo the Tuskar name, and simply be called... the TripleO
> API.  There's one more point of discussion: where should the API
> live?  There are two possibilities:
> 
> a) Put it in tripleo-common, where the business logic lives.  If we
> do this, it would make sense to rename tripleo-common to simply
> tripleo.

This option makes the most sense to me now that I understand the
tripleoclient will also consume the REST API and not the underlying
python library with the "business logic".

>From a packaging point of view, I think this is pretty painless. I think
we would have a "python-tripleo" package for the library that
provides/obsoletes the current tripleo-common package. As well as a
"tripleo-api" package for the REST API service. We would not need to
touch the tripleo-incubator package that currently is named just "tripleo".

> 
> b) Put it in its own repo, tripleo-api
> 
> 
> The first option made a lot of sense to people on IRC, as the proposed
> API is a very thin layer that's bound closely to the code in tripleo-
> common.  The major objection is that renaming is not trivial; however
> it was mentioned that renaming might not be *too* bad... as long as
> it's done sooner rather than later.
> 
> What do people think?
> 
> 
> Thanks,
> Tzu-Mainn Chen
> 
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> 

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [TripleO] [Ironic] Let's stop hijacking other projects' OSC namespaces

2015-11-10 Thread Dmitry Tantsur

On 11/10/2015 05:18 PM, Ben Nemec wrote:

+1 to moving anything we can into Ironic itself.

I do want to note that if we rename anything, we can't just rip and
replace.  We have users of the current commands, and we need to
deprecate those to give people a chance to move to the new ones.


Definitely. I think I used word deprecation somewhere in this long letter :)



A few more thoughts inline.

On 11/09/2015 06:44 AM, Dmitry Tantsur wrote:

Hi OOO'ers, hopefully the subject caught your attentions :)

Currently, tripleoclient exposes several commands in "openstack
baremetal" and "openstack baremetal introspection" namespaces belonging
to ironic and ironic-inspector accordingly. TL;DR of this email is to
deprecate them and move to TripleO-specific namespaces. Read on to know why.

Problem
===

I realized that we're doing a wrong thing when people started asking me
why "baremetal introspection start" and "baremetal introspection bulk
start" behave so differently (the former is from ironic-inspector, the
latter is from tripleoclient). The problem with TripleO commands is that
they're highly opinionated workflows commands, but there's no way a user
can distinguish them from general-purpose ironic/ironic-inspector
commands. The way some of them work is not generic enough ("baremetal
import"), or uses different defaults from an upstream project
("configure boot"), or does something completely unacceptable upstream
(e.g. the way "introspection bulk start" deals with node states).

So, here are commands that tripleoclient exposes with my comments:

1. baremetal instackenv validate

   This command assumes there's an "baremetal instackenv" object, while
instackenv is a tripleo-specific file format.

2. baremetal import

   This command supports a limited subset of ironic drivers and driver
properties, only those known to os-cloud-config.


True, although I feel like an "import from JSON" feature would not be
inappropriate for inclusion in Ironic.  I can't believe that we're the
only ones who would be interested in mass importing nodes from a very
common format like this.


I would warmly welcome such command, but it should not require patching 
os-cloud-config every time we add a new driver or driver property.






3. baremetal introspection bulk start

   This command does several bad (IMO) things:
   a. Messes with ironic node states
   b. Operates implicitly on all nodes (in a wrong state)


I thought that was fixed?  It used to try to introspect nodes that were
in an invalid state (like active), but it shouldn't anymore.


It introspects nodes in "available" state, which is a rude violation of 
the ironic state machine ;)




Unless your objection is that it introspects things in an available
state, which I think has to do with the state Ironic puts (or used to
put) nodes in after registration.  In any case, this one likely requires
some more discussion over how it should work.


Well, if we upgrade baremetal API version we used from 1.6 (essentially 
Kilo) to Liberty one, nodes would appear in a new ENROLL state. I've 
started a patch for it, but I don't have time to finish. Anyone is free 
to overtake: https://review.openstack.org/#/c/235158/





   c. Defaults to polling

4. baremetal show capabilities

   This is the only commands that is generic enough and could actually
make it to ironicclient itself.

5. baremetal introspection bulk status

   See "bulk start" above.

6. baremetal configure ready state

   First of all, this and the next command use "baremetal configure"
prefix. I would not promise we'll never start using it in ironic,
breaking the whole TripleO.

   Seconds, it's actually DELL-specific.


Well, as I understand it we don't intend for it to be Dell-specific,
it's just that the Dell implementation is the only one that has been
done so far.

That said, since I think this is just layering some TripleO-specific
logic on top of the vendor-specific calls Ironic provides I agree that
it probably doesn't belong in the baremetal namespace.



7. baremetal configure boot

   This one is nearly ok, but it defaults to local boot, which is not an
upstream default. Default values for images may not work outside of
TripleO as well.

Proposal


As we already have "openstack undercloud" and "openstack overcloud"
prefixes for TripleO, I suggest we move these commands under "openstack
overcloud nodes" namespace. So we end up with:

   overcloud nodes import
   overcloud nodes configure ready state --drac
   overcloud nodes configure boot

As you see, I require an explicit --drac argument for "ready state"
command. As to the remaining commands:

1. baremetal introspection status --all

This is fine to move to inspector-client, as inspector knows which
nodes are/were on introspection. We'll need a new API though.


+1



2. baremetal show capabilities

We'll have this or similar command in ironic, hopefully this cycle.


\o/



3. overcloud nodes introspect --poll --allow-available

  

Re: [openstack-dev] [Openstack-operators] [stable][all] Keeping Juno "alive" for longer.

2015-11-10 Thread Matt Riedemann



On 11/9/2015 10:15 PM, Matthew Treinish wrote:

On Mon, Nov 09, 2015 at 05:05:36PM +1100, Tony Breeds wrote:

On Fri, Nov 06, 2015 at 10:12:21AM -0800, Clint Byrum wrote:


The argument in the original post, I think, is that we should not
stand in the way of the vendors continuing to collaborate on stable
maintenance in the upstream context after the EOL date. We already have
distro vendors doing work in the stable branches, but at EOL we push
them off to their respective distro-specific homes.


That is indeed a better summary than I started with.

I have a half formed idea that creates a state between the current EOL (where
we delete the branches) to what we have today (where we have full CI/VMT/Release
management.


So this idea has come up before and it has been rejected for good reasons. If
vendors want to collaborate, but not to actually work to keep things working in
the gate is that really collaboration? It's just vendors pushing whatever random
backports they want into a shared repo. There is a reason we do all the testing
and it's deeply encoded into the openstack culture. If managing to keep things
verifiably working with the same set of test jobs when a branch was created is
too much of a burden for people then there isn't a reason to keep it around.

This is the crux of why we have shorter branch support windows. No matter how
much complaining people do about how they want LTS releases or longer support
windows, and we'll have world peace when they can use Grizzly with upstream
support for 17 years it doesn't change that barely anyone ever steps up to work
on keeping the gate working on stable branches.


Heh, speaking of Grizzly, I was just looking at backporting 
https://review.openstack.org/#/c/219301/ to Grizzly yesterday since we 
still support it. There is an unholy trifecta of things in that change 
that aren't in Grizzly: nova objects, an API method to get filtered 
resources from the DB, and a database migration in the change that added 
the virt driver method used as part of the fix (all added in Havana).


So fixing that in Grizzly is basically going to be a Frankenstein change 
to avoid the REST API backport and DB migration backport. Good times.


Why am I pointing this out? Just to give an example of the kind of mess 
that is involved in maintaining a branch that old.




Tony, as someone who has done a good job coming up to speed on fixing issues on 
the
stable branches you know this firsthand. We're always end up talking to the same
handful of people to debug issues. We're also almost always in firefighting mode
and regular failure rates on stable just keep going up when we look away. People
also burn out quickly debugging these issues all the time. Personally, I know I
don't keep an eye on things nearly as closely as I did before.



There is a massive ammount of trust involved in that simple statement and I 
don't
underestimate that.

This would provide a place where interested vendors can work together.
We could disable grenade in juno which isn't awesome but removes the gotta land
patch in juno, kilo and liberty to unblock the master gate.
We could reduce the VMT impact by nominating a point of contact for juno and
granting that person access to the embargoed bugs.  Similarly we could
trust/delegate a certain about of the release management to that team (I say
team but so far we've only got one volunteer)

We can't ignore the fact that fixing things in Juno *may* still require fixes
in Kilo (and later releases) esp. given the mess that is requirements in
stable/juno


As much as I'd like everyone to get on the CD train, I think it might
make sense to enable the vendors to not diverge, but instead let them
show up with people and commitment and say "Hey we're going to keep
Juno/Mitaka/etc alive!".

So perhaps what would make sense is defining a process by which they can
make that happen.

Note that it's not just backporters though. It's infra resources too.


Sure.  CI and Python 2.6 I have a little understanding of.  I guess I can
extrapolate the additional burden on {system,project}-config.  I willingly
admit I don't have a detailed feel for what I'm asking here.


It's more than just that too, there is additional resources on Tempest and other
QA projects like devstack and grenade too. Tempest is branchless, so to keep
branches around longer you have to have additional jobs running on tempest to
ensure incoming changes work across all releases. In Tokyo we were discussing
doing the same thing on the client repos too because they make similar
guarantees about backwards compat but we never test it. There is a ton of extra
load generated by keeping things around for longer, it's not to be taken
lightly. Especially given the historical lack of contribution in this space.
This is honestly why our 1 experiment in a longer support ended in failure,
nobody stepped up to support the extra branch. To even attempt it again we need
proof that things have improved, which it clearly

Re: [openstack-dev] [Fuel] Master node upgrade

2015-11-10 Thread Evgeniy L
Hi Vladimir,

Just to make sure that we are on the same page. We'll have to use upgrade
script anyway, since you will need to run database migration and register
new releases.

Thanks,

On Monday, 9 November 2015, Vladimir Kozhukalov 
wrote:

> Looks like most people thing that building backup/re-install approach is
> more viable. So, we certainly need to invent completely new upgrade from
> and thus my suggestion is disable building/testing upgrade tarball right
> now, because anyway it makes no sense.
>
>
> Vladimir Kozhukalov
>
> On Fri, Nov 6, 2015 at 8:21 PM, Vladimir Kuklin  > wrote:
>
>> Just my 2 cents here - let's do docker backup and roll it up onto brand
>> new Fuel 8 node.
>>
>> On Fri, Nov 6, 2015 at 7:54 PM, Oleg Gelbukh > > wrote:
>>
>>> Matt,
>>>
>>> You are talking about this part of Operations guide [1], or you mean
>>> something else?
>>>
>>> If yes, then we still need to extract data from backup containers. I'd
>>> prefer backup of DB in simple plain text file, since our DBs are not that
>>> big.
>>>
>>> [1]
>>> https://docs.mirantis.com/openstack/fuel/fuel-7.0/operations.html#howto-backup-and-restore-fuel-master
>>>
>>> --
>>> Best regards,
>>> Oleg Gelbukh
>>>
>>> On Fri, Nov 6, 2015 at 6:03 PM, Matthew Mosesohn >> > wrote:
>>>
 Oleg,

 All the volatile information, including a DB dump, are contained in the
 small Fuel Master backup. There should be no information lost unless there
 was manual customization done inside the containers (such as puppet
 manifest changes). There shouldn't be a need to back up the entire
 containers.

 The information we would lose would include the IP configuration
 interfaces besides the one used for the Fuel PXE network and any custom
 configuration done on the Fuel Master.

 I want #1 to work smoothly, but #2 should also be a safe route.

 On Fri, Nov 6, 2015 at 5:39 PM, Oleg Gelbukh >>> > wrote:

> Evgeniy,
>
> On Fri, Nov 6, 2015 at 3:27 PM, Evgeniy L  > wrote:
>
>> Also we should decide when to run containers
>> upgrade + host upgrade? Before or after new CentOS is installed?
>> Probably
>> it should be done before we run backup, in order to get the latest
>> scripts for
>> backup/restore actions.
>>
>
> We're working to determine if we need to backup/upgrade containers at
> all. My expectation is that we should be OK with just backup of DB, IP
> addresses settings from astute.yaml for the master node, and credentials
> from configuration files for the services.
>
> --
> Best regards,
> Oleg Gelbukh
>
>
>>
>> Thanks,
>>
>> On Fri, Nov 6, 2015 at 1:29 PM, Vladimir Kozhukalov <
>> vkozhuka...@mirantis.com
>> > wrote:
>>
>>> Dear colleagues,
>>>
>>> At the moment I'm working on deprecating Fuel upgrade tarball.
>>> Currently, it includes the following:
>>>
>>> * RPM repository (upstream + mos)
>>> * DEB repository (mos)
>>> * openstack.yaml
>>> * version.yaml
>>> * upgrade script itself (+ virtualenv)
>>>
>>> Apart from upgrading docker containers this upgrade script makes
>>> copies of the RPM/DEB repositories and puts them on the master node 
>>> naming
>>> these repository directories depending on what is written in 
>>> openstack.yaml
>>> and version.yaml. My plan was something like:
>>>
>>> 1) deprecate version.yaml (move all fields from there to various
>>> places)
>>> 2) deliver openstack.yaml with fuel-openstack-metadata package
>>> 3) do not put new repos on the master node (instead we should use
>>> online repos or use fuel-createmirror to make local mirrors)
>>> 4) deliver fuel-upgrade package (throw away upgrade virtualenv)
>>>
>>> Then UX was supposed to be roughly like:
>>>
>>> 1) configure /etc/yum.repos.d/nailgun.repo (add new RPM MOS repo)
>>> 2) yum install fuel-upgrade
>>> 3) /usr/bin/fuel-upgrade (script was going to become lighter,
>>> because there should have not be parts coping RPM/DEB repos)
>>>
>>> However, it turned out that Fuel 8.0 is going to be run on Centos 7
>>> and it is not enough to just do things which we usually did during
>>> upgrades. Now there are two ways to upgrade:
>>> 1) to use the official Centos upgrade script for upgrading from 6 to
>>> 7
>>> 2) to backup the master node, then reinstall it from scratch and
>>> then apply backup
>>>
>>> Upgrade team is trying to understand which way is more appropriate.
>>> Regarding to my tarball related activities, I'd say that this package 
>>> based
>>> upgrade approach can be aligned with (1) (fuel-upgrade would use 
>>> official
>>> Centos upgrade script as a first step for upgrade), but it definitely 
>>> can
>>> not be aligned with (2), because it assumes reinstalling the master node
>>> from

Re: [openstack-dev] [Fuel][Fuel-QA][Fuel-TechDebt] Code Quality: Do Not Hardcode - Fix Things Instead

2015-11-10 Thread Stanislaw Bogatkin
I think that it is excellent thought.
+1

On Tue, Nov 10, 2015 at 6:52 PM, Vladimir Kuklin 
wrote:

> Folks
>
> I wanted to raise awareness about one of the things I captured while doing
> reviews recently - we are sacrificing quality to bugfixing and feature
> development velocity, essentially moving from one heap to another - from
> bugs/features to 'tech-debt' bugs.
>
> I understand that we all have deadlines and need to meet them. But, folks,
> let's create the following policy:
>
> 1) do not introduce hacks/workarounds/kludges if it is possible.
> 2) while fixing things if you have a hack/workaround/kludge that you need
> to work with - think of removing it instead of enhancing and extending it.
> If it is possible - fix it. Do not let our technical debt grow.
> 3) if there is no way to avoid kludge addition/enhancing, if there is no
> way to remove it - please, add a 'TODO/FIXME' line above it, so that we can
> collect them in the future and fix them gradually.
>
> I suggest to add this requirement into code-review policy.
>
> What do you think about this?
>
> --
> Yours Faithfully,
> Vladimir Kuklin,
> Fuel Library Tech Lead,
> Mirantis, Inc.
> +7 (495) 640-49-04
> +7 (926) 702-39-68
> Skype kuklinvv
> 35bk3, Vorontsovskaya Str.
> Moscow, Russia,
> www.mirantis.com 
> www.mirantis.ru
> vkuk...@mirantis.com
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [keystone] [Mistral] [Heat] Autoprovisioning, per-user projects, and Federation

2015-11-10 Thread Adam Young

On 11/10/2015 10:28 AM, Renat Akhmerov wrote:




On 09 Nov 2015, at 20:43, Adam Young  wrote:

On 11/06/2015 06:28 PM, Tim Hinrichs wrote:

Congress allows users to write a policy that executes an action under certain 
conditions.

The conditions can be based on any data Congress has access to, which includes 
nova servers, neutron networks, cinder storage, keystone users, etc.  We also 
have some Ceilometer statistics; I'm not sure about whether it's easy to get 
the Keystone notifications that you're talking about today, but notifications 
are on our roadmap.  If the user's login is reflected in the Keystone API, we 
may already be getting that event.

The action could in theory be a mistral/heat API or an arbitrary script.  Right 
now we're set up to invoke any method on any of the python-clients we've 
integrated with.  We've got an integration with heat but not mistral.  New 
integrations are typically easy.

Sounds like Mistral and Congress are competing here, then.  Maybe we should 
merge those efforts.

I may be wrong on this but the difference is that Mistral provides workflow. 
Meaning you can have a graph of tasks related by conditional logic whereas 
Congress action is something simple like calling a function. Correct me if my 
understanding is wrong. I actually don’t know at this point whether a workflow 
is really needed, IMO it does make sense if we need to create a bunch of heavy 
resources so it should be an HA service managing the process of 
configuring/creating the new tenant. The power of workflow is in automating 
long-running stuff.

But both technologies are missing notifications part now.


This does not need to be super complicated; we need a listener that can 
kick off workflows.  If congress is that listener, super.



I would think that it would then be

1. Keystone sends "new federated user" notification out on the message bus.
2. Congress Picks up the message and checks policy to see what should be 
done.

3. Congress calls Heat to fire off template for autoprovisioning user.

It could also be:

1. Keystone sends "new federated user" notification out on the message bus.
2. Murano Picks up the message and checks policy to see what should be done.
3. Murano calls Heat to fire off template for autoprovisioning user.


You are suggesting it should be:

1. Keystone sends "new federated user" notification out on the message bus.
2. Congress Picks up the message and checks policy to see what should be 
done.

3. Congress calls Murano to fire off template for autoprovisioning user.

And, the most complex solution:

1. Keystone sends "new federated user" notification out on the message bus.
2. Congress Picks up the message and checks policy to see what should be 
done.

3. Congress calls Murano to fire off template for autoprovisioning user.
4. Murano calls Heat to fire off template for autoprovisioning user.

Personally, I would prefer:

1. Keystone sends "new federated user" notification out on the message bus.
2. Heat Picks up the message and checks policy to see what should be done.
3. Heat fires off template for autoprovisioning user.

Why the last:  Heat already exists and already has a message listener.








Renat Akhmerov
@ Mirantis Inc.



__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Ironic] Quick poll: OpenStackClient command for provision action

2015-11-10 Thread Dmitry Tantsur

On 11/10/2015 05:21 PM, Brad P. Crochet wrote:

On Tue, Nov 10, 2015 at 4:09 AM, Dmitry Tantsur  wrote:

Hi all!

I'd like to seek consensus (or at least some opinions) on patch
https://review.openstack.org/#/c/206119/
It proposed the following command:



I think it's time to actually just write up a spec on this. I think we
would be better served to spell it out now, and then more people can
contribute to both the spec and to the actual implementation once the
spec is approved.

WDYT?


+1

I'll block the first patch until we get consensus. Thanks for working on it!




   openstack baremetal provision state --provide UUID

(where --provide can also be --active, --deleted, --inspect, etc).

I have several issues with this proposal:

1. IIUC the structure of an OSC command is "openstack noun verb". "provision
state" is not a verb.
2. --active is not consistent with other options, which are verbs.

Let's have a quick poll, which would you prefer and why:

1. openstack baremetal provision state --provide UUID
2. openstack baremetal provision --provide UUID
3. openstack baremetal provide UUID
4. openstack baremetal set provision state --provide UUID
5. openstack baremetal set state --provide UUID
6. openstack baremetal action --provide UUID

I vote for #3. Though it's much more versbose, it reads very easily, except
for "active". For active I'm thinking about changing it to "activate" or
"provision".

My next candidate is #6. Though it's also not a verb, it reads pretty
easily.

Thanks!

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev







__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [TripleO] [Ironic] Let's stop hijacking other projects' OSC namespaces

2015-11-10 Thread John Trowbridge


On 11/10/2015 10:43 AM, Ben Nemec wrote:
> On 11/10/2015 05:26 AM, John Trowbridge wrote:
>>
>>
>> On 11/09/2015 07:44 AM, Dmitry Tantsur wrote:
>>> Hi OOO'ers, hopefully the subject caught your attentions :)
>>>
>>> Currently, tripleoclient exposes several commands in "openstack
>>> baremetal" and "openstack baremetal introspection" namespaces belonging
>>> to ironic and ironic-inspector accordingly. TL;DR of this email is to
>>> deprecate them and move to TripleO-specific namespaces. Read on to know
>>> why.
>>>
>>> Problem
>>> ===
>>>
>>> I realized that we're doing a wrong thing when people started asking me
>>> why "baremetal introspection start" and "baremetal introspection bulk
>>> start" behave so differently (the former is from ironic-inspector, the
>>> latter is from tripleoclient). The problem with TripleO commands is that
>>> they're highly opinionated workflows commands, but there's no way a user
>>> can distinguish them from general-purpose ironic/ironic-inspector
>>> commands. The way some of them work is not generic enough ("baremetal
>>> import"), or uses different defaults from an upstream project
>>> ("configure boot"), or does something completely unacceptable upstream
>>> (e.g. the way "introspection bulk start" deals with node states).
>>>
>>> So, here are commands that tripleoclient exposes with my comments:
>>>
>>> 1. baremetal instackenv validate
>>>
>>>  This command assumes there's an "baremetal instackenv" object, while
>>> instackenv is a tripleo-specific file format.
>>>
>>> 2. baremetal import
>>>
>>>  This command supports a limited subset of ironic drivers and driver
>>> properties, only those known to os-cloud-config.
>>>
>>> 3. baremetal introspection bulk start
>>>
>>>  This command does several bad (IMO) things:
>>>  a. Messes with ironic node states
>>>  b. Operates implicitly on all nodes (in a wrong state)
>>>  c. Defaults to polling
>>>
>>
>> I have considered this whole command as a bug for a while now. I
>> understand what we were trying to do and why, but it is pretty bad to
>> hijack another project's namespace with a command that would get a firm
>> -2 there.
>>
>>> 4. baremetal show capabilities
>>>
>>>  This is the only commands that is generic enough and could actually
>>> make it to ironicclient itself.
>>>
>>> 5. baremetal introspection bulk status
>>>
>>>  See "bulk start" above.
>>>
>>> 6. baremetal configure ready state
>>>
>>>  First of all, this and the next command use "baremetal configure"
>>> prefix. I would not promise we'll never start using it in ironic,
>>> breaking the whole TripleO.
>>>
>>>  Seconds, it's actually DELL-specific.
>>>
>>> 7. baremetal configure boot
>>>
>>>  This one is nearly ok, but it defaults to local boot, which is not an
>>> upstream default. Default values for images may not work outside of
>>> TripleO as well.
>>>
>>> Proposal
>>> 
>>>
>>> As we already have "openstack undercloud" and "openstack overcloud"
>>> prefixes for TripleO, I suggest we move these commands under "openstack
>>> overcloud nodes" namespace. So we end up with:
>>>
>>>  overcloud nodes import
>>>  overcloud nodes configure ready state --drac
>>>  overcloud nodes configure boot
>>>
>>> As you see, I require an explicit --drac argument for "ready state"
>>> command. As to the remaining commands:
>>>
>>> 1. baremetal introspection status --all
>>>
>>>   This is fine to move to inspector-client, as inspector knows which
>>> nodes are/were on introspection. We'll need a new API though.
>>>
>>> 2. baremetal show capabilities
>>>
>>>   We'll have this or similar command in ironic, hopefully this cycle.
>>>
>>> 3. overcloud nodes introspect --poll --allow-available
>>>
>>>   I believe that we need to make 2 things explicit in this replacement
>>> for "introspection bulk status": polling and operating on "available"
>>> nodes.
>>
>> I am not totally convinced that we gain a huge amount by hiding the
>> state manipulation in this command. We need to move that logic to
>> tripleo-common anyways, so I think it is worth considering splitting it
>> from the introspect command.
>>
>> Dmitry and I discussed briefly at summit having the ability to pass a
>> list of nodes to the inspector client for introspection as well. So if
>> we separated out the bulk state manipulation bit, we could just use that.
>>
>> I get that this is going in the opposite direction of the original
>> intention of lowering the amount of commands needed to get a functional
>> deployment. However, I think that goal is better solved elsewhere
>> (tripleo.sh, some ansible playbooks, etc.). Instead it would be nice if
>> the tripleoclient was more transparent.
> 
> -2.  This is exactly the thing that got us to a place where our GUI was
> unusable.  Business logic (and state management around Ironic node
> inspection is just that) has to live in the API so all consumers can
> take advantage of it.  Otherwise everyone has to reimplement it
> themselves and anything but the developer-u

Re: [openstack-dev] [oslo] Graduate apiclient library

2015-11-10 Thread Dean Troyer
On Tue, Nov 10, 2015 at 9:57 AM, Adam Young  wrote:

> +2.  keystoneauth is designed to be the library explicitly for this.  Do
> not duplicate auth behavior anywhere else in the supported client libraries.
>

To expand on this point a bit, if you are considering re-working the
existing client libs, when you use keystoneauth (ksa) you also get the
fully-functional reuests.Session-based Session for free and should use that
rather than creating yet another one.

I've already started working on this for OSC, some of the libs work, others
do not (yet) so I know this is possible.

dt

-- 

Dean Troyer
dtro...@gmail.com
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Ironic] [OSC] Quick poll: OpenStackClient command for provision action

2015-11-10 Thread Dmitry Tantsur

On 11/10/2015 05:14 PM, Dean Troyer wrote:

On Tue, Nov 10, 2015 at 9:46 AM, Dmitry Tantsur mailto:dtant...@redhat.com>> wrote:

  inspect, manage, provide, active and abort are all provisioning
verbs used in ironic API. they usually represent some complex
operations on a node. Inspection is not related to showing, it's
about fetching hardware properties from hardware itself and updating
ironic database. manage sets a node to a specific ("manageable")
state. etc.


inspect seems like a very specific action and is probably OK as-is.  We
should sanity-check other resources in OpenStack that it might also be
used with down the road and how different the action might be.


ironic-inspector uses term "introspection", but that's another long story.



Those that are states of a resource should be handled with a set command.


Speaking of consistency, all these move a node in the ironic state 
machine, so it might be weird to have "inspect" but "set manageable". 
Maybe it's only me, not sure.. The problem is that some states 
manipulation result in a simple actions (e.g. "manage" action either 
does nothing or does power credentials validation depending on the 
initial state). But most provision state changes involve complex long 
running operations ("active" to deploy, "deleted" to undeploy and clean, 
"inspect" to conduct inspection). Not sure how to make these consistent, 
any suggestions are very welcome.




boot and shutdown are natural opposites, aka power on and power off.


The analogous server commands (create/delete) may not make sense here
because, unlike with a server (VM), a resource is not being created or
deleted.  But a user might expect to use the same commands in both
places.  We need to consider which of those is more important.  I like
to break ties on the side of user experience consistency.

Honestly, at some point as a user, I'd like to forget whether my server
is a bare metal box or not and just use the same commands to manage it.


Well, it's not possible. Or more precisely, it is possible if you use 
ironic indirectly via nova API. But power on/off is not very similar to 
instance create/delete. Instance create is actually correlates to 
"active" provision state, instance deletion - to "deleted" (yeah, naming 
is no so good here).




Also, I'd LOVE to avoid using 'boot' at all just to get away from the
nova command's use of it.


+1



dt

--

Dean Troyer
dtro...@gmail.com 


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev




__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [stable] Making stable maintenance its own OpenStack project team

2015-11-10 Thread Matt Riedemann



On 11/10/2015 5:41 AM, Alan Pevec wrote:

The Stable Branch Policy, lets not put it in any repo.
Doing that would give indication that we expect changes into it and I'd prefer 
that policy mostly staying stable as well.


What indication did it give when it was in wiki? :)


It actually already is in a repo:
http://git.openstack.org/cgit/openstack/project-team-guide/tree/doc/source/stable-branches.rst


https://wiki.openstack.org/wiki/StableBranch should be replaced with
the link to it to avoid confusion.


Done.


There were changes since it was moved to the repo:
https://wiki.openstack.org/w/index.php?title=StableBranch&action=history
Which team approves the changes in the project-team-guide, can it be per file?

Regarding stable team spin-off, I think it's worth discussing further
to define exact scope, then see show of hands before formally setting
the team up as I'm afraid we don't have the critical mass yet.
I'm personally tied up with other obligations next few months so
cannot actively help or lead this initial effort.

Cheers,
Alan

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



--

Thanks,

Matt Riedemann


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] Learning to Debug the Gate

2015-11-10 Thread Sean Dague
There is also a neutron issue which is causing a 35% failure rate in
neutron jobs -
http://tinyurl.com/ne3ex4v

That one still needs resolution.

On 11/10/2015 10:54 AM, Davanum Srinivas wrote:
> Took about 35 mins or so :)
> 
> -- Dims
> 
> On Tue, Nov 10, 2015 at 10:45 AM, Matt Riedemann
>  wrote:
>>
>>
>> On 11/9/2015 3:54 PM, Anita Kuno wrote:
>>>
>>> On 11/05/2015 07:45 PM, Anita Kuno wrote:

 On 11/03/2015 05:30 PM, Anita Kuno wrote:
>
> On 11/02/2015 12:39 PM, Anita Kuno wrote:
>>
>> On 10/29/2015 10:42 PM, Anita Kuno wrote:
>>>
>>> On 10/29/2015 08:27 AM, Anita Kuno wrote:

 On 10/28/2015 12:14 AM, Matt Riedemann wrote:
>
>
>
> On 10/27/2015 4:08 AM, Anita Kuno wrote:
>>
>> Learning how to debug the gate was identified as a theme at the
>> "Establish Key Themes for the Mitaka Cycle" cross-project session:
>> https://etherpad.openstack.org/p/mitaka-crossproject-themes
>>
>> I agreed to take on this item and facilitate the process.
>>
>> Part one of the conversation includes referencing this video
>> created by
>> Sean Dague and Dan Smith:
>> https://www.youtube.com/watch?v=fowBDdLGBlU
>>
>> Please consume this as you are able.
>>
>> Other suggestions for how to build on this resource were mentioned
>> and
>> will be coming in the future but this was an easy, actionable first
>> step.
>>
>> Thank you,
>> Anita.
>>
>>
>> __
>>
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe:
>> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>
>
> https://www.openstack.org/summit/vancouver-2015/summit-videos/presentation/tales-from-the-gate-how-debugging-the-gate-helps-your-enterprise
>
>

 The source for the definition of "the gate":

 http://git.openstack.org/cgit/openstack-infra/project-config/tree/zuul/layout.yaml#n34

 Thanks for following along,
 Anita.

>>>
>>> This is the status page showing the status of our running jobs,
>>> including patches in the gate pipeline:
>>> http://status.openstack.org/zuul/
>>>
>>> Thank you,
>>> Anita.
>>>
>>
>> This is a simulation of how the gate tests patches:
>> http://docs.openstack.org/infra/publications/zuul/#%2818%29
>>
>> Click in the browser window to advance the simulation.
>>
>> Thank you,
>> Anita.
>>
>
> Here is a presentation that uses the slide deck linked above, I
> recommend watching: https://www.youtube.com/watch?v=WDoSCGPiFDQ
>
> Thank you,
> Anita.
>

 Three links in this edition of Learning to Debug the Gate:

 The view that tracks our top bugs:
 http://status.openstack.org/elastic-recheck/

 The logstash queries that create the above view:

 http://git.openstack.org/cgit/openstack-infra/elastic-recheck/tree/queries

 Logstash itself, where you too can practice creating queries:
 http://logstash.openstack.org

 Note: in logstash the query is the transferable piece of information.
 Filters can help you create a query, they do not populate a query. The
 information that is in the query bar is what is important here.

 Practice making some queries of your own.

 Thanks for reading,
 Anita.

>>>
>>> Today the elastic search cluster was upgraded to 1.7.3:
>>>
>>> http://lists.openstack.org/pipermail/openstack-dev/2015-November/078314.html
>>>
>>> Go over the 1.7 elastic search docs:
>>> https://www.elastic.co/guide/en/elasticsearch/reference/1.7/index.html
>>> and try a few queries.
>>>
>>> Thanks for following along,
>>> Anita.
>>>
>>> __
>>> OpenStack Development Mailing List (not for usage questions)
>>> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>
>>
>> If you want a fun little play by play of a critical blocking gate issue for
>> nova, it's in the nova irc logs starting here:
>>
>> http://eavesdrop.openstack.org/irclogs/%23openstack-nova/%23openstack-nova.2015-11-10.log.html#t2015-11-10T14:59:27
>>
>> Which led to a block of oslo.messaging 2.8.0 in g-r and a revert in the o.m
>> repo. The point isn't the outcome, but the process in which we figured out
>> what the regression was and quickly resolved it using logstash and seeing
>> what recent dependent library releases impacted nova.
>>
>> --
>>
>> Thank

Re: [openstack-dev] [Ironic] Quick poll: OpenStackClient command for provision action

2015-11-10 Thread Brad P. Crochet
On Tue, Nov 10, 2015 at 10:53 AM, Dean Troyer  wrote:
> On Tue, Nov 10, 2015 at 9:32 AM, Brad P. Crochet  wrote:
>>
>> In this case, the noun is actually 'baremetal provision state'. The
>> 'action' is the states themselves. It doesn't fit exactly, but seems
>> to at least be somewhat natural.
>
>
> resource == provision state (add baremetal if namespacing is required)
> action == set
> value == --state x|y|z
>
> provision state set --state active|deleted|provide 
>
> (note: I'd rethink those state names and see if they can feel more
> consistent)
>
>> > Let's have a quick poll, which would you prefer and why:
>> >
>> > 1. openstack baremetal provision state --provide UUID
>> > 2. openstack baremetal provision --provide UUID
>> > 3. openstack baremetal provide UUID
>> > 4. openstack baremetal set provision state --provide UUID
>> > 5. openstack baremetal set state --provide UUID
>> > 6. openstack baremetal action --provide UUID
>>
>> I think my vote would be for #4 (or #5 if 'state' alone is enough to
>> convey the intent). I would love to get an OSC person's view on that
>> one. (Question already asked in another post)
>
>
> state by itself is not very meaningful.  if 'baremetal state' is meaningful
> to a user that might be OK.  But what thing has that state?  A node?  I
> don't know what 'provision state' also refers to, a node? a port?
>

There was some discussion on one of the patches about changing to
'baremetal node'. I was a little concerned by that since the original
'baremetal' commands have landed, but I think since they landed
post-Liberty, we wouldn't need to worry about a deprecation period. Am
I correct in thinking that?

> dt
>
> --
>
> Dean Troyer
> dtro...@gmail.com
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>



-- 
Brad P. Crochet, RHCA, RHCE, RHCVA, RHCDS
Principal Software Engineer
(c) 704.236.9385 (w) 919.301.3231

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Ironic] Quick poll: OpenStackClient command for provision action

2015-11-10 Thread Brad P. Crochet
On Tue, Nov 10, 2015 at 4:09 AM, Dmitry Tantsur  wrote:
> Hi all!
>
> I'd like to seek consensus (or at least some opinions) on patch
> https://review.openstack.org/#/c/206119/
> It proposed the following command:
>

I think it's time to actually just write up a spec on this. I think we
would be better served to spell it out now, and then more people can
contribute to both the spec and to the actual implementation once the
spec is approved.

WDYT?

>   openstack baremetal provision state --provide UUID
>
> (where --provide can also be --active, --deleted, --inspect, etc).
>
> I have several issues with this proposal:
>
> 1. IIUC the structure of an OSC command is "openstack noun verb". "provision
> state" is not a verb.
> 2. --active is not consistent with other options, which are verbs.
>
> Let's have a quick poll, which would you prefer and why:
>
> 1. openstack baremetal provision state --provide UUID
> 2. openstack baremetal provision --provide UUID
> 3. openstack baremetal provide UUID
> 4. openstack baremetal set provision state --provide UUID
> 5. openstack baremetal set state --provide UUID
> 6. openstack baremetal action --provide UUID
>
> I vote for #3. Though it's much more versbose, it reads very easily, except
> for "active". For active I'm thinking about changing it to "activate" or
> "provision".
>
> My next candidate is #6. Though it's also not a verb, it reads pretty
> easily.
>
> Thanks!
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



-- 
Brad P. Crochet, RHCA, RHCE, RHCVA, RHCDS
Principal Software Engineer
(c) 704.236.9385 (w) 919.301.3231

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Ironic] Quick poll: OpenStackClient command for provision action

2015-11-10 Thread Dean Troyer
On Tue, Nov 10, 2015 at 10:14 AM, Brad P. Crochet  wrote:

> There was some discussion on one of the patches about changing to
> 'baremetal node'. I was a little concerned by that since the original
> 'baremetal' commands have landed, but I think since they landed
> post-Liberty, we wouldn't need to worry about a deprecation period. Am
> I correct in thinking that?
>

While clients do have stable branches, we do not manage the deprecations,
etc, according to those timelines.  It's all straight time (3-6 months) or
release based.

The only real issue I see (so far) would be if there is a conflict and a
breaking change needs to be made.

dt

-- 

Dean Troyer
dtro...@gmail.com
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Ironic] [OSC] Quick poll: OpenStackClient command for provision action

2015-11-10 Thread Brad P. Crochet
On Tue, Nov 10, 2015 at 11:14 AM, Dean Troyer  wrote:
> On Tue, Nov 10, 2015 at 9:46 AM, Dmitry Tantsur  wrote:
>>
>>  inspect, manage, provide, active and abort are all provisioning verbs
>> used in ironic API. they usually represent some complex operations on a
>> node. Inspection is not related to showing, it's about fetching hardware
>> properties from hardware itself and updating ironic database. manage sets a
>> node to a specific ("manageable") state. etc.
>
>
> inspect seems like a very specific action and is probably OK as-is.  We
> should sanity-check other resources in OpenStack that it might also be used
> with down the road and how different the action might be.
>
> Those that are states of a resource should be handled with a set command.
>
>>
>> boot and shutdown are natural opposites, aka power on and power off.
>
>
> The analogous server commands (create/delete) may not make sense here
> because, unlike with a server (VM), a resource is not being created or
> deleted.  But a user might expect to use the same commands in both places.
> We need to consider which of those is more important.  I like to break ties
> on the side of user experience consistency.

baremetal create (the current command) does something different. It
creates the node in ironic. So I would agree that create/delete are
not analogous to the way Nova uses those verbs.

>
> Honestly, at some point as a user, I'd like to forget whether my server is a
> bare metal box or not and just use the same commands to manage it.
>
> Also, I'd LOVE to avoid using 'boot' at all just to get away from the nova
> command's use of it.
>
> dt
>
> --
>
> Dean Troyer
> dtro...@gmail.com
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>



-- 
Brad P. Crochet, RHCA, RHCE, RHCVA, RHCDS
Principal Software Engineer
(c) 704.236.9385 (w) 919.301.3231

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [TripleO] [Ironic] Let's stop hijacking other projects' OSC namespaces

2015-11-10 Thread Ben Nemec
+1 to moving anything we can into Ironic itself.

I do want to note that if we rename anything, we can't just rip and
replace.  We have users of the current commands, and we need to
deprecate those to give people a chance to move to the new ones.

A few more thoughts inline.

On 11/09/2015 06:44 AM, Dmitry Tantsur wrote:
> Hi OOO'ers, hopefully the subject caught your attentions :)
> 
> Currently, tripleoclient exposes several commands in "openstack 
> baremetal" and "openstack baremetal introspection" namespaces belonging 
> to ironic and ironic-inspector accordingly. TL;DR of this email is to 
> deprecate them and move to TripleO-specific namespaces. Read on to know why.
> 
> Problem
> ===
> 
> I realized that we're doing a wrong thing when people started asking me 
> why "baremetal introspection start" and "baremetal introspection bulk 
> start" behave so differently (the former is from ironic-inspector, the 
> latter is from tripleoclient). The problem with TripleO commands is that 
> they're highly opinionated workflows commands, but there's no way a user 
> can distinguish them from general-purpose ironic/ironic-inspector 
> commands. The way some of them work is not generic enough ("baremetal 
> import"), or uses different defaults from an upstream project 
> ("configure boot"), or does something completely unacceptable upstream 
> (e.g. the way "introspection bulk start" deals with node states).
> 
> So, here are commands that tripleoclient exposes with my comments:
> 
> 1. baremetal instackenv validate
> 
>   This command assumes there's an "baremetal instackenv" object, while 
> instackenv is a tripleo-specific file format.
> 
> 2. baremetal import
> 
>   This command supports a limited subset of ironic drivers and driver 
> properties, only those known to os-cloud-config.

True, although I feel like an "import from JSON" feature would not be
inappropriate for inclusion in Ironic.  I can't believe that we're the
only ones who would be interested in mass importing nodes from a very
common format like this.

> 
> 3. baremetal introspection bulk start
> 
>   This command does several bad (IMO) things:
>   a. Messes with ironic node states
>   b. Operates implicitly on all nodes (in a wrong state)

I thought that was fixed?  It used to try to introspect nodes that were
in an invalid state (like active), but it shouldn't anymore.

Unless your objection is that it introspects things in an available
state, which I think has to do with the state Ironic puts (or used to
put) nodes in after registration.  In any case, this one likely requires
some more discussion over how it should work.

>   c. Defaults to polling
> 
> 4. baremetal show capabilities
> 
>   This is the only commands that is generic enough and could actually 
> make it to ironicclient itself.
> 
> 5. baremetal introspection bulk status
> 
>   See "bulk start" above.
> 
> 6. baremetal configure ready state
> 
>   First of all, this and the next command use "baremetal configure" 
> prefix. I would not promise we'll never start using it in ironic, 
> breaking the whole TripleO.
> 
>   Seconds, it's actually DELL-specific.

Well, as I understand it we don't intend for it to be Dell-specific,
it's just that the Dell implementation is the only one that has been
done so far.

That said, since I think this is just layering some TripleO-specific
logic on top of the vendor-specific calls Ironic provides I agree that
it probably doesn't belong in the baremetal namespace.

> 
> 7. baremetal configure boot
> 
>   This one is nearly ok, but it defaults to local boot, which is not an 
> upstream default. Default values for images may not work outside of 
> TripleO as well.
> 
> Proposal
> 
> 
> As we already have "openstack undercloud" and "openstack overcloud" 
> prefixes for TripleO, I suggest we move these commands under "openstack 
> overcloud nodes" namespace. So we end up with:
> 
>   overcloud nodes import
>   overcloud nodes configure ready state --drac
>   overcloud nodes configure boot
> 
> As you see, I require an explicit --drac argument for "ready state" 
> command. As to the remaining commands:
> 
> 1. baremetal introspection status --all
> 
>This is fine to move to inspector-client, as inspector knows which 
> nodes are/were on introspection. We'll need a new API though.

+1

> 
> 2. baremetal show capabilities
> 
>We'll have this or similar command in ironic, hopefully this cycle.

\o/

> 
> 3. overcloud nodes introspect --poll --allow-available
> 
>I believe that we need to make 2 things explicit in this replacement 
> for "introspection bulk status": polling and operating on "available" nodes.

The only thing is that because this does the state manipulation, polling
is essentially the only way it can work, although moving this to the API
will change that.  Like I said above, I think this one is going to
require some ongoing discussion as part of the API work.

> 
> 4. overcloud nodes import --dry-run
> 
>could

Re: [openstack-dev] [Ironic] [OSC] Quick poll: OpenStackClient command for provision action

2015-11-10 Thread Dean Troyer
On Tue, Nov 10, 2015 at 9:46 AM, Dmitry Tantsur  wrote:

>  inspect, manage, provide, active and abort are all provisioning verbs
> used in ironic API. they usually represent some complex operations on a
> node. Inspection is not related to showing, it's about fetching hardware
> properties from hardware itself and updating ironic database. manage sets a
> node to a specific ("manageable") state. etc.
>

inspect seems like a very specific action and is probably OK as-is.  We
should sanity-check other resources in OpenStack that it might also be used
with down the road and how different the action might be.

Those that are states of a resource should be handled with a set command.


> boot and shutdown are natural opposites, aka power on and power off.
>

The analogous server commands (create/delete) may not make sense here
because, unlike with a server (VM), a resource is not being created or
deleted.  But a user might expect to use the same commands in both places.
We need to consider which of those is more important.  I like to break ties
on the side of user experience consistency.

Honestly, at some point as a user, I'd like to forget whether my server is
a bare metal box or not and just use the same commands to manage it.

Also, I'd LOVE to avoid using 'boot' at all just to get away from the nova
command's use of it.

dt

-- 

Dean Troyer
dtro...@gmail.com
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [tripleo] Location of TripleO REST API

2015-11-10 Thread Giulio Fidente

On 11/10/2015 04:47 PM, Dmitry Tantsur wrote:

On 11/10/2015 04:37 PM, Giulio Fidente wrote:

On 11/10/2015 04:16 PM, Dmitry Tantsur wrote:

On 11/10/2015 04:08 PM, Tzu-Mainn Chen wrote:

Hi all,

At the last IRC meeting it was agreed that the new TripleO REST API
should forgo the Tuskar name, and simply be called... the TripleO
API.  There's one more point of discussion: where should the API
live?  There are two possibilities:

a) Put it in tripleo-common, where the business logic lives.  If we
do this, it would make sense to rename tripleo-common to simply
tripleo.


+1 for both



b) Put it in its own repo, tripleo-api


if both the api (coming) and the cli (currently python-tripleoclient)
are meant to consume the shared code (business logic) from
tripleo-common, then I think it makes sense to keep each in its own repo
... so that we avoid renaming tripleo-common as well


tripleoclient should not consume tripleo-common


so FWIW I think my vote is different depending on the plans:

a. if python-tripleoclient will be changed so that it uses tripleo-api, 
then I'd vote for option 1) (have tripleo-api in -common and rename)


b. if python-tripleoclient will be changed so that it uses the shared 
library in -common, then I'd vote for for option 2) (have tripleo-api in 
its own repo)



on a side note, I think it should always be possible for the deployer to 
skip any business logic to give complete control over the template 
params for both the initial deployments and the updates

--
Giulio Fidente
GPG KEY: 08D733BA | IRC: giulivo

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Neutron] Reminder: Team meeting on Monday at 2100 UTC

2015-11-10 Thread Edgar Magana
Dear Vikram,

Our deepest condolences for you and your family.

Do not even think about work these days.

Edgar

From: Vikram Choudhary mailto:viks...@gmail.com>>
Reply-To: "OpenStack Development Mailing List (not for usage questions)" 
mailto:openstack-dev@lists.openstack.org>>
Date: Monday, November 9, 2015 at 7:18 PM
To: "OpenStack Development Mailing List (not for usage questions)" 
mailto:openstack-dev@lists.openstack.org>>
Subject: Re: [openstack-dev] [Neutron] Reminder: Team meeting on Monday at 2100 
UTC


Hi Team,

I won't be able to work for couple of weeks as my mother passed away yesterday.

Hope the team will understand and support in this difficult time.

Thanks
Vikram

On 10-Nov-2015 2:00 am, "Ihar Hrachyshka" 
mailto:ihrac...@redhat.com>> wrote:
GMT time zone is available by following the official instructions:

https://support.google.com/calendar/answer/37064?hl=en

"See other time zones in your calendar” section.

Ihar

Russell Bryant mailto:rbry...@redhat.com>> wrote:

Where do you find this?  I've been using Iceland, but this sounds more
explicit.  For me it makes me pick a country and then a TZ, so I'm not
seeing the "GMT (no daylight saving)" option anywhere.

--
Russell

On 11/09/2015 02:39 PM, Ihar Hrachyshka wrote:
There is also 'GMT (no daylight saving)’ TZ available in Google Calendar
that is identical to UTC for all practical matters.

Ihar

Carl Baldwin mailto:c...@ecbaldwin.net>> wrote:

I've been using Iceland's TZ for this.  Seems to work well and handle
the TZ changes nicely.

Carl

On Sat, Nov 7, 2015 at 7:24 AM, Sean M. Collins 
mailto:s...@coreitpro.com>>
wrote:
Learn from my mistake, check your calendar for the timezone if you've
created an event for the weekly meetings. Google makes it a hassle to
set things in UTC time, so I was caught by surprise by the FwaaS meeting
due to the DST change in the US of A.

--
Sean M. Collins

__

OpenStack Development Mailing List (not for usage questions)
Unsubscribe:
openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

__

OpenStack Development Mailing List (not for usage questions)
Unsubscribe:
openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: 
openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: 
openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: 
openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Zaqar] OpenStack Tokyo Summit Summary

2015-11-10 Thread gord chung
for reference, Julien is currently signed up for that task[1]. if this 
is a high priority requirement, we might want to see if another resource 
can pick it up, we can help out to explain how it should all be done :)


cheers,

[1] https://etherpad.openstack.org/p/mitaka-telemetry-todos

On 07/11/2015 2:09 PM, Fei Long Wang wrote:

Thanks, Doug. I will talk with Ceilometer team to see what we can do.

On 07/11/15 07:36, Doug Hellmann wrote:

Excerpts from Fei Long Wang's message of 2015-11-07 01:31:09 +1300:

Greetings,

Firstly, thank you for everyone joined Zaqar sessions at Tokyo summit.
We definitely made some great progress for those working sessions. Here
are the high level summary and those are basically our Mitaka
priorities. I may miss something so please feel free to comment/reply
this mail.

Sahara + Zaqar
-

We have a great discussion with Ethan Gafford from Sahara team. Sahara
team is happy to use Zaqar to fix some potential security issues. The
main user case will be covered in Mitaka is protecting tenant guest and
data from administrative user. So what Zaqar team needs to do in Mitaka
is completing the zaqar client function gaps for v2 to support signed
URL, which will be used by Sahara guest agent. Ethan will create a spec
in Sahara to track this work. This is a POC of what it'd look like to
have a guest agent in Sahara on top of Zaqar. The Sahara team has not
decided to use Zaqar yet but this would be the bases for that 
discussion.


Horizon + Zaqar
--

We used 1 horizon work session and 1 Zaqar work session to discuss this
topic. The main user case we would like to address is the async
notification so that Horizon won't have to poll the other OpenStack
components(e.g. Nova, Glance or Cinder) per second to get the latest
status. And I'm really happy to see we worked out a basic plan by
leveraging Zaqar's notification and websocket.

1. Implement a basic filter for Zaqar subscription, so that Zaqar can
decide if the message should be posted/forwarded to the subscriber when
there is a new message posted the queue. With this feature, Horizon 
will

only be notified by its interested notifications.

https://blueprints.launchpad.net/zaqar/+spec/suport-filter-for-subscription 



2. Listen OpenStack notifications

We may need more discussion about this to make sure if it should be in
the scope of Zaqar's services. It could be separated process/service of
Zaqar to listen/collect interested notifications/messages and post them
in to particular Zaqar queues. It sounds very interesting and useful 
but

we need to define the scope carefully for sure.

The Telemetry team discussed splitting out the code in Ceilometer that
listens for notifications to make it a more generic service that accepts
plugins to process events. One such plugin might be an interface to
filter events and republish them to Zaqar, so if you're interested in
working on that you might want to coordinate with the Telemetry team to
avoid duplicating effort.



Pool Group and Flavor
-

Thanks MD MADEEM proposed this topic so that we have a chance to review
the design of pool, pool group and flavor. Now the pool group and 
flavor

has a 1:1 mapping relationship and the pool group and pool has a 1:n
mapping relationship. But end user don't know the existence of pool, so
flavor is the way for end user to select what kind of storage(based on
capabilities) he want to use. Since pool group can't provide more
information than flavor so it's not really necessary, so we decide to
deprecate/remove it in Mitaka. Given this is hidden from users (done
automatically by Zaqar), there won't be an impact on the end user and
the API backwards compatibility will be kept.

https://blueprints.launchpad.net/zaqar/+spec/deprecate-pool-group

Zaqar Client


Some function gaps need to be filled in Mitaka. Personally, I would 
rate

the client work as the 1st priority of M since it's very key for the
integration with other OpenStack components. For v1.1, the support for
pool and flavor hasn't been completed. For v2, we're stilling missing
the support for subscription and signed URL.

https://blueprints.launchpad.net/zaqar/+spec/finish-client-support-for-v1.1-features 



SqlAlchemy Migration
-

Now we're missing the db migration support for SqlAlchemy, the control
plane driver. We will fix it in M as well.

https://blueprints.launchpad.net/zaqar/+spec/sqlalchemy-migration



Guys, please contribute this thread to fill the points/things I missed
or pop up in #openstack-zaqar channel directly with questions and
suggestions.

__ 


OpenStack Development Mailing List (not for usage questions)
Unsubscribe: 
openstack-dev-requ...@lists.openstack.org?subject:unsubscribe

http://lists.openstack.org/cgi-bin/mailman/listi

Re: [openstack-dev] [oslo] Graduate apiclient library

2015-11-10 Thread Adam Young

On 11/10/2015 06:45 AM, Victor Stinner wrote:

Hi,

Le 10/11/2015 07:29, Kekane, Abhishek a écrit :
We have done extensive analysis to figure out the differences of 
usages of

different oslo-incubator/openstack/common/apiclient modules used in the
respective Openstack python client libraries and added all our
observations in the google spreadsheet [2].


Good job! I added my +2 to your spec.
https://review.openstack.org/#/c/235200/



Possible resolutions:-
1) Remove auth_plugin.py from respective python client libraries and add
all required common functionality in auth.py. Add auth.py module into 
the

new apiclient library.
2) Remove auth.py completely and add auth_plugins.py module in the
respective python client libraries. No need to add auth.py into the new
apiclient library.
3) Check if keystoneauth library has all needed functionality present in
auth.py and auth_plugin.py. If present, don¹t include auth.py in the new
client library and eliminate auth_plugin.py from python client libraries
and instead use keystoneauth wherever needed.


I don't like (2): security matters, it's better to use the same code 
in all clients.


IMHO the best would be (3): put required functions directly in 
keystoneauth.



+2.  keystoneauth is designed to be the library explicitly for this.  Do 
not duplicate auth behavior anywhere else in the supported client 
libraries.


Is there, at this point, anything that you feel is missing from 
keystoneauth?  WRT Federation, I suspect it is ahead of what is in 
apiclient.  Feedback requested and greatly appreciated.





If it's not possible for whatever reason, (1) is my second favorite 
choice.




exceptions.py
===

Please refer to the exception classes present in the respective python
client libraries in the google spreadsheet ³exception_details².
All common exceptions from respective python client libraries should be
moved to the exception.py module and this module should be part of 
the new

apiclient library.


Agreed, but only common exceptions. For example, only solumclient 
defines and uses NotUnique, there is no need to put it the 
oslo.apiclient.




fake_client.py
===

Retain this module as it is for unit testing purpose.


It might be moved to oslo_apiclient/tests/ directory.



Possible resolutions:
1. Move utils.py to the new apiclient library and delete find_resource
method from all python client libraries and make them use it from the
apiclient library
2. Simply not include utils.py to the new apiclient library and 
implement

find_resource method in the manila client.
We prefer to implement option #1.


Since all clients need this function, I would prefer to put it in the 
new library. I hope that it's not to hard to refactorize the code to 
have one unique implementation.



Victor

__ 


OpenStack Development Mailing List (not for usage questions)
Unsubscribe: 
openstack-dev-requ...@lists.openstack.org?subject:unsubscribe

http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] Learning to Debug the Gate

2015-11-10 Thread Davanum Srinivas
Took about 35 mins or so :)

-- Dims

On Tue, Nov 10, 2015 at 10:45 AM, Matt Riedemann
 wrote:
>
>
> On 11/9/2015 3:54 PM, Anita Kuno wrote:
>>
>> On 11/05/2015 07:45 PM, Anita Kuno wrote:
>>>
>>> On 11/03/2015 05:30 PM, Anita Kuno wrote:

 On 11/02/2015 12:39 PM, Anita Kuno wrote:
>
> On 10/29/2015 10:42 PM, Anita Kuno wrote:
>>
>> On 10/29/2015 08:27 AM, Anita Kuno wrote:
>>>
>>> On 10/28/2015 12:14 AM, Matt Riedemann wrote:



 On 10/27/2015 4:08 AM, Anita Kuno wrote:
>
> Learning how to debug the gate was identified as a theme at the
> "Establish Key Themes for the Mitaka Cycle" cross-project session:
> https://etherpad.openstack.org/p/mitaka-crossproject-themes
>
> I agreed to take on this item and facilitate the process.
>
> Part one of the conversation includes referencing this video
> created by
> Sean Dague and Dan Smith:
> https://www.youtube.com/watch?v=fowBDdLGBlU
>
> Please consume this as you are able.
>
> Other suggestions for how to build on this resource were mentioned
> and
> will be coming in the future but this was an easy, actionable first
> step.
>
> Thank you,
> Anita.
>
>
> __
>
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe:
> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>


 https://www.openstack.org/summit/vancouver-2015/summit-videos/presentation/tales-from-the-gate-how-debugging-the-gate-helps-your-enterprise


>>>
>>> The source for the definition of "the gate":
>>>
>>> http://git.openstack.org/cgit/openstack-infra/project-config/tree/zuul/layout.yaml#n34
>>>
>>> Thanks for following along,
>>> Anita.
>>>
>>
>> This is the status page showing the status of our running jobs,
>> including patches in the gate pipeline:
>> http://status.openstack.org/zuul/
>>
>> Thank you,
>> Anita.
>>
>
> This is a simulation of how the gate tests patches:
> http://docs.openstack.org/infra/publications/zuul/#%2818%29
>
> Click in the browser window to advance the simulation.
>
> Thank you,
> Anita.
>

 Here is a presentation that uses the slide deck linked above, I
 recommend watching: https://www.youtube.com/watch?v=WDoSCGPiFDQ

 Thank you,
 Anita.

>>>
>>> Three links in this edition of Learning to Debug the Gate:
>>>
>>> The view that tracks our top bugs:
>>> http://status.openstack.org/elastic-recheck/
>>>
>>> The logstash queries that create the above view:
>>>
>>> http://git.openstack.org/cgit/openstack-infra/elastic-recheck/tree/queries
>>>
>>> Logstash itself, where you too can practice creating queries:
>>> http://logstash.openstack.org
>>>
>>> Note: in logstash the query is the transferable piece of information.
>>> Filters can help you create a query, they do not populate a query. The
>>> information that is in the query bar is what is important here.
>>>
>>> Practice making some queries of your own.
>>>
>>> Thanks for reading,
>>> Anita.
>>>
>>
>> Today the elastic search cluster was upgraded to 1.7.3:
>>
>> http://lists.openstack.org/pipermail/openstack-dev/2015-November/078314.html
>>
>> Go over the 1.7 elastic search docs:
>> https://www.elastic.co/guide/en/elasticsearch/reference/1.7/index.html
>> and try a few queries.
>>
>> Thanks for following along,
>> Anita.
>>
>> __
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>
> If you want a fun little play by play of a critical blocking gate issue for
> nova, it's in the nova irc logs starting here:
>
> http://eavesdrop.openstack.org/irclogs/%23openstack-nova/%23openstack-nova.2015-11-10.log.html#t2015-11-10T14:59:27
>
> Which led to a block of oslo.messaging 2.8.0 in g-r and a revert in the o.m
> repo. The point isn't the outcome, but the process in which we figured out
> what the regression was and quickly resolved it using logstash and seeing
> what recent dependent library releases impacted nova.
>
> --
>
> Thanks,
>
> Matt Riedemann
>
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



-- 
Davanum Srinivas :: https:/

Re: [openstack-dev] [heat][keystone] How to handle request for global admin in policy.json?

2015-11-10 Thread Adam Young

On 11/10/2015 05:08 AM, Henry Nash wrote:

Steve,

Currently, your best option is to use something similar to the 
policy.v3cloudsample.json, where you basically “bless” a project (or domain) as 
being the “cloud admin project/domain”.  Having a role on that gives you 
super-powers.  The only trouble with this right now is that you have to paste 
the ID of your blessed project/domain into the policy file (you only have to do 
that once, of course) - basically you replace the “admin_domain_id” with the ID 
of your blessed project/domain.

What we are considering for Mitaka is make this a bit more friendly, so you 
don’t have to modify the policy file - rather you define your “blessed project” 
in your config file, and tokens that are issue on this blessed project will 
have an extra attribute (e.g. “is_admin_project”), which your policy file can 
check for.


Henry is using a bitof the British tendency toward understatement here.  
Let me make this more explicit:


We are going to add a value to the Keystone token validation response 
that will indicate that the proejct is an admin project. Use that.  
Don't develop something for Mitaka that does not use that.


The spec is here

https://review.openstack.org/#/c/242232/

And, as you can see, Henry and I are working on getting it tight enough 
to stand the test of time.


There is a sample implementation underway here:

https://review.openstack.org/240719


Henry

On 10 Nov 2015, at 09:50, Steven Hardy  wrote:

Hi all,

Seeking some guidance (particularly from the keystone folks) ref this bug:

https://bugs.launchpad.net/heat/+bug/1466694

tl;dr - Heat has historically been careful to make almost all requests
scoped to exactly one project.  Being aware of the long-standing bug
#968696, we've deliberately avoided using any global "is admin" flag
derived from the admin role.

However, we're now being told this is operator hostile, and that we should
provide an option for policy.json to enable global admin, because other
projects do it.

What is the best-practice solution to this requirement?

I'm assuming (to avoid being added to bug #968696) that we must not enable
global admin by default, but is it acceptable to support optional custom
policy.json which defeats the default tenant scoping for a requst?

For example, in policy.v3cloudsample.json[1] there are several options in
terms of "admin-ness", including admin_required which appears to be the
traditional global-admin based on the admin role.

It's quite confusing, are there any docs around best-practices for policy
authors and/or what patterns services are supposed to support wrt policy?

I'm wondering if we should we be doing something like this in our default
policy.json[2]?

"admin_required": "role:admin",
"cloud_admin": "rule:admin_required and domain_id:admin_domain_id",
"owner" : "user_id:%(user_id)s or user_id:%(target.token.user_id)s",

"stacks:delete": "rule:owner or rule:cloud_admin"

I'm not yet quite clear where admin_domain_id is specified?

Any guidance or thoughts would be much appreciated - I'm keen to resolve
this pain-point for operators, but not in a way that undermines the
OpenStack-wide desire to move away from global-admin by default.

Thanks!

Steve

[1] 
https://github.com/openstack/keystone/blob/master/etc/policy.v3cloudsample.json
[2] https://github.com/openstack/heat/blob/master/etc/heat/policy.json

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [TripleO] Proposing Ian Wienand as core reviewer on diskimage-builder

2015-11-10 Thread Paul Belanger
On Tue, Nov 03, 2015 at 03:25:27PM +, Gregory Haynes wrote:
> Hello everyone,
> 
> I would like to propose adding Ian Wienand as a core reviewer on the
> diskimage-builder project. Ian has been making a significant number of
> contributions for some time to the project, and has been a great help in
> reviews lately. Thus, I think we could benefit greatly by adding him as
> a core reviewer.
> 
> Current cores - Please respond with any approvals/objections by next Friday
> (November 13th).
> 
+1 he did some great work getting Fedora 22 working with DNF.

> Cheers,
> Greg
> 
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


  1   2   3   >