Re: [openstack-dev] [cinder] [nova] os-brick privsep failures and an upgrade strategy?

2016-07-22 Thread Angus Lees
On Thu, 21 Jul 2016 at 09:27 Sean Dague wrote: > On 07/12/2016 06:25 AM, Matt Riedemann wrote: > > > We probably aren't doing anything while Sean Dague is on vacation. He's > > back next week and we have the nova/cinder meetups, so I'm planning on > > talking about the grenade

Re: [openstack-dev] [grenade] upgrades vs rootwrap

2016-07-06 Thread Angus Lees
On Thu, 7 Jul 2016 at 03:06 Matthew Treinish wrote: > On Wed, Jul 06, 2016 at 11:41:56AM -0500, Matt Riedemann wrote: > > I just wonder how many deployments are actually relying on this, since as > > noted elsewhere in this thread we don't really enforce this for all >

Re: [openstack-dev] [grenade] upgrades vs rootwrap

2016-07-03 Thread Angus Lees
On Sat, 2 Jul 2016 at 01:02 Matt Riedemann <mrie...@linux.vnet.ibm.com> wrote: > On 6/28/2016 4:56 PM, Sean Dague wrote: > > On 06/28/2016 01:46 AM, Angus Lees wrote: > >> Ok, thanks for the in-depth explanation. > >> > >> My take away is that we need

Re: [openstack-dev] [grenade] upgrades vs rootwrap

2016-06-27 Thread Angus Lees
Ok, thanks for the in-depth explanation. My take away is that we need to file any rootwrap updates as exceptions for now (so releasenotes and grenade scripts). - Gus On Mon, 27 Jun 2016 at 21:25 Sean Dague <s...@dague.net> wrote: > On 06/26/2016 10:02 PM, Angus Lees wrote: > >

Re: [openstack-dev] [grenade] upgrades vs rootwrap

2016-06-26 Thread Angus Lees
On Mon, 27 Jun 2016 at 12:59 Tony Breeds <t...@bakeyournoodle.com> wrote: > On Mon, Jun 27, 2016 at 02:02:35AM +0000, Angus Lees wrote: > > > *** > > What are we trying to impose on ourselves for upgrades for the present > and > > near future (ie

Re: [openstack-dev] [grenade] upgrades vs rootwrap

2016-06-26 Thread Angus Lees
On Fri, 24 Jun 2016 at 19:13 Thierry Carrez wrote: > In summary, I think the choice is between (1)+(4) and doing (4) > directly. How doable is (4) in the timeframe we have ? Do we all agree > that (4) is the endgame ? > I don't make predictions about development timelines

Re: [openstack-dev] [grenade] upgrades vs rootwrap

2016-06-26 Thread Angus Lees
On Fri, 24 Jun 2016 at 20:48 Sean Dague wrote: > On 06/24/2016 05:12 AM, Thierry Carrez wrote: > > I'm adding Possibility (0): change Grenade so that rootwrap filters from > > N+1 are put in place before you upgrade. > > If you do that as general course what you are saying is

Re: [openstack-dev] [grenade] upgrades vs rootwrap

2016-06-24 Thread Angus Lees
On Fri, 24 Jun 2016 at 21:04 Sean Dague wrote: > On 06/24/2016 05:19 AM, Daniel P. Berrange wrote: > > On Fri, Jun 24, 2016 at 11:12:27AM +0200, Thierry Carrez wrote: > >> No perfect answer here... I'm hesitating between (0), (1) and (4). (4) > is > >> IMHO the right solution,

Re: [openstack-dev] [grenade] upgrades vs rootwrap

2016-06-24 Thread Angus Lees
On Fri, 24 Jun 2016 at 00:40 Sean Dague <s...@dague.net> wrote: > On 06/23/2016 10:07 AM, Sean McGinnis wrote: > > On Thu, Jun 23, 2016 at 12:19:34AM +0000, Angus Lees wrote: > >> So how does rootwrap fit into the "theory of upgrade"? > >>

[openstack-dev] [grenade] upgrades vs rootwrap

2016-06-22 Thread Angus Lees
So how does rootwrap fit into the "theory of upgrade"? The doc talks about deprecating config, but is silent on when new required config (rootwrap filters) should be installed. By virtue of the way the grenade code works (install N from scratch, attempt to run code from N+1), we effectively have

Re: [openstack-dev] [cinder] [nova] os-brick privsep failures and an upgrade strategy?

2016-06-21 Thread Angus Lees
On Wed, 22 Jun 2016 at 05:59 Matt Riedemann wrote: > Angus, what should we be looking at from the privsep side for debugging > this? > The line above the screen-n-cpu.txt.gz failure you linked to is: 2016-06-21 16:21:30.994

Re: [openstack-dev] [cinder] [nova] os-brick privsep failures and an upgrade strategy?

2016-06-15 Thread Angus Lees
exploring? - Gus On Wed, 15 Jun 2016 at 11:43 Sean Dague <s...@dague.net> wrote: > On 06/14/2016 06:11 PM, Angus Lees wrote: > > Yep (3) is quite possible, and the only reason it doesn't just do this > > already is because there's no way to find the name of the rootwrap > &g

Re: [openstack-dev] [cinder] [nova] os-brick privsep failures and an upgrade strategy?

2016-06-14 Thread Angus Lees
On Tue, 14 Jun 2016 at 23:04 Daniel P. Berrange wrote: > On Tue, Jun 14, 2016 at 07:49:54AM -0400, Sean Dague wrote: > > [snip] > Urgh, thanks for the in-depth analysis :/ > The crux of the problem is that os-brick 1.4 and privsep can't be used > > without a config file

Re: [openstack-dev] [nova] Initial oslo.privsep conversion?

2016-06-13 Thread Angus Lees
One of the challenges with nova (and I'm working from some earlier conversations, not a first-hand reading of the code) is that we can't restrict file operations to any particular corner of the filesystem, because the location of the libvirt data is stored (only) in the database, and the database

Re: [openstack-dev] [tc] supporting Go

2016-05-10 Thread Angus Lees
No, it doesn't. Several applications written in go are already packaged for Debian (for example). Indeed the equivalent of "installing from master/pip" (ie: not using distro packages) is _much_ easier in go, since there is no need for the equivalent of venvs. Like all compiled languages, there

Re: [openstack-dev] [tc] supporting Go

2016-05-09 Thread Angus Lees
On Tue, 10 May 2016 at 05:19 Edward Leafe wrote: > On May 9, 2016, at 1:58 PM, Hayes, Graham wrote: > > > This is not a "Go seems cool - lets go try that" decision from us - we > > know we have a performance problem with one of our components, and we > >

Re: [openstack-dev] [nova][neutron] os-vif status report

2016-04-21 Thread Angus Lees
In case it wasn't already assumed, anyone is welcome to contact me directly (irc: gus, email, or in Austin) if they have questions or want help with privsep integration work. It's early days still and the docs aren't extensive (ahem). os-brick privsep change just recently merged (yay), and I

Re: [openstack-dev] [os-brick][nova][cinder] os-brick/privsep change is done and awaiting your review

2016-03-22 Thread Angus Lees
On Sat, 19 Mar 2016 at 06:27 Matt Riedemann wrote: > I stared pretty hard at the nova rootwrap filter change today [1] and > tried to keep that in my head along with the devstack change and the > changes to os-brick (which depend on the devstack/cinder/nova changes).

Re: [openstack-dev] [os-brick][nova][cinder] os-brick/privsep change is done and awaiting your review

2016-02-25 Thread Angus Lees
On Fri, 26 Feb 2016 at 01:28 John Garbutt wrote: > Agreed with the concerns here, but I thought these are the same we > raised at the midcycle. > Yes, afaik everything is exactly as we discussed and following the direction we agreed at Nova+CInder mid-cycles. In

Re: [openstack-dev] [os-brick][nova][cinder] os-brick/privsep change is done and awaiting your review

2016-02-24 Thread Angus Lees
re). Please continue to ask questions, since I know very few people have actually looked at any of privsep nor the os-brick change until now. - Gus > On 24 February 2016 at 14:59, Michał Dulko <michal.du...@intel.com> wrote: > >> On 02/24/2016 04:51 AM, Angus Lees wrot

[openstack-dev] [os-brick][nova][cinder] os-brick/privsep change is done and awaiting your review

2016-02-23 Thread Angus Lees
Re: https://review.openstack.org/#/c/277224 Most of the various required changes have flushed out by now, and this change now passes the dsvm-full integration tests(*). (*) well, the experimental job anyway. It still relies on a merged-but-not-yet-released change in oslo.privsep so gate + 3rd

Re: [openstack-dev] [hyper-v] oslo.privsep vs Windows

2015-12-03 Thread Angus Lees
On Tue, 1 Dec 2015 at 00:20 Thierry Carrez wrote: > Sean Dague wrote: > > Right, so to me this seems that privsep just needs a NULL mode, and > > we're done. If oslo.rootrwap was never supported on windows, I don't > > think privsep really needs to be in a real way. > > +1

Re: [openstack-dev] [hyper-v] oslo.privsep vs Windows

2015-11-25 Thread Angus Lees
s://github.com/openstack/oslo.rootwrap/blob/master/oslo_rootwrap/wrapper.py#L19 > > If you have any further questions, feel free to ask. :) > > Best regards, > Claudiu Belu > > > -- > *From:* Angus Lees [g...@inodes.org] > *Sent:* Tuesday, November 24, 2015 6:18

Re: [openstack-dev] [hyper-v] oslo.privsep vs Windows

2015-11-25 Thread Angus Lees
ng confident enough to > send a patch to Gerrit and wait for CI results. And for that you don’t need > tox or anything fancy, just a Python environment and some very basic OS > skills. > > When in doubt, an option is to drop by on #openstack-hyper-v and ask. But > take care,

Re: [openstack-dev] [hyper-v] oslo.privsep vs Windows

2015-11-25 Thread Angus Lees
portable code (or code that doesn’t behave as expected). > > Alessandro > > > > From: Angus Lees <g...@inodes.org> > Reply-To: "OpenStack Development Mailing List (not for usage questions)" < > openstack-dev@lists.openstack.org> > Date: Thursday 26 November

Re: [openstack-dev] [hyper-v] oslo.privsep vs Windows

2015-11-25 Thread Angus Lees
On Thu, 26 Nov 2015 at 14:19 Robert Collins wrote: > On 26 November 2015 at 15:54, Alessandro Pilotti > wrote: > > When done, open a PowerShell or command prompt and set your PATH and > > PYTHONPATH e.g.: > > > > $ENV:PATH +=

[openstack-dev] [hyper-v] oslo.privsep vs Windows

2015-11-23 Thread Angus Lees
Dims has just raised[1] the excellent concern that oslo.privsep will need to at least survive on Windows, because hyper-v. I have no real experience coding on windows (I wrote a windows C program once, but I only ever ran it under wine ;) and certainly none within an OpenStack/python context: 1)

Re: [openstack-dev] [all] [oslo] Please stop removing usedevelop from tox.ini (at least for now)

2015-11-23 Thread Angus Lees
lins <robe...@robertcollins.net> wrote: > Workaround for what? > > -Rob > > On 23 November 2015 at 19:39, Angus Lees <g...@inodes.org> wrote: > > The workaround I found is to set this in tox.ini: > > > > install_command = > > sh -c 'pip install

Re: [openstack-dev] [all] [oslo] Please stop removing usedevelop from tox.ini (at least for now)

2015-11-22 Thread Angus Lees
The workaround I found is to set this in tox.ini: install_command = sh -c 'pip install -U "pip>=6" && pip install -U "$@"' pip {opts} {packages} It's pretty ugly, but works fine and was unfortunately the only way I could find to update a dependency before parsing requirements.txt. - Gus On

Re: [openstack-dev] [neutron] [oslo.privsep] Any progress on privsep?

2015-09-17 Thread Angus Lees
On Fri, 18 Sep 2015 at 14:13 Li Ma wrote: > Hi stackers, > > Currently we are discussing the possibility of using a pure python > library to configure network in neutron [1]. We find out that it is > impossible to do it without privsep, because we run external commands

[openstack-dev] [oslo.serialization] Security or convenience?

2015-07-23 Thread Angus Lees
I'm working on a draft spec[1] for a new privilege separation mechanism (oslo.privsep) and one of the reviewers mentioned oslo.serialization. Yay. My question is: From a quick glance over the current objects, it looks fine atm - but is the intention that this library remain suitable for

Re: [openstack-dev] [nova][cinder][neutron][security] Rootwrap discussions at OSSG mid-cycle

2015-05-14 Thread Angus Lees
On Wed, 13 May 2015 at 02:16 Thierry Carrez thie...@openstack.org wrote: Lucas Fisher wrote: We spent some time at the OSSG mid-cycle meet-up this week discussing root wrap, looking at the existing code, and considering some of the mailing list discussions. Summary of our discussions:

Re: [openstack-dev] [all] Replace mysql-python with mysqlclient

2015-05-11 Thread Angus Lees
On Tue, 12 May 2015 at 05:08 Mike Bayer mba...@redhat.com wrote: On 5/11/15 2:02 PM, Attila Fazekas wrote: The scary part of a blocking I/O call is when you have two python thread (or green thread) and one of them is holding a DB lock the other is waiting for the same lock in a native

Re: [openstack-dev] [neutron] Need guidance - functional tests and rootwrap

2015-04-26 Thread Angus Lees
The tox.ini entry for dsvm-fullstack sets OS_ROOTWRAP_CMD=sudo {envbindir}/neutron-rootwrap {envdir}/etc/neutron/rootwrap.conf (and something similar for rootwrap-daemon). Is this the answer you were looking for, or are you saying OS_ROOTWRAP_CMD doesn't appear to be honoured in your case? -

Re: [openstack-dev] [oslo] eventlet 0.17.3 is now fully Python 3 compatible

2015-04-23 Thread Angus Lees
On Thu, 23 Apr 2015 at 17:11 Victor Stinner vstin...@redhat.com wrote: As announced, changes are boring, just obvious Python2/Python3 issues: - strip L from long integer literals: 123L = 123 - replace dict.iteritems() with six.iteritems(dict) - replace list.sort(cmp_func) with

Re: [openstack-dev] Fwd: [Neutron][DVR]Neutron distributed SNAT

2015-02-18 Thread Angus Lees
On Mon Feb 16 2015 at 9:37:22 PM Kevin Benton blak...@gmail.com wrote: It's basically very much like floating IPs, only you're handing out a sub-slice of a floating-IP to each machine - if you like. This requires participation of the upstream router (L4 policy routing pointing to next hops

Re: [openstack-dev] Fwd: [Neutron][DVR]Neutron distributed SNAT

2015-02-16 Thread Angus Lees
Conntrack synchronisation gets us HA on the SNAT node, but that's a long way from distributed SNAT. Distributed SNAT (in at least one implementation) needs a way to allocate unique [IP + ephemeral port ranges] to hypervisors, and then some sort of layer4 loadbalancer capable of forwarding the

Re: [openstack-dev] [neutron][security][rootwrap] Proposal to replace rootwrap/sudo with privsep helper process (for neutron, but others too)

2015-02-15 Thread Angus Lees
/sudoers. - Gus Miguel Ángel Ajo On Friday, 13 de February de 2015 at 10:54, Angus Lees wrote: On Fri Feb 13 2015 at 5:45:36 PM Eric Windisch e...@windisch.us wrote: ᐧ from neutron.agent.privileged.commands import ip_lib as priv_ip def foo(): # Need to create a new veth

Re: [openstack-dev] [neutron][security][rootwrap] Proposal to replace rootwrap/sudo with privsep helper process (for neutron, but others too)

2015-02-13 Thread Angus Lees
On Fri Feb 13 2015 at 5:45:36 PM Eric Windisch e...@windisch.us wrote: ᐧ from neutron.agent.privileged.commands import ip_lib as priv_ip def foo(): # Need to create a new veth interface pair - that usually requires root/NET_ADMIN priv_ip.CreateLink('veth', 'veth0',

Re: [openstack-dev] [neutron][security][rootwrap] Proposal to replace rootwrap/sudo with privsep helper process (for neutron, but others too)

2015-02-12 Thread Angus Lees
So inspired by the Rootwrap on root-intensive nodes thread, I went and wrote a proof-of-concept privsep daemon for neutron: https://review.openstack.org/#/c/155631 There's nothing neutron-specific in the core mechanism and it could easily be moved out into a common (oslo) library and reused across

Re: [openstack-dev] [neutron][security][rootwrap] Proposal to replace rootwrap/sudo with privsep helper process (for neutron, but others too)

2015-02-12 Thread Angus Lees
On Fri Feb 13 2015 at 4:05:33 PM Robert Collins robe...@robertcollins.net wrote: On 13 Feb 2015 17:42, Angus Lees g...@inodes.org wrote: So inspired by the Rootwrap on root-intensive nodes thread, I went and wrote a proof-of-concept privsep daemon for neutron: https://review.openstack.org

Re: [openstack-dev] [all][oslo.db][nova] TL; DR Things everybody should know about Galera

2015-02-06 Thread Angus Lees
Thanks for the additional details Peter. This confirms the parts I'd deduced from the docs I could find, and is useful knowledge. On Sat Feb 07 2015 at 2:24:23 AM Peter Boros peter.bo...@percona.com wrote: - Like many others said it before me, consistent reads can be achieved with

Re: [openstack-dev] [kolla][tripleo] Update on Kolla

2015-02-05 Thread Angus Lees
Without taking anything away from this fine and sensible direction, where _would_ be a good place to discuss more radical container-based deployments? (I'm fine if the answer is use ad-hoc communication channels for now) On Fri Feb 06 2015 at 7:24:12 AM Steven Dake (stdake) std...@cisco.com

Re: [openstack-dev] [all][oslo.db][nova] TL; DR Things everybody should know about Galera

2015-02-05 Thread Angus Lees
On Fri Feb 06 2015 at 4:25:43 AM Clint Byrum cl...@fewbar.com wrote: In a single thread, using a single database session, then a read after successful commit is guaranteed to read a version of the database that existed after that commit. Ah, I'm relieved to hear this clarification - thanks.

Re: [openstack-dev] [all][oslo.db][nova] TL; DR Things everybody should know about Galera

2015-02-05 Thread Angus Lees
On Fri Feb 06 2015 at 12:59:13 PM Gregory Haynes g...@greghaynes.net wrote: Excerpts from Joshua Harlow's message of 2015-02-06 01:26:25 +: Angus Lees wrote: On Fri Feb 06 2015 at 4:25:43 AM Clint Byrum cl...@fewbar.com mailto:cl...@fewbar.com wrote: I'd also like to see

Re: [openstack-dev] [neutron] high dhcp lease times in neutron deployments considered harmful (or not???)

2015-02-04 Thread Angus Lees
On Wed Feb 04 2015 at 8:02:04 PM Kevin Benton blak...@gmail.com wrote: I proposed an alternative to adjusting the lease time early on the in the thread. By specifying the renewal time (DHCP option 58), we can have the benefits of a long lease time (resiliency to long DHCP server outages)

Re: [openstack-dev] [all][oslo.db][nova] TL; DR Things everybody should know about Galera

2015-02-04 Thread Angus Lees
On Thu Feb 05 2015 at 9:02:49 AM Robert Collins robe...@robertcollins.net wrote: On 5 February 2015 at 10:24, Joshua Harlow harlo...@outlook.com wrote: How interesting, Why are people using galera if it behaves like this? :-/ Because its actually fairly normal. In fact its an instance of

Re: [openstack-dev] [neutron] high dhcp lease times in neutron deployments considered harmful (or not???)

2015-02-03 Thread Angus Lees
There's clearly not going to be any amount of time that satisfies both concerns here. Just to get some other options on the table, here's some things that would allow a non-zero dhcp lease timeout _and_ address Kevin's original bug report: - Just don't allow users to change their IPs without a

Re: [openstack-dev] [oslo.db] PyMySQL review

2015-01-29 Thread Angus Lees
The other factor I haven't seen mentioned here is that our usual eventlet+mysqldb setup can deadlock rather easily(*), resulting in the entire python process sitting idle for 30s until the mysqldb deadlock timer goes off and raises an exception. At this point (in nova at least), the request is

[openstack-dev] [neutron] Linux capabilities vs sudo/rootwrap?

2014-12-09 Thread Angus Lees
[I tried to find any previous discussion of this and failed - I'd appreciate a pointer to any email threads / specs where this has already been discussed.] Currently neutron is given the ability to do just about anything to networking via rootwrap, sudo, and the IpFilter check (allow anything

Re: [openstack-dev] [neutron] Linux capabilities vs sudo/rootwrap?

2014-12-09 Thread Angus Lees
sudo/rootwrap is still required. I'm not suggesting we can (or should!) get rid of it entirely - just for basic network configuration. On Wed Dec 10 2014 at 12:42:46 PM George Shuklin george.shuk...@gmail.com wrote: Is ovs-vsctl gonna be happy with CAP_NET_ADMIN? On 12/10/2014 02:43 AM, Angus

Re: [openstack-dev] [tc][neutron] Proposal to split Neutron into separate repositories

2014-12-01 Thread Angus Lees
On Mon Dec 01 2014 at 2:06:18 PM henry hly henry4...@gmail.com wrote: My suggestion is that starting with LB and VPN as a trial, which can never be distributed. .. Sure they can! Loadbalancing in particular _should_ be distributed if both the clients and backends are in the same cluster...

Re: [openstack-dev] [stable] Re: [neutron] the hostname regex pattern fix also changed behaviour :(

2014-11-30 Thread Angus Lees
On Fri Nov 28 2014 at 10:49:21 PM Ihar Hrachyshka ihrac...@redhat.com wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA512 On 28/11/14 01:26, Angus Lees wrote: Context: https://review.openstack.org/#/c/135616 If we're happy disabling the check for components being all-digits

[openstack-dev] [neutron] the hostname regex pattern fix also changed behaviour :(

2014-11-27 Thread Angus Lees
Context: https://review.openstack.org/#/c/135616 As far as I can make out, the fix for CVE-2014-7821 removed a backslash that effectively disables the negative look-ahead assertion that verifies that hostname can't be all-digits. Worse, the new version now rejects hostnames where a component

Re: [openstack-dev] [neutron][nova] New specs on routed networking

2014-10-28 Thread Angus Lees
On Tue, 28 Oct 2014 09:07:03 PM Rohit Agarwalla wrote: Agreed. The way I'm thinking about this is that tenants shouldn't care what the underlying implementation is - L2 or L3. As long as the connectivity requirements are met using the model/API, end users should be fine. The data center

Re: [openstack-dev] [Neutron][LBaaS][Octavia] Usage Requirements

2014-10-28 Thread Angus Lees
. add complexity). At the end of the day there is no free lunch so more insight is appreciated. Thanks for the feedback. Cheers, --Jorge On 10/27/14 6:55 PM, Angus Lees g...@inodes.org wrote: On Wed, 22 Oct 2014 11:29:27 AM Robert van Leeuwen wrote: I,d like to start a conversation

Re: [openstack-dev] [neutron][nova] New specs on routed networking

2014-10-28 Thread Angus Lees
On Wed, 29 Oct 2014 12:21:10 AM Fred Baker wrote: On Oct 28, 2014, at 4:59 PM, Angus Lees g...@inodes.org wrote: On Tue, 28 Oct 2014 09:07:03 PM Rohit Agarwalla wrote: Agreed. The way I'm thinking about this is that tenants shouldn't care what the underlying implementation is - L2 or L3

Re: [openstack-dev] [all] Key signing at the summit?

2014-10-27 Thread Angus Lees
On Mon, 27 Oct 2014 02:45:27 PM Jeremy Stanley wrote: If there is interest in doing another Sassaman-Projected Method exercise at future events, a USB document camera would be useful to procure in advance Is this what the young kids these days are calling a phone? -- - Gus

Re: [openstack-dev] [kolla] on Dockerfile patterns

2014-10-16 Thread Angus Lees
On Wed, 15 Oct 2014 08:19:03 PM Clint Byrum wrote: I think it would be a good idea for containers' filesystem contents to be a whole distro. What's at question in this thread is what should be running. If we can just chroot into the container's FS and run apt-get/yum install our

Re: [openstack-dev] [kolla] on Dockerfile patterns

2014-10-15 Thread Angus Lees
On Wed, 15 Oct 2014 09:51:03 AM Clint Byrum wrote: Excerpts from Vishvananda Ishaya's message of 2014-10-15 07:52:34 -0700: On Oct 14, 2014, at 1:12 PM, Clint Byrum cl...@fewbar.com wrote: Excerpts from Lars Kellogg-Stedman's message of 2014-10-14 12:50:48 -0700: On Tue, Oct 14, 2014 at

Re: [openstack-dev] [kolla] on Dockerfile patterns

2014-10-15 Thread Angus Lees
On Wed, 15 Oct 2014 12:40:16 AM Fox, Kevin M wrote: Systemd has invested a lot of time/effort to be able to relaunch failed services, support spawning and maintaining unix sockets and services across them, etc, that you'd have to push out of and across docker containers. All of that can be

Re: [openstack-dev] [kolla] on Dockerfile patterns

2014-10-14 Thread Angus Lees
On Tue, 14 Oct 2014 07:51:54 AM Steven Dake wrote: Angus, On 10/13/2014 08:51 PM, Angus Lees wrote: I've been reading a bunch of the existing Dockerfiles, and I have two humble requests: 1. It would be good if the interesting code came from python sdist/bdists rather than rpms

Re: [openstack-dev] [neutron][all] Naming convention for unused variables

2014-10-14 Thread Angus Lees
On Tue, 14 Oct 2014 12:28:29 PM Angus Lees wrote: (Context: https://review.openstack.org/#/c/117418/) I'm looking for some rough consensus on what naming conventions we want for unused variables in Neutron, and across the larger OpenStack python codebase since there's no reason for Neutron

[openstack-dev] [neutron][all] Naming convention for unused variables

2014-10-13 Thread Angus Lees
(Context: https://review.openstack.org/#/c/117418/) I'm looking for some rough consensus on what naming conventions we want for unused variables in Neutron, and across the larger OpenStack python codebase since there's no reason for Neutron to innovate here. As far as I can see, there are two

[openstack-dev] [kolla] on Dockerfile patterns

2014-10-13 Thread Angus Lees
I've been reading a bunch of the existing Dockerfiles, and I have two humble requests: 1. It would be good if the interesting code came from python sdist/bdists rather than rpms. This will make it possible to rebuild the containers using code from a private branch or even unsubmitted code,

Re: [openstack-dev] [kolla] kolla, keystone, and endpoints (oh my!)

2014-10-07 Thread Angus Lees
On Mon, 6 Oct 2014 10:33:21 AM Lars Kellogg-Stedman wrote: Hello all, I wanted to expand on a discussion we had in #tripleo on Friday regarding Kubernetes and the Keystone service catalog. ## The problem The essential problem is that Kubernetes and Keystone both provide service

Re: [openstack-dev] [Ceilometer] Adding pylint checking of new ceilometer patches

2014-10-03 Thread Angus Lees
You can turn off lots of the refactor recommendation checks. I've been running pylint across neutron and it's uncovered half a dozen legitimate bugs so far - and that's with many tests still disabled. I agree that the defaults are too noisy, but its about the only tool that does linting across

[openstack-dev] [kolla] What I already have for openstack/kubernetes/docker

2014-10-02 Thread Angus Lees
Sorry I couldn't make the IRC meeting. sdake quite rightly suggested I send this to the broader list for dissection. I spent yesterday templatising my k8s configs so I could publish them without revealing all my passwords ;) https://github.com/anguslees/kube-openstack Please take a look

Re: [openstack-dev] [kolla] Kolla Blueprints

2014-10-01 Thread Angus Lees
On Wed, 1 Oct 2014 09:05:23 PM Fox, Kevin M wrote: Has anyone figured out a way of having a floating ip like feature with docker so that you can have rabbitmq, mysql, or ceph mon's at fixed ip's and be able to migrate them around from physical host to physical host and still have them at fixed

Re: [openstack-dev] [all][tripleo] New Project - Kolla: Deploy and Manage OpenStack using Kubernetes and Docker

2014-09-25 Thread Angus Lees
On Thu, 25 Sep 2014 04:01:38 PM Fox, Kevin M wrote: Doesn't nova with a docker driver and heat autoscaling handle case 2 and 3 for control jobs? Has anyone tried yet? For reference, the cases were: - Something to deploy the code (docker / distro packages / pip install / etc) - Something to

Re: [openstack-dev] [all][tripleo] New Project - Kolla: Deploy and Manage OpenStack using Kubernetes and Docker

2014-09-24 Thread Angus Lees
On Wed, 24 Sep 2014 10:31:19 PM Alan Kavanagh wrote: Steven I have to ask what is the motivation and benefits we get from integrating Kubernetes into Openstack? Would be really useful if you can elaborate and outline some use cases and benefits Openstack and Kubernetes can gain. I've no idea

Re: [openstack-dev] [all] [clients] [keystone] lack of retrying tokens leads to overall OpenStack fragility

2014-09-17 Thread Angus Lees
On Wed, 17 Sep 2014 04:53:28 PM Duncan Thomas wrote: On 16 September 2014 01:28, Nathan Kinder nkin...@redhat.com wrote: The idea would be to leave normal tokens with a smaller validity period (like the current default of an hour), but also allow one-time use tokens to be requested.

Re: [openstack-dev] [all] OpenStack bootstrapping hour - Friday Sept 19th - 3pm EST

2014-09-15 Thread Angus Lees
On Tue, 16 Sep 2014 12:04:53 AM Rochelle.RochelleGrober wrote: +1000 This is *great*. Not only for newbies, but refreshers, learning different approaches, putting faces to the signatures, etc. And Mock best practices is a brilliant starting place for developers. I'd like to vote for a few

Re: [openstack-dev] [all] [clients] [keystone] lack of retrying tokens leads to overall OpenStack fragility

2014-09-12 Thread Angus Lees
On Thu, 11 Sep 2014 03:00:02 PM Duncan Thomas wrote: On 11 September 2014 03:17, Angus Lees g...@inodes.org wrote: (As inspired by eg kerberos) 2. Ensure at some environmental/top layer that the advertised token lifetime exceeds the timeout set on the request, before making the request

Re: [openstack-dev] [all] [clients] [keystone] lack of retrying tokens leads to overall OpenStack fragility

2014-09-12 Thread Angus Lees
On Thu, 11 Sep 2014 03:21:52 PM Steven Hardy wrote: On Wed, Sep 10, 2014 at 08:46:45PM -0400, Jamie Lennox wrote: For service to service communication there are two types. 1) using the user's token like nova-cinder. If this token expires there is really nothing that nova can do except raise

Re: [openstack-dev] [all] i need some help on this bug Bug #1365892

2014-09-12 Thread Angus Lees
On Wed, 10 Sep 2014 01:04:02 PM Mike Bayer wrote: In this case it appears to be a “safety” in case someone uses the ConnectionContext object outside of being a context manager. I’d fix that and require that it be used as a context manager only. Oh look, I have a new pylint hammer that is

Re: [openstack-dev] [neutron][all] switch from mysqldb to another eventlet aware mysql client

2014-09-12 Thread Angus Lees
On Fri, 12 Sep 2014 01:08:04 PM Doug Hellmann wrote: On Sep 12, 2014, at 1:03 PM, Ihar Hrachyshka ihrac...@redhat.com wrote: Signed PGP part On 12/09/14 17:30, Mike Bayer wrote: On Sep 12, 2014, at 10:40 AM, Ihar Hrachyshka ihrac...@redhat.com wrote: Signed PGP part On

Re: [openstack-dev] [all] [clients] [keystone] lack of retrying tokens leads to overall OpenStack fragility

2014-09-10 Thread Angus Lees
On Wed, 10 Sep 2014 10:14:32 AM Sean Dague wrote: Going through the untriaged Nova bugs, and there are a few on a similar pattern: Nova operation in progress takes a while Crosses keystone token expiration time Timeout thrown Operation fails Terrible 500 error sent back to user It

Re: [openstack-dev] Kilo Cycle Goals Exercise

2014-09-10 Thread Angus Lees
So easy/obvious it probably isn't even worth mentioning: Drop support for python2.6 -- - Gus ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [neutron] [nova] non-deterministic gate failures due to unclosed eventlet Timeouts

2014-09-10 Thread Angus Lees
On Mon, 8 Sep 2014 05:25:22 PM Jay Pipes wrote: On 09/07/2014 10:43 AM, Matt Riedemann wrote: On 9/7/2014 8:39 AM, John Schwarz wrote: Hi, Long story short: for future reference, if you initialize an eventlet Timeout, make sure you close it (either with a context manager or simply

Re: [openstack-dev] [neutron][all] switch from mysqldb to another eventlet aware mysql client

2014-08-21 Thread Angus Lees
On Wed, 20 Aug 2014 05:03:51 PM Clark Boylan wrote: On Mon, Aug 18, 2014, at 01:59 AM, Ihar Hrachyshka wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA512 On 17/08/14 02:09, Angus Lees wrote: On 16 Aug 2014 06:09, Doug Hellmann d...@doughellmann.com mailto:d

Re: [openstack-dev] [oslo] Issues with POSIX semaphores and other locks in lockutils

2014-08-20 Thread Angus Lees
On Mon, 18 Aug 2014 10:05:28 PM Pádraig Brady wrote: On 08/18/2014 03:38 PM, Julien Danjou wrote: On Thu, Aug 14 2014, Yuriy Taraday wrote: Hi Yuriy, […] Looking forward to your opinions. This looks like a good summary of the situation. I've added a solution E based on

Re: [openstack-dev] [neutron][all] switch from mysqldb to another eventlet aware mysql client

2014-08-16 Thread Angus Lees
On 16 Aug 2014 06:09, Doug Hellmann d...@doughellmann.com wrote: On Aug 15, 2014, at 9:29 AM, Ihar Hrachyshka ihrac...@redhat.com wrote: Signed PGP part Some updates on the matter: - oslo-spec was approved with narrowed scope which is now 'enabled mysqlconnector as an alternative in

[openstack-dev] [neutron] Which changes need accompanying bugs?

2014-08-13 Thread Angus Lees
I'm doing various small cleanup changes as I explore the neutron codebase. Some of these cleanups are to fix actual bugs discovered in the code. Almost all of them are tiny and obviously correct. A recurring reviewer comment is that the change should have had an accompanying bug report and

Re: [openstack-dev] [neutron] Which changes need accompanying bugs?

2014-08-13 Thread Angus Lees
On Wed, 13 Aug 2014 11:11:51 AM Kevin Benton wrote: Is the pylint static analysis that caught that error prone to false positives? If not, I agree that it would be really nice if that were made part of the tox check so these don't have to be fixed after the fact. At the moment pylint on

Re: [openstack-dev] [neutron] Which changes need accompanying bugs?

2014-08-13 Thread Angus Lees
On Wed, 13 Aug 2014 12:46:03 AM Kevin Benton wrote: I'm not sure what the guideline is, but I would like to point out a good reason to have the bug report even for obvious fixes. When users encounters bugs, they go to launchpad to report them. They don't first scan the commits of the master

Re: [openstack-dev] [gate] The gate: a failure analysis

2014-07-28 Thread Angus Lees
On Mon, 21 Jul 2014 04:39:49 PM David Kranz wrote: On 07/21/2014 04:13 PM, Jay Pipes wrote: On 07/21/2014 02:03 PM, Clint Byrum wrote: Thanks Matthew for the analysis. I think you missed something though. Right now the frustration is that unrelated intermittent bugs stop your

Re: [openstack-dev] [gate] The gate: a failure analysis

2014-07-28 Thread Angus Lees
On Mon, 28 Jul 2014 10:22:07 AM Doug Hellmann wrote: On Jul 28, 2014, at 2:52 AM, Angus Lees g...@inodes.org wrote: On Mon, 21 Jul 2014 04:39:49 PM David Kranz wrote: On 07/21/2014 04:13 PM, Jay Pipes wrote: On 07/21/2014 02:03 PM, Clint Byrum wrote: Thanks Matthew for the analysis

Re: [openstack-dev] [neutron][all] switch from mysqldb to another eventlet aware mysql client

2014-07-20 Thread Angus Lees
Status, as I understand it: * oslo.db changes to support other mysql drivers: https://review.openstack.org/#/c/104425/ (merged) https://review.openstack.org/#/c/106928/ (awaiting oslo.db review) https://review.openstack.org/#/c/107221/ (awaiting oslo.db review) (These are sufficient to allow

Re: [openstack-dev] [neutron][all] switch from mysqldb to another eventlet aware mysql client

2014-07-10 Thread Angus Lees
On 10 July 2014 00:59, Roman Podoliaka rpodoly...@mirantis.com wrote: Not sure what issues you are talking about, but I just replaced mysql with mysql+mysqlconnector in my db connection string in neutron.conf and neutron-db-manage upgrade head worked like a charm for an empty schema. Yep,

[openstack-dev] Where should a test for eventlet and oslo.db interaction go?

2014-07-10 Thread Angus Lees
We have an issue with neutron (and presumably elsewhere), where mysqldb and eventlet may deadlock, until the mysqldb deadlock timer fires. I believe it's responsible for ~all of these failures:

Re: [openstack-dev] [neutron][nova] nova needs a new release of neutronclient for OverQuotaClient exception

2014-06-24 Thread Angus Lees
On Tue, 24 Jun 2014 02:46:33 PM Kyle Mestery wrote: On Mon, Jun 23, 2014 at 11:08 AM, Kyle Mestery mest...@noironetworks.com wrote: On Mon, Jun 23, 2014 at 8:54 AM, Matt Riedemann mrie...@linux.vnet.ibm.com wrote: There are at least two changes [1][2] proposed to Nova that use the new

Re: [openstack-dev] [nova] Distributed locking

2014-06-15 Thread Angus Lees
On Fri, 13 Jun 2014 09:40:30 AM Matthew Booth wrote: On 12/06/14 21:38, Joshua Harlow wrote: So just a few thoughts before going to far down this path, Can we make sure we really really understand the use-case where we think this is needed. I think it's fine that this use-case exists,

Re: [openstack-dev] [nova] Distributed locking

2014-06-12 Thread Angus Lees
On Thu, 12 Jun 2014 05:06:38 PM Julien Danjou wrote: On Thu, Jun 12 2014, Matthew Booth wrote: This looks interesting. It doesn't have hooks for fencing, though. What's the status of tooz? Would you be interested in adding fencing hooks? It's maintained and developer, we have plan to