date:20150515

On Thu, May 14, 2015 at 02:23:25PM -0500, Matt Riedemann wrote:
 The minimum required version of libvirt in the driver is 0.9.11 still [1].
 We've been gating against 1.2.2 in Ubuntu Trusty 14.04 since Juno.
 
 The libvirt distro support matrix is here: [2]
 
 Can we safely assume the people aren't going to be running Libvirt compute
 nodes on RHEL  7.1 or Ubuntu Precise?

I don't really think so - at the very least Fedora 20 and RHEL 7.0 are still
actively supported platforms by their vendors, which both have older libvirt
versions (1.1.3 and 1.1.1 respectively).

I'm not sure whether SUSE team consider any of the 12.x versions to still be
actively supported platforms or not, likewise which 13.x versions are under
active support.


 Regarding RHEL, I think this is a safe bet because in Kilo nova dropped
 python 2.6 support and RHEL  6 doesn't have py26 so you'd be in trouble
 running kilo+ nova on RHEL 6.x anyway.

There are add-on repos for RHEL-6 and RHEL-7 that provide newer python
versions so (py27, various py3x), so it is not unreasonable for people
to consider sticking with RHEL-6 if that is what their existing deployment
is based on. Certainly new deployments though I'd expect to be RHEL-7 based.

 There are some workarounds in the code [3] I'd like to see removed by
 bumping the minimum required version.

Sure, its nice to remove workarounds from a cleanliness POV, but I'm generally
pretty conservative about doing so, because in the majority of case (while it
looks ugly) it is not really a significant burden on maintainers to keep it
around.

This example is really just that. It certainly looks ugly, but we have the
code there now, it is doing the job for people who have that problem and it
isn't really having any measurable impact on our ability to maintain the
libvirt code. Removing this code won't lessen our maintainance burden in
any way, but it will unquestionably impact our users by removing support for
the platform they may be currently deployed on.

The reason why we picked 0.9.11 as the current minimum version was that we
needed to be able to switch over to using libvirt-python from PyPi instead
of relying on the version that shipped with the distro. 0.9.11 is the min
version supported by libvirt-python on PyPi. This had significant benefits
to Nova maintainence, as it our gate jobs deploy libvirt-python from PyPi
in common with all other python packages we depend on. It also unlocked the
ability to run libvirt with python3 bindings.  There was some small amount
of pain for people running Ubuntu 12.04 LTS, but this was mitigated by the
fact that Canonical provided the Cloud Archive repositories for that LTS
version which gave users direct access to new enough libvirt. So in the end
users were not negatively impacted in any serious way - certainly not by
enough to outweigh the benefits Nova maintainence saw.


In this case, I just don't see compelling benefits to Nova libvirt maint
to justify increasing the minimum version to the level you suggest, and
it has a clear negative impact on our users which they will not be able
to easily deal with. They will be left with 3 options all of which are
unsatisfactory

 - Upgrade from RHEL-6 to RHEL-7 - a major undertaking for most organizations
 - Upgrade libvirt on RHEL-6 - they essentially take on the support burden
   for the hypervisor themselves, loosing support from the vendor
 - Re-add the code we remove from Nova - we've given users a maint burden,
   to rid ourselves of code that was posing no real maint burden on ourselves.


As a more general point, I think we are lacking clear guidance on our
policies around hypervisor platform support and thus have difficulty
in deciding when it is reasonable for us to drop support for platforms.

I think it is important to distinguish the hypervisor platform, from
the openstack platform because there are different support implications
there for users. With direct python dependancies for openstack, I we
are generally pretty aggressive at updating to newer versions of packages
from PyPi. There are a number of reasons why this is reasonable to do.
Foremost is that many python modules do a very bad job at maintaining
API compatibility, so we are essentially forced to upgrade and drop old
version support whether we like it or not. Second, the provisioning tools
we use (devstack, packstack, triple-o, and many vendor tools) all handle
deployment of arbitrary newer python modules without any trouble. OpenStack
itself and the vendors all do quite comprehensive testing on the python
modules we use, so we have confidence that the versions we're depending
on are functioning correctly.

The hypervisor platform is very different. While OpenStack does achieve
some level of testing coverage of the hypervisor platform version used
in the gate, this testing is inconsequential compared to the level of
testing that vendors put into their hypervisor platforms.

For example, vendors will test countless different guest

Re: [openstack-dev] [devstack] What's required to accept a new distro rlease?

2015-05-15 Thread Sean Dague

On 05/15/2015 01:13 AM, Tony Breeds wrote:
 On Fri, May 15, 2015 at 01:35:06PM +1000, Ian Wienand wrote:
 On 05/15/2015 01:05 PM, Tony Breeds wrote:
 I'm wondering what are the requirements for accepting something
 like:

 -if [[ ! ${DISTRO} =~ 
 (precise|trusty|7.0|wheezy|sid|testing|jessie|f20|f21|rhel7) ]]; then
 +if [[ ! ${DISTRO} =~ 
 (precise|trusty|vivid|7.0|wheezy|sid|testing|jessie|f20|f21|rhel7) ]]; then

 Having a CI story makes it a no-brainer...  anyone working on getting
 vivid nodes up in nodepool?  In the past we've generally accepted if
 people have a convincing story that it works (i.e. they're using it
 and tempest is working, etc).
 
 Thanks.  Apart from a strange prblem with rabbitmq taking too long to start it
 works fine for me.  I'll do a compleet tempest run and look into what would be
 involved with getting (and maintaining) vivid in nodepool [until wily is
 released]
  
 Honestly I doubt 7.0|wheezy|sid|testing|jessie all work anyway -- they
 don't have any CI (I know of) -- so we're probably overselling it
 anyway.
 
 Could be.  If I find time after the summit I'll try them. 7.0/wheezy is
 oldstable now.  It'd be good to know that 8.0/jesse works.
  
 I figure there must be more to it as utopic was never added.

 I think that was approved, failed a test and was never re-merged [1].
 So I guess it's more nobody cared?
 
 Ahh okay.
 
 Yours Tony.

Right, for non LTS ubuntu, if it seems to work, and is still within the
ubuntu support window for said distro, just submit the patches to enable
them.

We're not going to CI non LTC ubuntu regularly on devstack patches (at
least not pre-merge), however treating them as sort of class B support -
users come with patches and we merge them, is fine, especially as it
means the next LTS switch becomes a lot easier, and the deltas for base
functionality are typically pretty low.

If it was a completely different distro, like arch/gentoo, that becomes
substantially non trivial to put in and maintain the infrastructure for
that. So things like that have been rejected in the past on those grounds.

-Sean

-- 
Sean Dague
http://dague.net

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] OpenStack 2015.1.0 for Debian Sid and Jessie

2015-05-15 Thread David Medberry

Neil, I haven't inspected these packages but historically they are
independent.

On Fri, May 15, 2015 at 2:37 AM, neil.jer...@metaswitch.com wrote:

 Out of interest, have you done this by re-releasing the Ubuntu packaging?
 Or have you taken an independent approach?

 Regards,
 Neil


   Original Message
 From: Thomas Goirand
 Sent: Thursday, 14 May 2015 22:21
 To: OpenStack Operators; Openstack; OpenStack Development Mailing List
 (not for usage questions)
 Reply To: OpenStack Development Mailing List (not for usage questions)
 Subject: [openstack-dev] OpenStack 2015.1.0 for Debian Sid and Jessie

 Hi,

 I am pleased to announce the general availability of OpenStack 2015.1.0
 (aka Kilo) in Debian unstable (aka Sid) and through the official Debian
 backports repository for Debian 8.0 (aka Sid).

 Debian 8.0 Jessie just released
 ===
 As you may know, Debian 8.0 was released on the 25th of April, just a
 few days before OpenStack Kilo (on the 30th of April). Just right after
 Debian Jessie got released, OpenStack Kilo was uploaded to unstable, and
 slowly migrated the usual way to the new Debian Testing, named Stretch.

 As a lot of new packages had to go through the Debian FTP master NEW
 queue for review (they check mainly for the copyright / licensing
 information, but also if the package is conform to the Debian policy).
 I'd like here to publicly thank Paul Tagliamonte from the Debian FTP
 team for his prompt work, which allowed Kilo to reach the Debian
 repositories just a few days after its release (in fact, Kilo was fully
 available in Unstable more than a week ago).

 Debian Jessie Backports
 ===
 Previously, each release of OpenStack, as a backport for Debian Stable,
 was only available through private repositories. This wasn't a
 satisfying solution, and we wanted to address it by uploading to the
 official Debian backports. And the result is now available: all of
 OpenStack Kilo has been uploaded to Debian jessie-backports. If you want
 to use these repositories, just add them to your sources.list (note that
 the Debian installer proposes to add it by default):

 deb http://httpredir.debian.org/debian jessie-backports main

 (of course, you can use any Debian mirror, not just the httpredir)

 All of the usual OpenStack components are currently available in the
 official backports, but there's still some more to come, like for
 example Heat, Murano, Trove or Sahara. For Heat, it's because we're
 still waiting for python-oslo.versionedobjects 0.1.1-2 to migrate to
 Stretch (as a rule: we can't upload to backports unless a package is
 already in Testing). For the last 3, I'm not sure if they will be
 backported to Jessie. Please provide your feedback and tell the Debian
 packaging team if they are important for you in the official
 jessie-backports repository, or if Sid is enough. Also, at the time of
 writing of this message, Horizon and Designate are still in the
 backports FTP master NEW queue (but it should be approved very soon).

 Also, I have just uploaded a first version of Barbican (still in the NEW
 queue waiting for approval...), and there's a package for Manila that is
 currently on the work by a new contributor.

 Note on Neutron off-tree drivers
 
 The neutron-lbaas, neutron-fwaas and neutron-vpnaas packages have been
 uploaded and are part of Sid. If you need it through jessie-backports,
 please just let me know.

 All vendor-specific drivers have been separated from Neutron, and are
 now available as separate packages. I wrote packages for them all, but
 the issue is that most of them wouldn't even build due to failed unit
 tests. For most of them, it used to work in the Kilo beta 3 of Neutron
 (it's the case for all but 2 of them who were broken at the time), but
 they appeared broken with the Kilo final release, as they didn't update
 after the Kilo release.

 I have repaired some of them, but working on these packages has shown to
 be a very frustrating work, as they receive very few updates from
 upstream. I do not plan to work much on them unless one of the below
 condition:
 - My employer needs them
 - things are moving forward upstream, and that these unit tests are
 repaired in the stackforge repository.

 If you are a network hardware vendor and read this, please push for more
 maintenance, as it's in a really bad state ATM. You are welcome to get
 in touch with me, and I'll be happy to help you to help.

 Bug report
 ==
 If you see any issue in the packages, please do report them to the
 Debian bug tracker. Instructions are available here:
 https://www.debian.org/Bugs/Reporting

 Happy installation,

 Thomas Goirand (zigo)

 __
 OpenStack Development Mailing List (not for usage questions)
 Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe

Re: [openstack-dev] OpenStack 2015.1.0 for Debian Sid and Jessie

2015-05-15 Thread David Medberry

and a quick checks shows that there are number of differences just in the
nova source package, so I believe they remain independent.

On Fri, May 15, 2015 at 5:01 AM, David Medberry openst...@medberry.net
wrote:

 Neil, I haven't inspected these packages but historically they are
 independent.

 On Fri, May 15, 2015 at 2:37 AM, neil.jer...@metaswitch.com wrote:

 Out of interest, have you done this by re-releasing the Ubuntu packaging?
 Or have you taken an independent approach?

 Regards,
 Neil


   Original Message
 From: Thomas Goirand
 Sent: Thursday, 14 May 2015 22:21
 To: OpenStack Operators; Openstack; OpenStack Development Mailing List
 (not for usage questions)
 Reply To: OpenStack Development Mailing List (not for usage questions)
 Subject: [openstack-dev] OpenStack 2015.1.0 for Debian Sid and Jessie

 Hi,

 I am pleased to announce the general availability of OpenStack 2015.1.0
 (aka Kilo) in Debian unstable (aka Sid) and through the official Debian
 backports repository for Debian 8.0 (aka Sid).

 Debian 8.0 Jessie just released
 ===
 As you may know, Debian 8.0 was released on the 25th of April, just a
 few days before OpenStack Kilo (on the 30th of April). Just right after
 Debian Jessie got released, OpenStack Kilo was uploaded to unstable, and
 slowly migrated the usual way to the new Debian Testing, named Stretch.

 As a lot of new packages had to go through the Debian FTP master NEW
 queue for review (they check mainly for the copyright / licensing
 information, but also if the package is conform to the Debian policy).
 I'd like here to publicly thank Paul Tagliamonte from the Debian FTP
 team for his prompt work, which allowed Kilo to reach the Debian
 repositories just a few days after its release (in fact, Kilo was fully
 available in Unstable more than a week ago).

 Debian Jessie Backports
 ===
 Previously, each release of OpenStack, as a backport for Debian Stable,
 was only available through private repositories. This wasn't a
 satisfying solution, and we wanted to address it by uploading to the
 official Debian backports. And the result is now available: all of
 OpenStack Kilo has been uploaded to Debian jessie-backports. If you want
 to use these repositories, just add them to your sources.list (note that
 the Debian installer proposes to add it by default):

 deb http://httpredir.debian.org/debian jessie-backports main

 (of course, you can use any Debian mirror, not just the httpredir)

 All of the usual OpenStack components are currently available in the
 official backports, but there's still some more to come, like for
 example Heat, Murano, Trove or Sahara. For Heat, it's because we're
 still waiting for python-oslo.versionedobjects 0.1.1-2 to migrate to
 Stretch (as a rule: we can't upload to backports unless a package is
 already in Testing). For the last 3, I'm not sure if they will be
 backported to Jessie. Please provide your feedback and tell the Debian
 packaging team if they are important for you in the official
 jessie-backports repository, or if Sid is enough. Also, at the time of
 writing of this message, Horizon and Designate are still in the
 backports FTP master NEW queue (but it should be approved very soon).

 Also, I have just uploaded a first version of Barbican (still in the NEW
 queue waiting for approval...), and there's a package for Manila that is
 currently on the work by a new contributor.

 Note on Neutron off-tree drivers
 
 The neutron-lbaas, neutron-fwaas and neutron-vpnaas packages have been
 uploaded and are part of Sid. If you need it through jessie-backports,
 please just let me know.

 All vendor-specific drivers have been separated from Neutron, and are
 now available as separate packages. I wrote packages for them all, but
 the issue is that most of them wouldn't even build due to failed unit
 tests. For most of them, it used to work in the Kilo beta 3 of Neutron
 (it's the case for all but 2 of them who were broken at the time), but
 they appeared broken with the Kilo final release, as they didn't update
 after the Kilo release.

 I have repaired some of them, but working on these packages has shown to
 be a very frustrating work, as they receive very few updates from
 upstream. I do not plan to work much on them unless one of the below
 condition:
 - My employer needs them
 - things are moving forward upstream, and that these unit tests are
 repaired in the stackforge repository.

 If you are a network hardware vendor and read this, please push for more
 maintenance, as it's in a really bad state ATM. You are welcome to get
 in touch with me, and I'll be happy to help you to help.

 Bug report
 ==
 If you see any issue in the packages, please do report them to the
 Debian bug tracker. Instructions are available here:
 https://www.debian.org/Bugs/Reporting

 Happy installation,

 Thomas Goirand (zigo)

Re: [openstack-dev] [Openstack-operators] [nova] Can we bump MIN_LIBVIRT_VERSION to 1.2.2 in Liberty?

On Fri, May 15, 2015 at 11:51:22AM +0100, Daniel P. Berrange wrote:
 On Thu, May 14, 2015 at 02:23:25PM -0500, Matt Riedemann wrote:
  The minimum required version of libvirt in the driver is 0.9.11 still [1].
  We've been gating against 1.2.2 in Ubuntu Trusty 14.04 since Juno.
  
  The libvirt distro support matrix is here: [2]
  
  Can we safely assume the people aren't going to be running Libvirt compute
  nodes on RHEL  7.1 or Ubuntu Precise?
 
 I don't really think so - at the very least Fedora 20 and RHEL 7.0 are still
 actively supported platforms by their vendors, which both have older libvirt
 versions (1.1.3 and 1.1.1 respectively).
 
 I'm not sure whether SUSE team consider any of the 12.x versions to still be
 actively supported platforms or not, likewise which 13.x versions are under
 active support.
 
 
  Regarding RHEL, I think this is a safe bet because in Kilo nova dropped
  python 2.6 support and RHEL  6 doesn't have py26 so you'd be in trouble
  running kilo+ nova on RHEL 6.x anyway.
 
 There are add-on repos for RHEL-6 and RHEL-7 that provide newer python
 versions so (py27, various py3x), so it is not unreasonable for people
 to consider sticking with RHEL-6 if that is what their existing deployment
 is based on. Certainly new deployments though I'd expect to be RHEL-7 based.
 
  There are some workarounds in the code [3] I'd like to see removed by
  bumping the minimum required version.
 
 Sure, its nice to remove workarounds from a cleanliness POV, but I'm generally
 pretty conservative about doing so, because in the majority of case (while it
 looks ugly) it is not really a significant burden on maintainers to keep it
 around.
 
 This example is really just that. It certainly looks ugly, but we have the
 code there now, it is doing the job for people who have that problem and it
 isn't really having any measurable impact on our ability to maintain the
 libvirt code. Removing this code won't lessen our maintainance burden in
 any way, but it will unquestionably impact our users by removing support for
 the platform they may be currently deployed on.
 
 The reason why we picked 0.9.11 as the current minimum version was that we
 needed to be able to switch over to using libvirt-python from PyPi instead
 of relying on the version that shipped with the distro. 0.9.11 is the min
 version supported by libvirt-python on PyPi. This had significant benefits
 to Nova maintainence, as it our gate jobs deploy libvirt-python from PyPi
 in common with all other python packages we depend on. It also unlocked the
 ability to run libvirt with python3 bindings.  There was some small amount
 of pain for people running Ubuntu 12.04 LTS, but this was mitigated by the
 fact that Canonical provided the Cloud Archive repositories for that LTS
 version which gave users direct access to new enough libvirt. So in the end
 users were not negatively impacted in any serious way - certainly not by
 enough to outweigh the benefits Nova maintainence saw.
 
 
 In this case, I just don't see compelling benefits to Nova libvirt maint
 to justify increasing the minimum version to the level you suggest, and
 it has a clear negative impact on our users which they will not be able
 to easily deal with. They will be left with 3 options all of which are
 unsatisfactory
 
  - Upgrade from RHEL-6 to RHEL-7 - a major undertaking for most organizations
  - Upgrade libvirt on RHEL-6 - they essentially take on the support burden
for the hypervisor themselves, loosing support from the vendor
  - Re-add the code we remove from Nova - we've given users a maint burden,
to rid ourselves of code that was posing no real maint burden on ourselves.
 
 
 As a more general point, I think we are lacking clear guidance on our
 policies around hypervisor platform support and thus have difficulty
 in deciding when it is reasonable for us to drop support for platforms.
 
 I think it is important to distinguish the hypervisor platform, from
 the openstack platform because there are different support implications
 there for users. With direct python dependancies for openstack, I we
 are generally pretty aggressive at updating to newer versions of packages
 from PyPi. There are a number of reasons why this is reasonable to do.
 Foremost is that many python modules do a very bad job at maintaining
 API compatibility, so we are essentially forced to upgrade and drop old
 version support whether we like it or not. Second, the provisioning tools
 we use (devstack, packstack, triple-o, and many vendor tools) all handle
 deployment of arbitrary newer python modules without any trouble. OpenStack
 itself and the vendors all do quite comprehensive testing on the python
 modules we use, so we have confidence that the versions we're depending
 on are functioning correctly.
 
 The hypervisor platform is very different. While OpenStack does achieve
 some level of testing coverage of the hypervisor platform version used
 in the gate, this testing is

[openstack-dev] [nova] Unvalidated user input passed to functions

2015-05-15 Thread Matthew Booth

I was looking at the migrations api, and I noticed that the api passes
the request query unchecked to get_migrations, where it ultimately ends
up in a db query. I was curious and spent a couple of hours checking
this morning. There are a few instances of this.

I didn't find any security bugs, however I feel that this extremely bad
practise, and is likely to result in a security bug eventually. For
example, note that os-assisted-volume-snapshots:delete does not validate
delete_info before passing it to volume_snapshot_delete. I looked at
this quite carefully, and I think we are only protected from a host
compromise because:

1. The api requires admin context
2. libvirt's security policy

I could be wrong on that, though, so perhaps somebody else could check?

Passing unvalidated input to a function isn't necessarily bad, for
example if it is only used for filtering, but it should be clearly
marked as such so it isn't used in an unsafe manner. This marking should
follow the data as far as it goes through any number of function calls.
libvirt's _volume_snapshot_delete function is a long way from the
originating api call, and it is not at all obvious that the commit_base
and commit_top arguments to virt_dom.blockCommit() are unvalidated.

Does python have anything like perl's taint mode? If so, it might be
worth investigating its use.

Matt
-- 
Matthew Booth
Red Hat Engineering, Virtualisation Team

Phone: +442070094448 (UK)
GPG ID:  D33C3490
GPG FPR: 3733 612D 2D05 5458 8A8A 1600 3441 EA19 D33C 3490

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Openstack-operators] [nova] Can we bump MIN_LIBVIRT_VERSION to 1.2.2 in Liberty?

On Fri, May 15, 2015 at 11:51:22AM +0100, Daniel P. Berrange wrote:
On Thu, May 14, 2015 at 02:23:25PM -0500, Matt Riedemann wrote:
The minimum required version of libvirt in the driver is 0.9.11 still [1].
We've been gating against 1.2.2 in Ubuntu Trusty 14.04 since Juno.

The libvirt distro support matrix is here: [2]

Can we safely assume the people aren't going to be running Libvirt compute
nodes on RHEL 7.1 or Ubuntu Precise?

I don't really think so - at the very least Fedora 20 and RHEL 7.0 are still
actively supported platforms by their vendors, which both have older libvirt
versions (1.1.3 and 1.1.1 respectively).

I'm not sure whether SUSE team consider any of the 12.x versions to still be
actively supported platforms or not, likewise which 13.x versions are under
active support.

Regarding RHEL, I think this is a safe bet because in Kilo nova dropped
python 2.6 support and RHEL 6 doesn't have py26 so you'd be in trouble
running kilo+ nova on RHEL 6.x anyway.

There are add-on repos for RHEL-6 and RHEL-7 that provide newer python
versions so (py27, various py3x), so it is not unreasonable for people
to consider sticking with RHEL-6 if that is what their existing deployment
is based on. Certainly new deployments though I'd expect to be RHEL-7 based.

There are some workarounds in the code [3] I'd like to see removed by
bumping the minimum required version.

Sure, its nice to remove workarounds from a cleanliness POV, but I'm generally
pretty conservative about doing so, because in the majority of case (while it
looks ugly) it is not really a significant burden on maintainers to keep it
around.

This example is really just that. It certainly looks ugly, but we have the
code there now, it is doing the job for people who have that problem and it
isn't really having any measurable impact on our ability to maintain the
libvirt code. Removing this code won't lessen our maintainance burden in
any way, but it will unquestionably impact our users by removing support for
the platform they may be currently deployed on.

BTW, the code you quote here:

http://git.openstack.org/cgit/openstack/nova/tree/nova/virt/libvirt/host.py?id=2015.1.0#n754

is not actually working around a bug the libvirt hypervisor. It is in fact
a bug in the libvirt-python API binding. As such we don't actually need to
increase the min required libvirt to be able to remove that check. In fact
increasing the min required libvirt is the wrong thing todo, because it is
possible for someone to have the min required libvirt, but by accessing it
via an older libvirt-python which still has the bug.

So what's really needed is a dep on libvirt-python = 1.2.0, not libvirt.

We don't express min required versions for libvirt-python in the
requirements.txt file though, since it is an optional package and we
don't have any mechanism for recording min versions for those AFAIK.

Regards,
Daniel
--
|: http://berrange.com -o-http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org -o- http://virt-manager.org :|
|: http://autobuild.org -o- http://search.cpan.org/~danberr/ :|
|: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc :|

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [openstackclient] About my blueprint every-time-record-log-in-file

2015-05-15 Thread Fujita, Daisuke

Dear openstackclient team members,

I'm Daisuke Fujita of Fujitsu.

I would like to implement following my buleprint.
 
https://blueprints.launchpad.net/python-openstackclient/+spec/every-time-record-log-in-file

Python-openstackclient's Blueprints process is following:
 https://wiki.openstack.org/wiki/Blueprints#Blueprints_only_lifecycle

So, Could you tell me when and how blueprint is approved?
My understanding is that the blueprint is determined to approve or not
in Vancouver. Please let me know about your opinion.

Best Regards,
Daisuke Fujita



__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [nova] Unvalidated user input passed to functions

On 15 May 2015 at 14:13, Daniel P. Berrange berra...@redhat.com wrote:
 On Fri, May 15, 2015 at 12:41:20PM +0100, Matthew Booth wrote:
 I was looking at the migrations api, and I noticed that the api passes
 the request query unchecked to get_migrations, where it ultimately ends
 up in a db query. I was curious and spent a couple of hours checking
 this morning. There are a few instances of this.

In general, I want to encourage the use of private security bugs to
start discussions on these kinds of topics:
https://security.openstack.org/vmt-process.html#reception

 I didn't find any security bugs, however I feel that this extremely bad
 practise, and is likely to result in a security bug eventually. For
 example, note that os-assisted-volume-snapshots:delete does not validate
 delete_info before passing it to volume_snapshot_delete. I looked at
 this quite carefully, and I think we are only protected from a host
 compromise because:

 1. The api requires admin context
 2. libvirt's security policy

 I could be wrong on that, though, so perhaps somebody else could check?

 Item 1 is pretty much the protection here. In general this is a problem
 with the design of os-assisted-volume-snapshots:delete API - the very
 fact that it is intended to allow arbitrary file paths to be specified
 by the user makes it effectively impossible to validate - any path has
 to be considered valid :-( This means it should never be allowed for
 anyone except trusted cloud admin.

 The majority of our APIs though are better designed and do not allow the
 API user to supply file paths and similarly sensitive parameters that
 refer to host resources. Usually the user only provides unique identifiers
 (UUIDs) and high level requirements (ie MAC addresses, disk sizes) and
 not file paths or similar.

The v2.1 API introduces the concept of strong validation for all API calls:
http://specs.openstack.org/openstack/nova-specs/specs/kilo/implemented/v2-on-v3-api.html#rest-api-impact

Basically, I except v2.1 to white list all input, eventually.

But seems like only create has had the proper validation added at this point:
https://github.com/openstack/nova/blob/master/nova/api/openstack/compute/plugins/v3/assisted_volume_snapshots.py#L46
https://github.com/openstack/nova/blob/master/nova/api/openstack/compute/schemas/v3/assisted_volume_snapshots.py#L15

Adding the extra validation is most likely to involve a new
micro-version (and thus needs a spec), but thats still TBD.

 Passing unvalidated input to a function isn't necessarily bad, for
 example if it is only used for filtering, but it should be clearly
 marked as such so it isn't used in an unsafe manner. This marking should
 follow the data as far as it goes through any number of function calls.
 libvirt's _volume_snapshot_delete function is a long way from the
 originating api call, and it is not at all obvious that the commit_base
 and commit_top arguments to virt_dom.blockCommit() are unvalidated.

 I think the most important thing is really not to design more APIs like
 os-assisted-volume-snapshots which are inherantly dangerous due to the
 parameters they are design to allow :-( For those few we do have, we
 should definitely vet it as carefully as possible.

Is this not the API thats meant to be only be called by Cinder?
https://blueprints.launchpad.net/nova/+spec/qemu-assisted-snapshots
http://specs.openstack.org/openstack/nova-specs/specs/juno/implemented/libvirt-volume-snap-network-disk.html

If so, maybe the default policy should change to reflect that?
Restrict it to cinder, and not admins? This being more important
in v2.1 where you can't disable any extensions (eventually).

Is there a way we can evolve that to API be a safer interface? Is
anyone wiling to implement that?

Thanks,
John

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] OpenStack 2015.1.0 for Debian Sid and Jessie

2015-05-15 Thread Ihar Hrachyshka

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA256

Are there any attempts to avoid duplication of efforts? I would expect
Ubuntu to reuse and extend what is in their upstream distro - Debian.

Ihar

On 05/15/2015 01:11 PM, David Medberry wrote:
 and a quick checks shows that there are number of differences just
 in the nova source package, so I believe they remain independent.
 
 On Fri, May 15, 2015 at 5:01 AM, David Medberry
 openst...@medberry.net mailto:openst...@medberry.net wrote:
 
 Neil, I haven't inspected these packages but historically they are 
 independent.
 
 On Fri, May 15, 2015 at 2:37 AM, neil.jer...@metaswitch.com 
 mailto:neil.jer...@metaswitch.com wrote:
 
 Out of interest, have you done this by re-releasing the Ubuntu 
 packaging? Or have you taken an independent approach?
 
 Regards, Neil
 
 
 Original Message From: Thomas Goirand Sent: Thursday, 14 May 2015
 22:21 To: OpenStack Operators; Openstack; OpenStack Development 
 Mailing List (not for usage questions) Reply To: OpenStack
 Development Mailing List (not for usage questions) Subject:
 [openstack-dev] OpenStack 2015.1.0 for Debian Sid and Jessie
 
 Hi,
 
 I am pleased to announce the general availability of OpenStack 
 2015.1.0 (aka Kilo) in Debian unstable (aka Sid) and through the
 official Debian backports repository for Debian 8.0 (aka Sid).
 
 Debian 8.0 Jessie just released === As
 you may know, Debian 8.0 was released on the 25th of April, just a 
 few days before OpenStack Kilo (on the 30th of April). Just right
 after Debian Jessie got released, OpenStack Kilo was uploaded to 
 unstable, and slowly migrated the usual way to the new Debian
 Testing, named Stretch.
 
 As a lot of new packages had to go through the Debian FTP master
 NEW queue for review (they check mainly for the copyright /
 licensing information, but also if the package is conform to the
 Debian policy). I'd like here to publicly thank Paul Tagliamonte
 from the Debian FTP team for his prompt work, which allowed Kilo to
 reach the Debian repositories just a few days after its release (in
 fact, Kilo was fully available in Unstable more than a week ago).
 
 Debian Jessie Backports === Previously, each
 release of OpenStack, as a backport for Debian Stable, was only
 available through private repositories. This wasn't a satisfying
 solution, and we wanted to address it by uploading to the official
 Debian backports. And the result is now available: all of OpenStack
 Kilo has been uploaded to Debian jessie-backports. If you want to
 use these repositories, just add them to your sources.list (note
 that the Debian installer proposes to add it by default):
 
 deb http://httpredir.debian.org/debian jessie-backports main
 
 (of course, you can use any Debian mirror, not just the httpredir)
 
 All of the usual OpenStack components are currently available in
 the official backports, but there's still some more to come, like
 for example Heat, Murano, Trove or Sahara. For Heat, it's because
 we're still waiting for python-oslo.versionedobjects 0.1.1-2 to
 migrate to Stretch (as a rule: we can't upload to backports unless
 a package is already in Testing). For the last 3, I'm not sure if
 they will be backported to Jessie. Please provide your feedback and
 tell the Debian packaging team if they are important for you in the
 official jessie-backports repository, or if Sid is enough. Also, at
 the time of writing of this message, Horizon and Designate are
 still in the backports FTP master NEW queue (but it should be
 approved very soon).
 
 Also, I have just uploaded a first version of Barbican (still in 
 the NEW queue waiting for approval...), and there's a package for
 Manila that is currently on the work by a new contributor.
 
 Note on Neutron off-tree drivers  
 The neutron-lbaas, neutron-fwaas and neutron-vpnaas packages have
 been uploaded and are part of Sid. If you need it through 
 jessie-backports, please just let me know.
 
 All vendor-specific drivers have been separated from Neutron, and
 are now available as separate packages. I wrote packages for them 
 all, but the issue is that most of them wouldn't even build due to
 failed unit tests. For most of them, it used to work in the Kilo
 beta 3 of Neutron (it's the case for all but 2 of them who were
 broken at the time), but they appeared broken with the Kilo final
 release, as they didn't update after the Kilo release.
 
 I have repaired some of them, but working on these packages has 
 shown to be a very frustrating work, as they receive very few
 updates from upstream. I do not plan to work much on them unless
 one of the below condition: - My employer needs them - things are
 moving forward upstream, and that these unit tests are repaired in
 the stackforge repository.
 
 If you are a network hardware vendor and read this, please push for
 more maintenance, as it's in a really bad state ATM. You are
 welcome to get in

Re: [openstack-dev] [nova][heat] sqlalchemy-migrate tool to alembic

2015-05-15 Thread Doug Hellmann

Excerpts from Mike Bayer's message of 2015-05-14 21:54:27 -0400:
 
 On 5/14/15 7:12 PM, Angus Salkeld wrote:
 
 
  On Fri, May 15, 2015 at 4:46 AM, Mike Bayer mba...@redhat.com 
  mailto:mba...@redhat.com wrote:
 
 
 
  On 5/14/15 11:58 AM, Doug Hellmann wrote:
 
 
  At one point we were exploring having both sqlalchemy-migrate and
  alembic run, one after the other, so that we only need to
  create new
  migrations with alembic and do not need to change any of the
  existing
  migrations. Was that idea dropped?
 
 
  to my knowledge the idea wasn't dropped.   If a project wants to
  implement that using the oslo.db system, that is fine, however
  from my POV I'd prefer to just port the SQLA-migrate files over
  and drop the migrate dependency altogether.  Whether or not a
  project does the run both step as an interim step doesn't affect
  that effort very much.
 
 
  Hi Mike
 
  Just a quick question: How would the alembic scripts know where to 
  start the migration from if the current installation had been up until 
  that
  point been using migrate (I believe both alembic and migrate write to 
  a small table what the current version is, would you look for that?)?
 Thinking about that issue, the most controllable and clean-break way to 
 do it would be to add support to Alembic itself that augments its own 
 handling of the alembic_version table to transfer data from an 
 existing sqlalchemy_migrate table.  I can even see using traditional 
 alembic hex-style version numbers in migration files which then also can 
 refer to their previous numerically-based migrate file.
 
 It's not unreasonable that Alembic would support some standard upgrade 
 path from Migrate, the only other migration tool SQLAlchemy has ever 
 had, so I'd just add that as a feature.

This seems more complicated than needed. If we just stop writing the
sqlalchemy-migrate scripts and don't change them, then for 1 cycle we
have to run both sets of migrations and after that we can just run
alembic.

Doug

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [horizon][keystone][heat] Are AVAILABLE_REGIONS and multi-region service catalog mutually exclusive?

2015-05-15 Thread Douglas Fish

Anne Gentle annegen...@justwriteclick.com wrote on 05/14/2015 09:47:25
AM:

From: Anne Gentle annegen...@justwriteclick.com
To: OpenStack Development Mailing List (not for usage questions)
openstack-dev@lists.openstack.org
Date: 05/14/2015 10:08 AM
Subject: Re: [openstack-dev] [horizon][keystone][heat] Are
AVAILABLE_REGIONS and multi-region service catalog mutually exclusive?

On Thu, May 14, 2015 at 9:39 AM, Geoff Arnold ge...@geoffarnold.com
wrote:
+1

There seems to be a significant disconnect between Heat, Horizon and
Keystone on the subject of multi-region configurations, and the
documentation isn’t helpful. At the very least, it would be useful
if discussions at the summit could result in a decent Wiki page on
the subject.

We have a cross-project session and spec started about the service
catalog:

https://review.openstack.org/#/c/181393/

http://sched.co/3BL3

I hope more than a wiki page comes of it. :)
Anne

Geoff

On May 13, 2015, at 9:49 PM, Morgan Fainberg morgan.fainb...@gmail.com
wrote:

On May 13, 2015, at 21:34, David Lyle dkly...@gmail.com wrote:

On Wed, May 13, 2015 at 3:24 PM, Mathieu Gagné mga...@iweb.com wrote:
When using AVAILABLE_REGIONS, you get a dropdown at login time to choose
your region which is in fact your keystone endpoint.

Once logged in, you get a new dropdown at the top right to switch
between the keystone endpoints. This means you can configure an
Horizon installation to login to multiple independent OpenStack
installations.

So I don't fully understand what enhancing the multi-region support in
Keystone would mean. Would you be able to configure Horizon to login to
multiple independent OpenStack installations?

Mathieu

On 2015-05-13 5:06 PM, Geoff Arnold wrote:
Further digging suggests that we might consider deprecating
AVAILABLE_REGIONS in Horizon and enhancing the multi-region support in
Keystone. It wouldn’t take a lot; the main points:

* Implement the Regions API discussed back in the Havana time period
- https://etherpad.openstack.org/p/havana-availability-zone-
and-region-management -
but with full CRUD
* Enhance the Endpoints API to allow filtering by region

Supporting two different multi region models is problematic if we’re
serious about things like multi-region Heat.

Thoughts?

Geoff

On May 13, 2015, at 12:01 PM, Geoff Arnold ge...@geoffarnold.com
mailto:ge...@geoffarnold.com wrote:

I’m looking at implementing dynamically-configured multi-region
support for service federation, and the prior art on multi-region
support in Horizon is pretty sketchy. This thread:

http://lists.openstack.org/pipermail/openstack/2014-January/004372.html
is the only real discussion I’ve found, and it’s pretty inconclusive.

More precisely, if I configure a single Horizon with AVAILABLE_REGIONS
pointing at two different Keystones with region names “X” and “Y, and
each of those Keystones returns a service catalog with multiple
regions (“A” and “B” for one, “P”, “Q”, and “R” for the other), what’s
Horizon going to do? Or rather, what’s it expected to do?

Yes, I’m being lazy: I could actually configure this to see what
happens, but hopefully it was considered during the design.

Geoff

PS I’ve added Heat to the subject, because from a quick read of
https://wiki.openstack.org/wiki/Heat/Blueprints/
Multi_Region_Support_for_Heat
it looks as if Heat won’t support the AVAILABLE_REGIONS model. That
seems like an unfortunate disconnect.

Horizon only supports authenticating to one keystone endpoint at a
time, specifically to one of the entries in AVAILABLE_REGIONS as
defined in settings.py. Once you have an authenticated session in
Horizon, the region selection support is merely for filtering
between regions registered with the keystone endpoint you
authenticated to, where the list of regions is determined by parsing
the service catalog returned to you with your token.

What's really unclear to me is what you are intending to ask.

AVAILABLE_REGIONS is merely a list of keystone endpoints. They can
be backed by a different IdP per endpoint and thus not share a token
store. Or, they can just be keystone endpoints that are
geographically different but backed by the same IdP, which may or
may not share a token store. The funny thing is, for Horizon, it
doesn't matter. They are all supported. But as one keystone
endpoint may not know about another, unless nested, this has to be
done with settings as it's not typically discoverable.

If you are asking about token sharing between keystones which the
thread you linked seems to indicate. Then yes, you can have a synced
token store. But that is an exercise left to the operator.

I'd like to quickly go on record and say that a token store sync
like this is not recommended. It is possible to work around this in
Kilo with some limited data sync (resource, assignment) and the use
of Fernet tokens.

Re: [openstack-dev] [QA] Question about Heat Tempest tests

2015-05-15 Thread Yaroslav Lobankov

Forgot to provide the link to the test results :)
http://logs.openstack.org/71/182971/1/gate/gate-tempest-dsvm-heat/241e473/logs/testr_results.html.gz

Regards,
Yaroslav Lobankov.

On Fri, May 15, 2015 at 4:30 PM, Yaroslav Lobankov yloban...@mirantis.com
wrote:

 Angus,

 If you take a look at the tempest check job check-tempest-dsvm-heat for
 the patch https://review.openstack.org/#/c/182971/, you will see that all
 tests are skipped there. I have not found any job anywhere that runs
 these tests.

 Regards,
 Yaroslav Lobankov.

 On Fri, May 15, 2015 at 2:16 AM, Angus Salkeld asalk...@mirantis.com
 wrote:

 On Fri, May 15, 2015 at 1:53 AM, Yaroslav Lobankov 
 yloban...@mirantis.com wrote:

 Hello everyone,

 I have a question about Heat Tempest tests. Is there any dsvm job that
 runs these tests? At first glance no dsvm job runs them.


 We are using in tree functional tests now:
 https://github.com/openstack/heat/tree/master/heat_integrationtests

 And heat has a tempest check job too (for the API tests):
 check-tempest-dsvm-heat
 for example: https://review.openstack.org/#/c/182971/

 -Angus

 Thank you!

 Regards,
 Yaroslav Lobankov.


 __
 OpenStack Development Mailing List (not for usage questions)
 Unsubscribe:
 openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



 __
 OpenStack Development Mailing List (not for usage questions)
 Unsubscribe:
 openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [ceilometer] New Feature Request : Docker files for Power platform (SLES, RHEL, Ubuntu)

2015-05-15 Thread priya duggirala

Hi,

I have written dockerfiles for building ceilometer-client from source. I
have built and tested the source code available on git successfully through
the dockerfile for PPC64 Little Endian architecture. The containers
successfully run on the following platforms:
Ubuntu 14.10

Ubuntu 14.04
SUSE Linux 12.0
RHEL 7.1



As many projects have already begun shipping Dockerfiles along with the
source code, I would like to merge these into the source. Kindly suggest if
there are any preferences such as naming conventions of docker files and
their place in the source tree etc.




Regards,

Priya Duggirala.
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [nova] Unvalidated user input passed to functions

On Fri, May 15, 2015 at 12:41:20PM +0100, Matthew Booth wrote:
 I was looking at the migrations api, and I noticed that the api passes
 the request query unchecked to get_migrations, where it ultimately ends
 up in a db query. I was curious and spent a couple of hours checking
 this morning. There are a few instances of this.
 
 I didn't find any security bugs, however I feel that this extremely bad
 practise, and is likely to result in a security bug eventually. For
 example, note that os-assisted-volume-snapshots:delete does not validate
 delete_info before passing it to volume_snapshot_delete. I looked at
 this quite carefully, and I think we are only protected from a host
 compromise because:
 
 1. The api requires admin context
 2. libvirt's security policy
 
 I could be wrong on that, though, so perhaps somebody else could check?

Item 1 is pretty much the protection here. In general this is a problem
with the design of os-assisted-volume-snapshots:delete API - the very
fact that it is intended to allow arbitrary file paths to be specified
by the user makes it effectively impossible to validate - any path has
to be considered valid :-( This means it should never be allowed for
anyone except trusted cloud admin.

The majority of our APIs though are better designed and do not allow the
API user to supply file paths and similarly sensitive parameters that
refer to host resources. Usually the user only provides unique identifiers
(UUIDs) and high level requirements (ie MAC addresses, disk sizes) and
not file paths or similar.

 Passing unvalidated input to a function isn't necessarily bad, for
 example if it is only used for filtering, but it should be clearly
 marked as such so it isn't used in an unsafe manner. This marking should
 follow the data as far as it goes through any number of function calls.
 libvirt's _volume_snapshot_delete function is a long way from the
 originating api call, and it is not at all obvious that the commit_base
 and commit_top arguments to virt_dom.blockCommit() are unvalidated.

I think the most important thing is really not to design more APIs like
os-assisted-volume-snapshots which are inherantly dangerous due to the
parameters they are design to allow :-( For those few we do have, we
should definitely vet it as carefully as possible.

 Does python have anything like perl's taint mode? If so, it might be
 worth investigating its use.

I don't believe so.

Regards,
Daniel
-- 
|: http://berrange.com  -o-http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org  -o- http://virt-manager.org :|
|: http://autobuild.org   -o- http://search.cpan.org/~danberr/ :|
|: http://entangle-photo.org   -o-   http://live.gnome.org/gtk-vnc :|

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Openstack-operators] [nova] Are we happy with libvirt-python = 1.2.0 ?

On Fri, May 15, 2015 at 02:45:06PM +0100, John Garbutt wrote:
On 15 May 2015 at 13:28, Daniel P. Berrange berra...@redhat.com wrote:
On Fri, May 15, 2015 at 11:51:22AM +0100, Daniel P. Berrange wrote:
On Thu, May 14, 2015 at 02:23:25PM -0500, Matt Riedemann wrote:
There are some workarounds in the code [3] I'd like to see removed by
bumping the minimum required version.

Sure, its nice to remove workarounds from a cleanliness POV, but I'm
generally
pretty conservative about doing so, because in the majority of case (while
it
looks ugly) it is not really a significant burden on maintainers to keep it
around.

This example is really just that. It certainly looks ugly, but we have the
code there now, it is doing the job for people who have that problem and it
isn't really having any measurable impact on our ability to maintain the
libvirt code. Removing this code won't lessen our maintainance burden in
any way, but it will unquestionably impact our users by removing support
for
the platform they may be currently deployed on.

BTW, the code you quote here:

http://git.openstack.org/cgit/openstack/nova/tree/nova/virt/libvirt/host.py?id=2015.1.0#n754

So what's really needed is a dep on libvirt-python = 1.2.0, not libvirt.

Does this mean we can drop the above [3] code?
https://github.com/openstack/requirements/blob/master/global-requirements.txt#L56

Hmm, I didn't know it was listed in global-requirements.txt - I only
checked the requirements.txt and test-requirements.txt in Nova itself
which does not list libvirt-python.

Previously test-requirements.txt did have it, but we dropped it, since
the unit tests now exclusively use fakelibvirt.

To answer your question though, if global-requirements.txt is enforcing
that we have libvirt-python 1.2.5, then we can drop that particular
workaround.

Re: [openstack-dev] [QA] Question about Heat Tempest tests

2015-05-15 Thread Yaroslav Lobankov

Angus,

If you take a look at the tempest check job check-tempest-dsvm-heat for
the patch https://review.openstack.org/#/c/182971/, you will see that all
tests are skipped there. I have not found any job anywhere that runs these
tests.

Regards,
Yaroslav Lobankov.

On Fri, May 15, 2015 at 2:16 AM, Angus Salkeld asalk...@mirantis.com
wrote:

 On Fri, May 15, 2015 at 1:53 AM, Yaroslav Lobankov yloban...@mirantis.com
  wrote:

 Hello everyone,

 I have a question about Heat Tempest tests. Is there any dsvm job that
 runs these tests? At first glance no dsvm job runs them.


 We are using in tree functional tests now:
 https://github.com/openstack/heat/tree/master/heat_integrationtests

 And heat has a tempest check job too (for the API tests):
 check-tempest-dsvm-heat
 for example: https://review.openstack.org/#/c/182971/

 -Angus

 Thank you!

 Regards,
 Yaroslav Lobankov.

 __
 OpenStack Development Mailing List (not for usage questions)
 Unsubscribe:
 openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



 __
 OpenStack Development Mailing List (not for usage questions)
 Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Openstack-operators] [nova] Are we happy with libvirt-python = 1.2.0 ?

On 15 May 2015 at 13:28, Daniel P. Berrange berra...@redhat.com wrote:
On Fri, May 15, 2015 at 11:51:22AM +0100, Daniel P. Berrange wrote:
On Thu, May 14, 2015 at 02:23:25PM -0500, Matt Riedemann wrote:
There are some workarounds in the code [3] I'd like to see removed by
bumping the minimum required version.

Sure, its nice to remove workarounds from a cleanliness POV, but I'm
generally
pretty conservative about doing so, because in the majority of case (while it
looks ugly) it is not really a significant burden on maintainers to keep it
around.

BTW, the code you quote here:

http://git.openstack.org/cgit/openstack/nova/tree/nova/virt/libvirt/host.py?id=2015.1.0#n754

So what's really needed is a dep on libvirt-python = 1.2.0, not libvirt.

Does this mean we can drop the above [3] code?
https://github.com/openstack/requirements/blob/master/global-requirements.txt#L56

Thanks,
John

[3]
http://git.openstack.org/cgit/openstack/nova/tree/nova/virt/libvirt/host.py?id=2015.1.0#n754

Re: [openstack-dev] [Murano] [Mistral] [Zaqar] [Keystone] SSH workflow action

2015-05-15 Thread Adam Young


On 05/15/2015 09:56 AM, Zane Bitter wrote:

On 14/05/15 23:38, Adam Young wrote:

So the mechanisms are there. In the short term we'd need some
cross-project co-operation to define a system through which we can do
this across projects (i.e. Murano or any other service can create a
user and have Zaqar authorise it for listening on a particular queue).
Maybe this is an extra parameter when creating a queue in Zaqar to
also create a user account in a separate domain (the way Heat does)
with permission to send and/or receive only on that particular queue
and return its credentials. That would be easier to secure and easier
to implement than having other services create the user accounts
themselves.


I think the user accounts (or consumers as oauth calls them) should be
cheap, and easy to create .  I see no reason to try to limit them.
Ideally, we would use something like X509 in order for them to talk to
Keystone;  that work is under way in Keystone already. Kerberos works
for those who want to use it with a Keytab.


I was talking about a short term solution to follow Heat's existing 
practices, without any changes in Keystone. It'd be much easier for 
Zaqar to say 'here is your queue, and here is the user I created that 
is already correctly authorised for it' than it would be for every 
service to have its own separate Keystone domain to set up and for 
them all to have to learn how to create users in it and then to pass 
the user to Zaqar when creating the queue to set up the authorisation.


If you're saying that Keystone already supports, or will soon support, 
what we need to do by using OAuth then I'm *more* than happy to dump 
this idea and move straight to that.


Think of OAuth as a more standard API, and the Domains as the 
impklementation.  There is 0 difference in capability, but oauth is a 
(sorta) standard and Keystone is not.


In oauth, all consumers go into a single table.  We could replicate that 
by putting all of these service users in  a single  domain.





In the medium term hopefully we'll be able to come up with a less
hacky solution that uses Oslo libraries to manage all of the user
creation and authorisation.


Why Oslo oand not Keystone?  Why do we need a new abstraction?


Again, if Keystone does everything we need then I agree there's no 
problem to solve here.
Good.  I'd rather keep this stuff in a single place.  Oslo is good for 
things that are cross cutting concerns, which means that sometimes there 
are unclear boundaries between that goes ther and what goes into 
Keystone.  Poliocy is a good example:  the generic rules engine goes 
into Osol.  The use of that engine for access control stays with Keystone.


It will be interesting to discuss this with Keystone team. What is 
it is

possible to have a token which is restricted to be authenticated to
specific API URL like  GET /v1/queues/queue-id/


Yes, we should definitely discuss this with the Keystone team and make
sure they're getting feedback from all of the many groups who need it
so that they can prioritise the work appropriately :)



Endpoint binding of tokens is proposed already.  I have a spec out for
more constraints.  We need a way to annotate them in the token, and then
policy can certainly be enforced on any data in the token.


This is OAuth tokens?

Nope, all tokens.  OAuth ends up with a keystone token, too.


In the existing world of expiring bearer tokens I think we'd probably 
need application servers to hold their own auth credentials, not just 
a token, so they can keep refreshing it. Which implies that the 
scoping should be on the user, not the token.


Agreed. A service user to shadow the real user is the right 
abstraction;  it has its own credentials, and limited scope of access.


It's kind of unfortunate IMHO that the default policy.json files tend 
to give all users access to non-admin APIs, rather than requiring a 
specific role (like Member). 

Working on that.  Come to my policy session!


Although that's kind of counterbalanced by the fact that when you 
create users, it has to be in a separate domain with its own separate 
tenant anyway, and right now the application has to do its own mapping 
between tenants in different domains to know whom to trust (at least, 
this is how Heat currently does it).


Yes.  Let's work through the use cases here, and see what we can propose.


If OAuth makes all of these problems go away, then +1000 from me :)

No silver bullet. sorry.



I suspect that what we will have is a two pass policy check;  the first
will be all global options (endpoint binding, etc) and the second will
be API specific checks.


That makes sense to me.

cheers,
Zane.

__ 


OpenStack Development Mailing List (not for usage questions)
Unsubscribe: 
openstack-dev-requ...@lists.openstack.org?subject:unsubscribe

http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Openstack-operators] [nova] Can we bump MIN_LIBVIRT_VERSION to 1.2.2 in Liberty?




On 5/15/2015 6:28 AM, Daniel P. Berrange wrote:

On Fri, May 15, 2015 at 11:51:22AM +0100, Daniel P. Berrange wrote:

On Thu, May 14, 2015 at 02:23:25PM -0500, Matt Riedemann wrote:

The minimum required version of libvirt in the driver is 0.9.11 still [1].
We've been gating against 1.2.2 in Ubuntu Trusty 14.04 since Juno.

The libvirt distro support matrix is here: [2]

Can we safely assume the people aren't going to be running Libvirt compute
nodes on RHEL  7.1 or Ubuntu Precise?


I don't really think so - at the very least Fedora 20 and RHEL 7.0 are still
actively supported platforms by their vendors, which both have older libvirt
versions (1.1.3 and 1.1.1 respectively).

I'm not sure whether SUSE team consider any of the 12.x versions to still be
actively supported platforms or not, likewise which 13.x versions are under
active support.



Regarding RHEL, I think this is a safe bet because in Kilo nova dropped
python 2.6 support and RHEL  6 doesn't have py26 so you'd be in trouble
running kilo+ nova on RHEL 6.x anyway.


There are add-on repos for RHEL-6 and RHEL-7 that provide newer python
versions so (py27, various py3x), so it is not unreasonable for people
to consider sticking with RHEL-6 if that is what their existing deployment
is based on. Certainly new deployments though I'd expect to be RHEL-7 based.


There are some workarounds in the code [3] I'd like to see removed by
bumping the minimum required version.


Sure, its nice to remove workarounds from a cleanliness POV, but I'm generally
pretty conservative about doing so, because in the majority of case (while it
looks ugly) it is not really a significant burden on maintainers to keep it
around.

This example is really just that. It certainly looks ugly, but we have the
code there now, it is doing the job for people who have that problem and it
isn't really having any measurable impact on our ability to maintain the
libvirt code. Removing this code won't lessen our maintainance burden in
any way, but it will unquestionably impact our users by removing support for
the platform they may be currently deployed on.

The reason why we picked 0.9.11 as the current minimum version was that we
needed to be able to switch over to using libvirt-python from PyPi instead
of relying on the version that shipped with the distro. 0.9.11 is the min
version supported by libvirt-python on PyPi. This had significant benefits
to Nova maintainence, as it our gate jobs deploy libvirt-python from PyPi
in common with all other python packages we depend on. It also unlocked the
ability to run libvirt with python3 bindings.  There was some small amount
of pain for people running Ubuntu 12.04 LTS, but this was mitigated by the
fact that Canonical provided the Cloud Archive repositories for that LTS
version which gave users direct access to new enough libvirt. So in the end
users were not negatively impacted in any serious way - certainly not by
enough to outweigh the benefits Nova maintainence saw.


In this case, I just don't see compelling benefits to Nova libvirt maint
to justify increasing the minimum version to the level you suggest, and
it has a clear negative impact on our users which they will not be able
to easily deal with. They will be left with 3 options all of which are
unsatisfactory

  - Upgrade from RHEL-6 to RHEL-7 - a major undertaking for most organizations
  - Upgrade libvirt on RHEL-6 - they essentially take on the support burden
for the hypervisor themselves, loosing support from the vendor
  - Re-add the code we remove from Nova - we've given users a maint burden,
to rid ourselves of code that was posing no real maint burden on ourselves.


As a more general point, I think we are lacking clear guidance on our
policies around hypervisor platform support and thus have difficulty
in deciding when it is reasonable for us to drop support for platforms.

I think it is important to distinguish the hypervisor platform, from
the openstack platform because there are different support implications
there for users. With direct python dependancies for openstack, I we
are generally pretty aggressive at updating to newer versions of packages
from PyPi. There are a number of reasons why this is reasonable to do.
Foremost is that many python modules do a very bad job at maintaining
API compatibility, so we are essentially forced to upgrade and drop old
version support whether we like it or not. Second, the provisioning tools
we use (devstack, packstack, triple-o, and many vendor tools) all handle
deployment of arbitrary newer python modules without any trouble. OpenStack
itself and the vendors all do quite comprehensive testing on the python
modules we use, so we have confidence that the versions we're depending
on are functioning correctly.

The hypervisor platform is very different. While OpenStack does achieve
some level of testing coverage of the hypervisor platform version used
in the gate, this testing is inconsequential compared to the level of

Re: [openstack-dev] [neutron][lbaas]LBaaSv2 movies / links

2015-05-15 Thread Samuel Bercovici

Was thinking 1-2 minutes. Tomorrow would be fine.

From: Vijay Venkatachalam [mailto:vijay.venkatacha...@citrix.com]
Sent: Friday, May 15, 2015 6:03 PM
To: OpenStack Development Mailing List (not for usage questions); Evgeny 
Fedoruk; Adam Harwell; Kyle Mestery; Brandon Logan; Johnson, Michael (HP Cloud 
- Corvallis); Doug Wiegley; Samuel Bercovici
Subject: Re: [openstack-dev] [neutron][lbaas]LBaaSv2 movies / links

Hi Sam,
Thanks for probing. How many seconds/mins you have thought per vendor? By when 
do you need this? Will tomorrow work fine?


Thanks,
Vijay V.

Sent from Surface

From: Samuel Bercovicimailto:samu...@radware.com
Sent: ‎Friday‎, ‎15‎ ‎May‎ ‎2015 ‎00‎:‎30
To: OpenStack Development Mailing List (not for usage 
questions)mailto:openstack-dev@lists.openstack.org, Evgeny 
Fedorukmailto:evge...@radware.com, Adam 
Harwellmailto:adam.harw...@rackspace.com, Kyle 
Mesterymailto:mest...@mestery.com, Brandon 
Loganmailto:brandon.lo...@rackspace.com, Johnson, Michael (HP Cloud - 
Corvallis)mailto:michael.d.john...@hp.com, Doug 
Wiegleymailto:do...@a10networks.com, Samuel 
Bercovicimailto:samu...@radware.com

Hi Everyone,

As you may be aware, we have a speaking slot on the Vancouver summit to discuss 
“LBaaS v2, Kilo and beyond” on Monday 
https://openstacksummitmay2015vancouver.sched.org/event/3f1e9e24f36238152749afea9c21a264#.VVTwCPmqqko

We are considering to show vendors demos or list/link such movies.
Please let us know if you have such so we can list/include in the talk.

Regards,
-Sam.

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [nova] Unvalidated user input passed to functions

2015-05-15 Thread Eric Blake

On 05/15/2015 05:41 AM, Matthew Booth wrote:
 I was looking at the migrations api, and I noticed that the api passes
 the request query unchecked to get_migrations, where it ultimately ends
 up in a db query. I was curious and spent a couple of hours checking
 this morning. There are a few instances of this.
 
 I didn't find any security bugs, however I feel that this extremely bad
 practise, and is likely to result in a security bug eventually. For
 example, note that os-assisted-volume-snapshots:delete does not validate
 delete_info before passing it to volume_snapshot_delete. I looked at
 this quite carefully, and I think we are only protected from a host
 compromise because:
 
 1. The api requires admin context
 2. libvirt's security policy
 
 I could be wrong on that, though, so perhaps somebody else could check?
 
 Passing unvalidated input to a function isn't necessarily bad, for
 example if it is only used for filtering, but it should be clearly
 marked as such so it isn't used in an unsafe manner. This marking should
 follow the data as far as it goes through any number of function calls.
 libvirt's _volume_snapshot_delete function is a long way from the
 originating api call, and it is not at all obvious that the commit_base
 and commit_top arguments to virt_dom.blockCommit() are unvalidated.

Libvirt validates that the base and top arguments to blockcommit make
sense (in part because it may have to rewrite the string passed in to a
different but equivalent file name for qemu to do the right thing, since
qemu does strcmp rather than inode matching).  Qemu also has the ability
to set an arbitrary backing file string into the metadata; if this
arbitrary string is under user control, then it is up to the user to
validate that the string is correct to avoid breaking the chain (and
doing something nasty like setting /etc/passwd as the new backing file
the next time the chain is parsed from qcow2 files).  But I don't think
libvirt exposes the arbitrary backing name to the user, but rather
computes a relative backing string itself, so that also doesn't seem to
be a problem.

And yes, you are protected by requiring admin context - anyone that can
cause libvirt to start a new domain and write arbitrary XML already has
effective root permissions on the host, because they can design the XML
to hand any file of their choosing to the guest.  Security is only at
risk when there is elevation - if a guest could do things to cause the
host to hand away privileged files, rather than only the host changing
XML or backing file strings, is when we have to start worrying.  The
host changing strings is not elevation, just the user shooting
themselves in the foot.

But you are also right that it might be nice to validate strings prior
to handing them to libvirt - while libvirt is able to validate that
strings make sense within the chains that libvirt is aware of, it cannot
know if there are additional restrictions that should be in place at the
upper level (such as whether a user is entitled to access the storage
locations referenced in the strings, according to nova rules).


-- 
Eric Blake   eblake redhat com+1-919-301-3266
Libvirt virtualization library http://libvirt.org



signature.asc
Description: OpenPGP digital signature
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] devstack local.conf file fro sriov pci nic passthrough

2015-05-15 Thread Kamsali, RaghavendraChari (Artesyn)

Hi,
I am bringing up the single node openstack cloud and having pci sriov supported 
nic controller (intel XL710), created 4virtual functions on top of the nic. My 
goal is to bring up the cloud setup with neutron and nova network services 
using devstack.
How to configure local.conf file  so that when any VM spawns , it used the 
virtual function of the sriov nic and able to communicate .

Could anyone help me ???


Thanks and Regards,
Raghavendrachari kamsali | Software Engineer II  | Embedded Computing
Artesyn Embedded Technologies | 5th Floor, Capella Block, The V, Madhapur| 
Hyderabad, AP 500081 India
T +91-40-66747059 | M +919705762153

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Murano] [Mistral] [Zaqar] [Keystone] SSH workflow action

2015-05-15 Thread Zane Bitter


On 14/05/15 23:38, Adam Young wrote:

So the mechanisms are there. In the short term we'd need some
cross-project co-operation to define a system through which we can do
this across projects (i.e. Murano or any other service can create a
user and have Zaqar authorise it for listening on a particular queue).
Maybe this is an extra parameter when creating a queue in Zaqar to
also create a user account in a separate domain (the way Heat does)
with permission to send and/or receive only on that particular queue
and return its credentials. That would be easier to secure and easier
to implement than having other services create the user accounts
themselves.


I think the user accounts (or consumers as oauth calls them) should be
cheap, and easy to create .  I see no reason to try to limit them.
Ideally, we would use something like X509 in order for them to talk to
Keystone;  that work is under way in Keystone already. Kerberos works
for those who want to use it with a Keytab.


I was talking about a short term solution to follow Heat's existing 
practices, without any changes in Keystone. It'd be much easier for 
Zaqar to say 'here is your queue, and here is the user I created that is 
already correctly authorised for it' than it would be for every service 
to have its own separate Keystone domain to set up and for them all to 
have to learn how to create users in it and then to pass the user to 
Zaqar when creating the queue to set up the authorisation.


If you're saying that Keystone already supports, or will soon support, 
what we need to do by using OAuth then I'm *more* than happy to dump 
this idea and move straight to that.



In the medium term hopefully we'll be able to come up with a less
hacky solution that uses Oslo libraries to manage all of the user
creation and authorisation.


Why Oslo oand not Keystone?  Why do we need a new abstraction?


Again, if Keystone does everything we need then I agree there's no 
problem to solve here.



It will be interesting to discuss this with Keystone team. What is it is
possible to have a token which is restricted to be authenticated to
specific API URL like  GET /v1/queues/queue-id/


Yes, we should definitely discuss this with the Keystone team and make
sure they're getting feedback from all of the many groups who need it
so that they can prioritise the work appropriately :)



Endpoint binding of tokens is proposed already.  I have a spec out for
more constraints.  We need a way to annotate them in the token, and then
policy can certainly be enforced on any data in the token.


This is OAuth tokens?

In the existing world of expiring bearer tokens I think we'd probably 
need application servers to hold their own auth credentials, not just a 
token, so they can keep refreshing it. Which implies that the scoping 
should be on the user, not the token. It's kind of unfortunate IMHO that 
the default policy.json files tend to give all users access to non-admin 
APIs, rather than requiring a specific role (like Member). Although 
that's kind of counterbalanced by the fact that when you create users, 
it has to be in a separate domain with its own separate tenant anyway, 
and right now the application has to do its own mapping between tenants 
in different domains to know whom to trust (at least, this is how Heat 
currently does it).


If OAuth makes all of these problems go away, then +1000 from me :)


I suspect that what we will have is a two pass policy check;  the first
will be all global options (endpoint binding, etc) and the second will
be API specific checks.


That makes sense to me.

cheers,
Zane.

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Murano] [Mistral] [Zaqar] [Keystone] SSH workflow action

2015-05-15 Thread Fox, Kevin M

And if its tied into nova, perhaps the service account password never is given 
to the vm, but the vm has a mechanism for fetching fresh keystone tokens, maybe 
a special url on the metadata server.

Thanks,
Kevin

From: Fox, Kevin M
Sent: Friday, May 15, 2015 7:56 AM
To: OpenStack Development Mailing List (not for usage questions)
Subject: Re: [openstack-dev] [Murano] [Mistral] [Zaqar] [Keystone] SSH workflow 
action

I think we have to be very careful with cheep, easy, user provisionable user 
accounts. I know we've had a hard enough time getting clouds to land at our 
organization because policies had to be adapted. I think the policies around 
User account allocation and particularly password strength, password rotation 
and administratively disabling them, will be the biggest challenge to the 
policies yet.

Now if it was kind of a different thing, service accounts, that might be more 
palatable to the powers that be.

Thanks,
Kevin

From: Adam Young [ayo...@redhat.com]
Sent: Thursday, May 14, 2015 8:38 PM
To: openstack-dev@lists.openstack.org
Subject: Re: [openstack-dev] [Murano] [Mistral] [Zaqar] [Keystone] SSH workflow 
action

On 05/12/2015 09:43 PM, Zane Bitter wrote:
 On 12/05/15 13:06, Georgy Okrokvertskhov wrote:
 There is one thing which still bothers me. It is authentication. Right
 now with separate RabbitMQ instance we keep VMs authentication isolated
 from OpenStack infra.
 This is still a problem if you want to use webhooks (Heat autoscaling,
 Murano actions) via our own authentication models. If we plan to use
 Zaqar it will be interesting to know how Zaqar solves this issue.

 Aha, if you'd read my blog post you would know the answer ;)

 There's no specific provision for this in Zaqar at the moment AFAIK.
 The problem is really Keystone: it was never designed for
 authenticating applications to the cloud, only real live users.

 We need to fix this, in Keystone  Oslo, for the benefit of all
 application-facing services. Some work has already started:

 - Keystone can now support separate backends for separate domains, so
 even if you are backed by read-only LDAP you can create service users
 in a separate DB-backed domain:
 http://adam.younglogic.com/2014/08/getting-service-users-out-of-ldap/

 - Work is planned on Dynamic Policy to make the authorisation model
 more powerful:
 http://adam.younglogic.com/2014/11/dynamic-policy-in-keystone/

 I'm not sure that this goes far enough though (although I don't
 completely grok the implications of Dynamic Policy). We really want
 users to be able to define their own policy for application accounts
 that they create, and at an even more fine-grained level, so a common
 library for this sort of authorisation would be helpful.

Yeah, we need Authorization as a Service.  Was discussing this earlier
today;  how do we assign Admin for Wordpress without accidentally
granting the user the ability to create new virtual machines or networks.

One part is namespacing of roles.  Nova:admin vs wordpress:admin.

User would also need to be able to register new policy enforcement
points for the code.  These would be predefined for big applications
(Wordpress) but would have to be well thought out for others.

I also don't think we want to make users get a token before perform
every action.  THe apps should be able to query the information they
need from Keystone when the user comes in.  It is a combination of
Fedeartion, mapping, list_roles-for_user_and_project and extending of
policy.

I'm sure this will come up next week.

 Frankly, I don't think that this is a good idea to use Keystone
 credentials or tokens for MQ clients on VMs. This topic, probably,
 deserves its own e-mail thread.

 Keystone credentials _are_ the answer, but they must not be the
 *user's* Keystone credentials.

 I can tell you how Heat does this right now for authenticating
 application callbacks for WaitConditions. It's not exactly pretty, but
 it works. Basically we create the application users in a separate
 domain and then do extra authorisation checks ourselves. Steve Hardy
 wrote a comprehensive summary on his blog:
 http://hardysteven.blogspot.com/2014/04/heat-auth-model-updates-part-2-stack.html

 So the mechanisms are there. In the short term we'd need some
 cross-project co-operation to define a system through which we can do
 this across projects (i.e. Murano or any other service can create a
 user and have Zaqar authorise it for listening on a particular queue).
 Maybe this is an extra parameter when creating a queue in Zaqar to
 also create a user account in a separate domain (the way Heat does)
 with permission to send and/or receive only on that particular queue
 and return its credentials. That would be easier to secure and easier
 to implement than having other services create the user accounts
 themselves.

I think the user accounts (or consumers as oauth calls them) should be
cheap,

Re: [openstack-dev] [nova][cinder] can we deprecate the volume CLIs in novaclient?

2015-05-15 Thread John Griffith

On Thu, May 14, 2015 at 8:29 PM, Matt Riedemann mrie...@linux.vnet.ibm.com
wrote:

 This came up while talking about bug 1454369 [1].  This also came up at
 one point in kilo when we found out the volume CLIs in novaclient didn't
 work at one point and we broke the cells devstack exercises job because of
 it.

 python-novaclient uses cinder API to handle the volume CLI rather than
 going to the nova volume API.  There are issues with this because
 novaclient needs a certain endpoint/service_type setup in the service
 catalog to support cinder v1/v2 APIs (whatever devstack sets up today).
 novaclient defaults to volume (v1) and if you disable that in cinder then
 novaclient doesn't work because it's not using volumev2.

 So like anyone might ask, why doesn't novaclient talk to nova volume APIs
 to do volume thingies and the answer is because the nova volume API doesn't
 handle all of the volume thingies like snapshots and volume types.

 So I got to to thinking, why the hell are we still supporting volume
 operations via novaclient anyway?  Isn't that cinderclient's job?  Or
 python-openstackclient's job?  Can't we deprecate the volume CLIs in
 novaclient and tell people to use cinderclient instead since it now has
 version discovery [2] so that problem would be handled for us.

 Since we have nova volume APIs maybe we can't remove the volume CLIs in
 novaclient, but could they be limited to just operations that the nova API
 supports and then we make novaclient talk to nova volume APIs rather than
 cinder APIs (because the nova API will talk to cinderclient which again has
 the version discovery done for us).

 Or assuming we could deprecate the volume CLIs in novaclient, what would
 the timeline on deprecation be since it's not a server project with a 6
 month release cycle?  I'm assuming we'd still have 6-12 months deprecation
 on a client like this because of all of the tooling potentially written
 around it.

 [1] https://bugs.launchpad.net/python-novaclient/+bug/1454369
 [2] https://review.openstack.org/#/c/145613/

 --

 Thanks,

 Matt Riedemann


 __
 OpenStack Development Mailing List (not for usage questions)
 Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


I can't speak for the nova folks, however i do think removing the volume
calls from novaclient seems ok.  It was always sort of left for compat I
think, and not sure any of us really thought about just removing it.  At
this point it probably just introduces confusion and as you're running into
problems.

Seems like a good plan, and somewhat less confusing.  On a side note, might
be some other *things* in novaclient that we could look at as well,
particularly around networking.  
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [monasca] [java]

2015-05-15 Thread Everett Toews

On May 15, 2015, at 3:49 AM, Thierry Carrez thie...@openstack.org wrote:

 Dieterly, Deklan wrote:
 We’ve seen that Swift has introduced components in Go. So, this looks like a 
 precedent for allowing other languages where deemed appropriate. Before we 
 spend many man-hours hacking on the Python components, it seems reasonable 
 to determine if there really exists a reason to do so. I’m interested in 
 soliciting any feedback from the community be it pleasant or unpleasant.
 
 Swift has not introduced components in Go. It is at the early stages of
 *exploring* the possibility of doing so, through a specific feature branch.
 
 The Technical Committee position has always been python unless there is
 a compelling reason otherwise. Every language supported increases
 fragmentation of our community and increases the CI effort. The argument
 for adding a language has to be pretty compelling to counterbalance the
 damage it does to OpenStack as a development community.
 
 In Monasca's case, there is always the possibility to stay out of the
 OpenStack tent and stay in Java. There is the possibility to rewrite
 things in Python. And there is the possibility to convince the Technical
 Committee that (1) we want Monasca featureset in so badly we would add
 Java as a supported language just so that can happen and (2) Monasca
 featureset can't be written in Python.

Not to move this discussion off-list but I feel the need to at least point out 
a highly relevant summit session.

Is it time to have more than just Python in OpenStack?
https://libertydesignsummit.sched.org/event/6bb3f4fe34a4a0236266d99a2039c963

Everett


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Murano] [Mistral] [Zaqar] [Keystone] SSH workflow action

2015-05-15 Thread Adam Young


On 05/15/2015 11:12 AM, Fox, Kevin M wrote:

And if its tied into nova, perhaps the service account password never is given 
to the vm, but the vm has a mechanism for fetching fresh keystone tokens, maybe 
a special url on the metadata server.

Thanks,
Kevin

From: Fox, Kevin M
Sent: Friday, May 15, 2015 7:56 AM
To: OpenStack Development Mailing List (not for usage questions)
Subject: Re: [openstack-dev] [Murano] [Mistral] [Zaqar] [Keystone] SSH workflow 
action

I think we have to be very careful with cheep, easy, user provisionable user 
accounts. I know we've had a hard enough time getting clouds to land at our 
organization because policies had to be adapted. I think the policies around 
User account allocation and particularly password strength, password rotation 
and administratively disabling them, will be the biggest challenge to the 
policies yet.

Now if it was kind of a different thing, service accounts, that might be more 
palatable to the powers that be.

Thanks,
Kevin


I'd like service accounts to be tied in with X509 validation and not 
even be able to do Password.  We can use Barbican to do the actual X509 
issue.


Revocations might be an issue;  We need to make sure the Keystone server 
checks a CRL from Barbican.


Kerberos is a better option.  With FreeIPA, it is a viable alternative 
even for smaller deployments.





From: Adam Young [ayo...@redhat.com]
Sent: Thursday, May 14, 2015 8:38 PM
To: openstack-dev@lists.openstack.org
Subject: Re: [openstack-dev] [Murano] [Mistral] [Zaqar] [Keystone] SSH workflow 
action

On 05/12/2015 09:43 PM, Zane Bitter wrote:

On 12/05/15 13:06, Georgy Okrokvertskhov wrote:

There is one thing which still bothers me. It is authentication. Right
now with separate RabbitMQ instance we keep VMs authentication isolated
from OpenStack infra.
This is still a problem if you want to use webhooks (Heat autoscaling,
Murano actions) via our own authentication models. If we plan to use
Zaqar it will be interesting to know how Zaqar solves this issue.

Aha, if you'd read my blog post you would know the answer ;)

There's no specific provision for this in Zaqar at the moment AFAIK.
The problem is really Keystone: it was never designed for
authenticating applications to the cloud, only real live users.

We need to fix this, in Keystone  Oslo, for the benefit of all
application-facing services. Some work has already started:

- Keystone can now support separate backends for separate domains, so
even if you are backed by read-only LDAP you can create service users
in a separate DB-backed domain:
http://adam.younglogic.com/2014/08/getting-service-users-out-of-ldap/

- Work is planned on Dynamic Policy to make the authorisation model
more powerful:
http://adam.younglogic.com/2014/11/dynamic-policy-in-keystone/

I'm not sure that this goes far enough though (although I don't
completely grok the implications of Dynamic Policy). We really want
users to be able to define their own policy for application accounts
that they create, and at an even more fine-grained level, so a common
library for this sort of authorisation would be helpful.

Yeah, we need Authorization as a Service.  Was discussing this earlier
today;  how do we assign Admin for Wordpress without accidentally
granting the user the ability to create new virtual machines or networks.

One part is namespacing of roles.  Nova:admin vs wordpress:admin.

User would also need to be able to register new policy enforcement
points for the code.  These would be predefined for big applications
(Wordpress) but would have to be well thought out for others.

I also don't think we want to make users get a token before perform
every action.  THe apps should be able to query the information they
need from Keystone when the user comes in.  It is a combination of
Fedeartion, mapping, list_roles-for_user_and_project and extending of
policy.

I'm sure this will come up next week.



Frankly, I don't think that this is a good idea to use Keystone
credentials or tokens for MQ clients on VMs. This topic, probably,
deserves its own e-mail thread.

Keystone credentials _are_ the answer, but they must not be the
*user's* Keystone credentials.

I can tell you how Heat does this right now for authenticating
application callbacks for WaitConditions. It's not exactly pretty, but
it works. Basically we create the application users in a separate
domain and then do extra authorisation checks ourselves. Steve Hardy
wrote a comprehensive summary on his blog:
http://hardysteven.blogspot.com/2014/04/heat-auth-model-updates-part-2-stack.html

So the mechanisms are there. In the short term we'd need some
cross-project co-operation to define a system through which we can do
this across projects (i.e. Murano or any other service can create a
user and have Zaqar authorise it for listening on a particular queue).
Maybe this is an extra parameter when creating a queue in Zaqar

Re: [openstack-dev] [Murano] [Mistral] [Zaqar] [Keystone] SSH workflow action

2015-05-15 Thread Zane Bitter


On 15/05/15 11:57, Adam Young wrote:

It's kind of unfortunate IMHO that the default policy.json files tend
to give all users access to non-admin APIs, rather than requiring a
specific role (like Member).

Working on that.  Come to my policy session!


This one, I assume: 
http://libertydesignsummit.sched.org/event/0c0aa8aa4b99c5f2c1781c7651f8e604#.VVaBEX_U-4M


Is there going to be a design summit session related to this stuff as well?


If OAuth makes all of these problems go away, then +1000 from me :)

No silver bullet. sorry.


Dang ;)

-ZB

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [nova][vmware] Minimum VC version

2015-05-15 Thread Gary Kotton

Hi,
We would like to indicate that we do not support versions below 5.1.0 of the 
VC. Is anyone aware of people using versions below with OpenStack. Patch 
https://review.openstack.org/#/c/183711/ proposes exiting Nova compute if a 
lower version is used.
Thanks
Gary
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Openstack-operators] [nova][vmware] Minimum VC version

2015-05-15 Thread Steve Gordon

- Original Message -
 From: Dan Smith d...@danplanet.com
 To: OpenStack Development Mailing List (not for usage questions) 
 openstack-dev@lists.openstack.org,

  But 4.x was EOL over a year ago:

  http://kb.vmware.com/selfservice/microsites/search.do?language=en_UScmd=displayKCexternalId=2039567

 ...and was released in 2010.

 We're supporting a minimum version of libvirt from 2014, so I think that
 dropping support for five-year-old EOL'd VMware is good. We don't test
 it, which means it's probably broken. I also feel like this is a thing
 we can do without a deprecation cycle, assuming there aren't a ton of
 users still using unsupported commercial software out there.

 --Dan

The proposed patch also drops support for 5.0, which as I understand it is not 
EOL'd? The documentation appears to indicate that some functionality will not 
work with  5.1, but it's not explicitly clear what that it is.

Thanks,

Steve

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Openstack-operators] [nova] Can we bump MIN_LIBVIRT_VERSION to 1.2.2 in Liberty?

2015-05-15 Thread Robert Collins

On 16 May 2015 at 00:28, Daniel P. Berrange berra...@redhat.com wrote:

 is not actually working around a bug the libvirt hypervisor. It is in fact
 a bug in the libvirt-python API binding. As such we don't actually need to
 increase the min required libvirt to be able to remove that check. In fact
 increasing the min required libvirt is the wrong thing todo, because it is
 possible for someone to have the min required libvirt, but by accessing it
 via an older libvirt-python which still has the bug.

 So what's really needed is a dep on libvirt-python = 1.2.0, not libvirt.

 We don't express min required versions for libvirt-python in the
 requirements.txt file though, since it is an optional package and we
 don't have any mechanism for recording min versions for those AFAIK.

The next release of pbr (due Mondayish) has support for optional
dependencies ('extras' in setuptools parlance) - which will allow
this. We're having a session about such things in Vancouver too.

-Rob

-- 
Robert Collins rbtcoll...@hp.com
Distinguished Technologist
HP Converged Cloud

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Openstack-operators] [nova] Are we happy with libvirt-python = 1.2.0 ?




On 5/15/2015 9:52 AM, Jeremy Stanley wrote:

On 2015-05-15 14:54:37 +0100 (+0100), Daniel P. Berrange wrote:

Hmm, I didn't know it was listed in global-requirements.txt - I only
checked the requirements.txt and test-requirements.txt in Nova itself
which does not list libvirt-python.

Previously test-requirements.txt did have it, but we dropped it, since
the unit tests now exclusively use fakelibvirt.

To answer your question though, if global-requirements.txt is enforcing
that we have libvirt-python 1.2.5, then we can drop that particular
workaround.


If it's listed in openstack/requirements:global-requirements.txt but
not in any individual repo's requirements.txt or
test-requirements.txt then it's not actually doing anything. It's
just cruft which someone failed to remove once nova stopped
explicitly referencing it as a test requirement.

Note I've added a script in the requirements repo (tools/cruft.sh)
to find things like this, so that interested parties can use it as a
starting point to research possible cleanup work there.



https://review.openstack.org/#/c/183706/ adds libvirt-python back into 
nova's test-requirements.txt.


--

Thanks,

Matt Riedemann


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [nova][vmware] Minimum VC version




On 5/15/2015 4:50 PM, Gary Kotton wrote:

Hi,
We would like to indicate that we do not support versions below 5.1.0 of
the VC. Is anyone aware of people using versions below with OpenStack.
Patch https://review.openstack.org/#/c/183711/ proposes exiting Nova
compute if a lower version is used.
Thanks
Gary


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



Cross-posting to the operators mailing list.

Also note the kilo docs mention supporting less than 5.0:

http://docs.openstack.org/kilo/config-reference/content/vmware.html

But 4.x was EOL over a year ago:

http://kb.vmware.com/selfservice/microsites/search.do?language=en_UScmd=displayKCexternalId=2039567

Also note that the NSX CI (vmware CI) originally ran with vcenter 5.1 
and is now running 5.5.


--

Thanks,

Matt Riedemann


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [nova][vmware] Minimum VC version

2015-05-15 Thread Dan Smith

 But 4.x was EOL over a year ago:
 
 http://kb.vmware.com/selfservice/microsites/search.do?language=en_UScmd=displayKCexternalId=2039567

...and was released in 2010.

We're supporting a minimum version of libvirt from 2014, so I think that
dropping support for five-year-old EOL'd VMware is good. We don't test
it, which means it's probably broken. I also feel like this is a thing
we can do without a deprecation cycle, assuming there aren't a ton of
users still using unsupported commercial software out there.

--Dan



signature.asc
Description: OpenPGP digital signature
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [Horizon] Horizon Usage Survey Results

2015-05-15 Thread Michael Krotscheck

Hey everyone!

I've compiled some *preliminary* results from the horizon usage survey I'm
running. I'll be soliciting more responses during the summit, and if you
forgot to contribute, you can still participate. The link is here:
http://tinyurl.com/horizon-usage-survey . If you've already participated,
THANK YOU! It's awesome to see all this come together.

The current summary of findings is published here:
http://www.krotscheck.net/2015/05/15/horizon-usage-survey.html

Feel free to find me at the summit (around the HP booth during the booth
crawl), or reach out to me either here via discussion, on IRC as
krotscheck, or via email if you have any additional questions.

Michael
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Openstack-operators] [nova][vmware] Minimum VC version

2015-05-15 Thread Dan Smith

 The proposed patch also drops support for 5.0, which as I understand
 it is not EOL'd? The documentation appears to indicate that some
 functionality will not work with  5.1, but it's not explicitly clear
 what that it is.

Yeah, I guess I assumed that anyone on 5.0 was just late moving to
=5.1, where people on 4.x might have a license standing in their way.

Maybe refusing 5.0 and warning about 5.5 (which is what is now being
tested) is the right thing to do?

--Dan

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Openstack-operators] [nova][vmware] Minimum VC version

2015-05-15 Thread Gary Kotton



On 5/15/15, 3:57 PM, Dan Smith d...@danplanet.com wrote:

 The proposed patch also drops support for 5.0, which as I understand
 it is not EOL'd? The documentation appears to indicate that some
 functionality will not work with  5.1, but it's not explicitly clear
 what that it is.

Yeah, I guess I assumed that anyone on 5.0 was just late moving to
=5.1, where people on 4.x might have a license standing in their way.

Maybe refusing 5.0 and warning about 5.5 (which is what is now being
tested) is the right thing to do?

That sounds reasonable to me. I would even go for as much as refusing
below 5.1.0. 
It would be interesting to know if anyone is actually using below 5.1.0


--Dan

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [Fuel] Please don't close bugs associated with blueprints as Won't Fix

2015-05-15 Thread Dmitry Borodaenko

Fuelers,

If a bug is a feature request in disguise, or simply requires a new
feature before it can be fixed, please don't close it as Won't Fix
after associating it with the corresponding blueprint. Instead, please
target it to the same release as the blueprint and leave it in Triaged
(if root cause and solution is understood) or Confirmed state.

I have clarified the description of Wishlist priority in the Fuel bug
triage instructions to make sure there's no ambiguity on that point:
https://wiki.openstack.org/wiki/Fuel/How_to_contribute#Confirm_and_triage_bugs

Thank you,

-- 
Dmitry Borodaenko

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Murano] [Mistral] [Zaqar] [Keystone] SSH workflow action

2015-05-15 Thread Adam Young


On 05/15/2015 07:30 PM, Zane Bitter wrote:

On 15/05/15 11:57, Adam Young wrote:

It's kind of unfortunate IMHO that the default policy.json files tend
to give all users access to non-admin APIs, rather than requiring a
specific role (like Member).

Working on that.  Come to my policy session!


This one, I assume: 
http://libertydesignsummit.sched.org/event/0c0aa8aa4b99c5f2c1781c7651f8e604#.VVaBEX_U-4M


Yes



Is there going to be a design summit session related to this stuff as 
well?


Yescross project.

http://libertydesignsummit.sched.org/event/14775f798c7694cf43c1a630d64c05d6#.VVaWDbseEa0






If OAuth makes all of these problems go away, then +1000 from me :)

No silver bullet. sorry.


Dang ;)

-ZB

__ 


OpenStack Development Mailing List (not for usage questions)
Unsubscribe: 
openstack-dev-requ...@lists.openstack.org?subject:unsubscribe

http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Murano] [Mistral] [Zaqar] [Keystone] SSH workflow action

2015-05-15 Thread Fox, Kevin M

I think we have to be very careful with cheep, easy, user provisionable user 
accounts. I know we've had a hard enough time getting clouds to land at our 
organization because policies had to be adapted. I think the policies around 
User account allocation and particularly password strength, password rotation 
and administratively disabling them, will be the biggest challenge to the 
policies yet.

Now if it was kind of a different thing, service accounts, that might be more 
palatable to the powers that be.

Thanks,
Kevin

From: Adam Young [ayo...@redhat.com]
Sent: Thursday, May 14, 2015 8:38 PM
To: openstack-dev@lists.openstack.org
Subject: Re: [openstack-dev] [Murano] [Mistral] [Zaqar] [Keystone] SSH workflow 
action

On 05/12/2015 09:43 PM, Zane Bitter wrote:
 On 12/05/15 13:06, Georgy Okrokvertskhov wrote:
 There is one thing which still bothers me. It is authentication. Right
 now with separate RabbitMQ instance we keep VMs authentication isolated
 from OpenStack infra.
 This is still a problem if you want to use webhooks (Heat autoscaling,
 Murano actions) via our own authentication models. If we plan to use
 Zaqar it will be interesting to know how Zaqar solves this issue.

 Aha, if you'd read my blog post you would know the answer ;)

 There's no specific provision for this in Zaqar at the moment AFAIK.
 The problem is really Keystone: it was never designed for
 authenticating applications to the cloud, only real live users.

 We need to fix this, in Keystone  Oslo, for the benefit of all
 application-facing services. Some work has already started:

 - Keystone can now support separate backends for separate domains, so
 even if you are backed by read-only LDAP you can create service users
 in a separate DB-backed domain:
 http://adam.younglogic.com/2014/08/getting-service-users-out-of-ldap/

 - Work is planned on Dynamic Policy to make the authorisation model
 more powerful:
 http://adam.younglogic.com/2014/11/dynamic-policy-in-keystone/

 I'm not sure that this goes far enough though (although I don't
 completely grok the implications of Dynamic Policy). We really want
 users to be able to define their own policy for application accounts
 that they create, and at an even more fine-grained level, so a common
 library for this sort of authorisation would be helpful.

Yeah, we need Authorization as a Service.  Was discussing this earlier
today;  how do we assign Admin for Wordpress without accidentally
granting the user the ability to create new virtual machines or networks.

One part is namespacing of roles.  Nova:admin vs wordpress:admin.

User would also need to be able to register new policy enforcement
points for the code.  These would be predefined for big applications
(Wordpress) but would have to be well thought out for others.

I also don't think we want to make users get a token before perform
every action.  THe apps should be able to query the information they
need from Keystone when the user comes in.  It is a combination of
Fedeartion, mapping, list_roles-for_user_and_project and extending of
policy.

I'm sure this will come up next week.



 Frankly, I don't think that this is a good idea to use Keystone
 credentials or tokens for MQ clients on VMs. This topic, probably,
 deserves its own e-mail thread.

 Keystone credentials _are_ the answer, but they must not be the
 *user's* Keystone credentials.

 I can tell you how Heat does this right now for authenticating
 application callbacks for WaitConditions. It's not exactly pretty, but
 it works. Basically we create the application users in a separate
 domain and then do extra authorisation checks ourselves. Steve Hardy
 wrote a comprehensive summary on his blog:
 http://hardysteven.blogspot.com/2014/04/heat-auth-model-updates-part-2-stack.html

 So the mechanisms are there. In the short term we'd need some
 cross-project co-operation to define a system through which we can do
 this across projects (i.e. Murano or any other service can create a
 user and have Zaqar authorise it for listening on a particular queue).
 Maybe this is an extra parameter when creating a queue in Zaqar to
 also create a user account in a separate domain (the way Heat does)
 with permission to send and/or receive only on that particular queue
 and return its credentials. That would be easier to secure and easier
 to implement than having other services create the user accounts
 themselves.

I think the user accounts (or consumers as oauth calls them) should be
cheap, and easy to create .  I see no reason to try to limit them.
Ideally, we would use something like X509 in order for them to talk to
Keystone;  that work is under way in Keystone already. Kerberos works
for those who want to use it with a Keytab.


 In the medium term hopefully we'll be able to come up with a less
 hacky solution that uses Oslo libraries to manage all of the user
 creation and authorisation.

Why Oslo oand not Keystone?  Why do we need a new

Re: [openstack-dev] [Openstack-operators] [nova] Are we happy with libvirt-python = 1.2.0 ?

2015-05-15 Thread Jeremy Stanley

On 2015-05-15 14:54:37 +0100 (+0100), Daniel P. Berrange wrote:
 Hmm, I didn't know it was listed in global-requirements.txt - I only
 checked the requirements.txt and test-requirements.txt in Nova itself
 which does not list libvirt-python.
 
 Previously test-requirements.txt did have it, but we dropped it, since
 the unit tests now exclusively use fakelibvirt.
 
 To answer your question though, if global-requirements.txt is enforcing
 that we have libvirt-python 1.2.5, then we can drop that particular
 workaround.

If it's listed in openstack/requirements:global-requirements.txt but
not in any individual repo's requirements.txt or
test-requirements.txt then it's not actually doing anything. It's
just cruft which someone failed to remove once nova stopped
explicitly referencing it as a test requirement.

Note I've added a script in the requirements repo (tools/cruft.sh)
to find things like this, so that interested parties can use it as a
starting point to research possible cleanup work there.
-- 
Jeremy Stanley

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [neutron][lbaas]LBaaSv2 movies / links

2015-05-15 Thread Vijay Venkatachalam

Hi Sam,
Thanks for probing. How many seconds/mins you have thought per vendor? By when 
do you need this? Will tomorrow work fine?


Thanks,
Vijay V.

Sent from Surface

From: Samuel Bercovicimailto:samu...@radware.com
Sent: ‎Friday‎, ‎15‎ ‎May‎ ‎2015 ‎00‎:‎30
To: OpenStack Development Mailing List (not for usage 
questions)mailto:openstack-dev@lists.openstack.org, Evgeny 
Fedorukmailto:evge...@radware.com, Adam 
Harwellmailto:adam.harw...@rackspace.com, Kyle 
Mesterymailto:mest...@mestery.com, Brandon 
Loganmailto:brandon.lo...@rackspace.com, Johnson, Michael (HP Cloud - 
Corvallis)mailto:michael.d.john...@hp.com, Doug 
Wiegleymailto:do...@a10networks.com, Samuel 
Bercovicimailto:samu...@radware.com

Hi Everyone,

As you may be aware, we have a speaking slot on the Vancouver summit to discuss 
“LBaaS v2, Kilo and beyond” on Monday 
https://openstacksummitmay2015vancouver.sched.org/event/3f1e9e24f36238152749afea9c21a264#.VVTwCPmqqko

We are considering to show vendors demos or list/link such movies.
Please let us know if you have such so we can list/include in the talk.

Regards,
-Sam.

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [Glance] glance social event at Vancouver summit

2015-05-15 Thread Brian Rosmaita

Greetings to all Glance people,

kragniz has proposed that we have an informal social event for Glance and
Glance-related people attending the summit next week.  Please keep your
calendars open for about 2 hours or so starting at 7:30 p.m. on Wednesday,
May 20.  Neither kragniz nor I are familiar with Vancouver, so we'll
figure out logistics when we get there and announce details during the
Glance design sessions on Wednesday.

Looking forward to seeing everyone and getting some good design work done
next week ... safe travels!

rosmaita



__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Openstack-operators] [nova] Are we happy with libvirt-python = 1.2.0 ?

On 5/15/2015 8:54 AM, Daniel P. Berrange wrote:

On Fri, May 15, 2015 at 02:45:06PM +0100, John Garbutt wrote:

On 15 May 2015 at 13:28, Daniel P. Berrange berra...@redhat.com wrote:

On Fri, May 15, 2015 at 11:51:22AM +0100, Daniel P. Berrange wrote:

On Thu, May 14, 2015 at 02:23:25PM -0500, Matt Riedemann wrote:

There are some workarounds in the code [3] I'd like to see removed by
bumping the minimum required version.

BTW, the code you quote here:

http://git.openstack.org/cgit/openstack/nova/tree/nova/virt/libvirt/host.py?id=2015.1.0#n754

So what's really needed is a dep on libvirt-python = 1.2.0, not libvirt.

Does this mean we can drop the above [3] code?
https://github.com/openstack/requirements/blob/master/global-requirements.txt#L56

Hmm, I didn't know it was listed in global-requirements.txt - I only
checked the requirements.txt and test-requirements.txt in Nova itself
which does not list libvirt-python.

Previously test-requirements.txt did have it, but we dropped it, since
the unit tests now exclusively use fakelibvirt.

To answer your question though, if global-requirements.txt is enforcing
that we have libvirt-python 1.2.5, then we can drop that particular
workaround.

Regards,
Daniel

Right, I plan to add libvirt-python back to nova's test-requirements.txt
to remove the workaround in host.py.

We originally removed libvirt-python from nova's test-requirements.txt
because it was mucking up nova unit tests which were at the time doing a
conditional import of libvirt-python so if you had that, the unit tests
ran against whatever version you got and if it didn't import, you ran
against fakelibvirt. We should be using fakelibvirt everywhere in the
unit tests now so we can add libvirt-python back to
test-requirements.txt as an indication of the minimum required version
of an optional dependency (required if you're using libvirt).

Thanks,

Matt Riedemann

[openstack-dev] [Fuel] python-fuelclient 6.1.0 is released

2015-05-15 Thread Roman Prykhodchenko

Hi folks!

I’m glad to announce that the first independent release of Fuel Client was 
published to PyPi: https://pypi.python.org/pypi/python-fuelclient 
https://pypi.python.org/pypi/python-fuelclient
You can either download it from the web page or install with pip install 
python-fuelclient.

What’s new:

 - Fuel client is now able to run most of it’s features remotely from Fuel’s 
master node.
 - Old CLI was deprecated, new fuel2 utility is a preview of the new Fuel 
client which will be available in the next major release
 - Versioning scheme of the Fuel Client is not tightly bound to Fuel’s version 
scheme anymore.
 - Other improvements and bug-fixes



signature.asc
Description: Message signed with OpenPGP using GPGMail
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [security] Consensus on security guidance license

2015-05-15 Thread Clark, Robert Graham

Agree

Sent from my iPhone

On 15 May 2015, at 10:17, Rob Fletcher 
rfletch@gmail.commailto:rfletch@gmail.com wrote:

sgtm

On Fri, May 15, 2015 at 10:04 AM, Paul McMillan 
p...@mcmillan.wsmailto:p...@mcmillan.ws wrote:

Works for me.

-Paul

On May 15, 2015 10:03 AM, Murphy, Grant 
grant.mur...@hp.commailto:grant.mur...@hp.com wrote:
At the last OSSG mid-cycle a number of us put together some security guidelines 
and best practices aimed at developers. We are now attempting to migrate this 
content to the https://security.openstack.org page. However we need to come to 
agreement on how to license this content.  I’m proposing that we move from an 
Apache license to CC-BY as per Jeremy’s comments in this review 
https://review.openstack.org/#/c/181123/.  Do any of the original authors 
(listed below) have an objection to this?

Authors:
  Dave Belcher (HP)
  Doug Chivers (HP)
  Lucas Fisher (Was Nebula)
  Grant Murphy (HP)
  Rob Clarke (HP)
  Travis McPeak (HP)
  Tim Kelsey (HP)
  Eric Brown (VMWare)
  Brant Knudson (IBM)
  Paul McMillan (Was Nebula)
  Rob Fletcher (Uber)
  Nathaniel Dillon (HP)
  Jamie Finnigan (HP)

- Grant


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [nova][all] Architecture Diagrams in ascii art?

On Fri, May 15, 2015 at 2:27 PM, Joe Gordon joe.gord...@gmail.com wrote:

On Thu, May 14, 2015 at 3:52 AM, John Garbutt j...@johngarbutt.com
wrote:

On 12 May 2015 at 20:33, Sean Dague s...@dague.net wrote:
On 05/12/2015 01:12 PM, Jeremy Stanley wrote:
On 2015-05-12 10:04:11 -0700 (-0700), Clint Byrum wrote:
It's a nice up side. However, as others have pointed out, it's only
capable of displaying the most basic pieces of the architecture.

For higher level views with more components, I don't think ASCII art
can provide enough bandwidth to help as much as a vector diagram.

Of course, simply a reminder that just because you have one or two
complex diagram callouts in a document doesn't mean it's necessary
to also go back and replace your simpler ASCII art diagrams with
unintelligible (without rendering) SVG or Postscript or whatever.
Doing so pointlessly alienates at least some fraction of readers.

Sure, it's all about trade offs.

But I believe that statement implicitly assumes that ascii art diagrams
do not alienate some fraction of readers. And I think that's a bad
assumption.

If we all feel alienated every time anyone does anything that's not
exactly the way we would have done it, it's time to give up and pack it
in. :) This thread specifically mentioned source based image formats
that were internationally adopted open standards (w3c SVG, ISO ODG) that
have free software editors that exist in Windows, Mac, and Linux
(Inkscape and Open/LibreOffice).

Some great points make here.

Lets try decide something, and move forward here.

Key requirements seem to be:
* we need something that gives us readable diagrams
* if its not easy to edit, it will go stale
* ideally needs to be source based, so it lives happily inside git
* needs to integrate into our sphinx pipeline
* ideally have an opensource editor for that format (import and
export), for most platforms

ascii art fails on many of these, but its always a trade off.

Possible way forward:
* lets avoid merging large hard to edit bitmap style images
* nova-core reviewers can apply their judgement on merging source based
formats
* however it *must* render correctly in the generated html (see result
of docs CI job)

Trying out SVG, and possibly blockdiag, seem like the front runners.
I don't think we will get consensus without trying them, so lets do that.

Will that approach work?

Sounds like a great plan.

After further investigation in blockdiag, is useless for moderately complex
diagrams.

Here is my attempt at graphing nova [0], but due to a blockdiag bug from
2013, [1] it is impossible to clearly read. For example, in the diagram
there is not supposed to be any arrow between the conductor and
cinder/glance/neutron. I looked into dia, and while it has plenty of
diagram shapes it doesn't have a good template for software architecture,
but maybe there is a way to make dia work. And that just leaves SVG
graphics, after spending an hour or two playing around with Inkscape and
it looks promising (although the learning curve is pretty steep). Here is
my first attempt in Inkscape [2].

[0]
http://interactive.blockdiag.com/?compression=deflatesrc=eJx9UMtOAzEMvOcrrL0vPwCtVHYryoG2EvSEOHiTtI0axavEFQK0_47dB1oOkEuSmbE9ni6SPbiAO_gyAJviM7yWPfYeJlChZcrV2-2VqafQxOAT62u2fhwTC8rhk9KIkWOMfuBOC0NyPtdLf-RMqX6ImKwXWbN6Wm9e5v9ppNcu07EXi_puVsv2LL-U6jAd8wsSTByJV-QgtibQU-aMgcft4G-RcBE7HzWH9h7QWl9KpaMKf0SNxxGzdyfkElgMSVcCS5GyFnYR7aESxCFjh8WPwt1Gerd7zHxzJc9J_2wiW8r93Czm7cnOYAZjhm9d4H0M
[1] https://bitbucket.org/blockdiag/blockdiag/issue/45/arrows-collisions
[2] https://i.imgur.com/TXwsRoB.png

Thanks,
John

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe:
openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [neutron] Neutron API rate limiting

2015-05-15 Thread Gal Sagie

Hello Rick,

First, we jumped into a different discussion as i was pointed out by Carl
so lets continue this on another thread (Sorry everyone)
But to your question:

There are two topics here, first on a Neutron API level there is no way to
define rate-limit for ports (at least that i know of).
There are two efforts trying to tackle this:

1. QoS API work that is being done [1]
2. New security rule classes spec i have written about in the previous
email [2]

What you are describing is the implementation aspects of the QoS effort,
and i agree with you
there are challenges (for example we are aware of HTB being problematic on
high thoughtputs)

What i was describing in [2] is different, maybe the name rate-limit is
wrong here and what we are doing is more of
a brute force prevention .
We are trying to solve common scenarios for east-west security attack
vectors, for example a common vector is a compromised
VM trying to port scan the network.
By matching these port-scanning with rate-limit security rules we can
detect this and either block
that traffic or alert the admin/user.
(An example of a rate-limit rule would be a VM pinging the same IP with
different ports on a short period of time)

For that there is the security API extension that is needed and the
reference implementation, and we should discuss
them both on the session [3] and how to support extending the API furthur
for other user cases/vendors, hope to see you there!

[1] https://review.openstack.org/#/c/88599/
[2]  https://review.openstack.org/#/c/151247/
[3] https://etherpad.openstack.org/p/YVR-neutron-sg-fwaas-future-direction

On Fri, May 15, 2015 at 10:01 PM, Rick Jones rick.jon...@hp.com wrote:

 On May 14, 2015 9:26 PM, Gal Sagie gal.sa...@gmail.commailto:
 gal.sa...@gmail.com wrote:
 Hello Ryan,

 We have proposed a spec to liberty to add rate limit functionality to
 security groups [1].
 We see two big use cases for it, one as you mentioned is DDoS for
 east-west and another
 is brute force prevention (for example port scanning).

 We are re-writing the spec as an extension to the current API, we also
 have a proposal
 to enhance the Security Group / FWaaS implementation in order to make it
 easily extendible by such
 new classes of security rules.

 We are planning to discuss all of that in the SG/FWaaS future directions
 session [2].
 I or Lionel will update you as soon as we have the fixed spec for review,
 and feel free to come to the discussion
 as we are more then welcoming everyone to help this effort.

 Gal.

 [1] https://review.openstack.org/#/c/151247/
 [2]
 https://etherpad.openstack.org/p/YVR-neutron-sg-fwaas-future-direction


 Isn't there already described a way to rate-limit instances (overall) by
 putting qdiscs onto their tap devices?

 Having looked only briefly at the spec, I would say you want to have the
 option to MARK that traffic which is EDN enabled the rate limiting might
 have otherwise dropped.

 The extant mechanism I mentioned uses HTB in one direction (instance
 inbound/tap outbound) and a policing filter in the other.  I've used it (as
 a mostly end user) and have noticed that as described, one can introduce
 non-trivial bufferbloat inbound to the instance.

 And I've always wished (as in if wishes were horses) that the instance
 outbound throttle were actually implemented in a way where it becomes
 immediately apparent to the instance by causing the tx queue in the
 instance to build-up.  That wouldn't be something on a tap device though.

 Does there need to be both a packet and bit rate limit?  I've some
 experience with bit rate limits and have seen otherwise rather throttled
 (bitrate) instances cause non-trivial problems with a network node.

 rick jones


 __
 OpenStack Development Mailing List (not for usage questions)
 Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev




-- 
Best Regards ,

The G.
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] OpenStack 2015.1.0 for Debian Sid and Jessie

2015-05-15 Thread Thomas Goirand



On 05/15/2015 10:37 AM, neil.jer...@metaswitch.com wrote:

Out of interest, have you done this by re-releasing the Ubuntu packaging? Or 
have you taken an independent approach?

Regards,
 Neil


It's been since Folsom that I've released packages on my own in Debian. 
Absolutely zero packaging work was imported from Ubuntu to Debian in 
this release also. In fact, it's the opposite which (often) happens: the 
last release, Juno, in Ubuntu, was using nearly 100% of my work for 
packaging the dependencies (including Oslo libraries and the 
python-*client packages). This last Kilo release is different because I 
couldn't upload to Debian during the freeze of Jessie, so Canonical had 
to work on Oslo packages of their own. This shows especially on the 
naming of the oslo packages, with a dash in Ubuntu (which seems to be a 
mistake), and a dot in Debian (which is compatible with what the 
egg-info declares).


By the way, the list of packages which I maintain is available at [1], 
and there you can see the difference of version numbers between Debian 
and Ubuntu. When you see the same version in both Debian and Ubuntu, it 
means ubuntu has synced from Debian, or in other words, imported the 
work I've done in Debian.


On 05/15/2015 03:50 PM, Ihar Hrachyshka wrote:
 Are there any attempts to avoid duplication of efforts? I would expect
 Ubuntu to reuse and extend what is in their upstream distro - Debian.

 Ihar

It's a decision from upper (or even *very* upper, shall I say...) 
management at Canonical that there's no collaboration between Debian and 
Ubuntu on the core packages. Maybe this may change in the future if the 
decision is reversed (I'm opened for it to happen...).


However, there's been some attempts to work more on the dependency 
packages together, but mostly, these attempts failed (partly due to the 
fact that Canonical insists on using BZR as a VCS). I've seen some bugs 
opened with patches by Ubuntu people to lessen the differences for these 
packages which is a good thing.


Let's hope things get better some time...

Cheers,

Thomas Goirand (zigo)

P.S: If you try deploying using Debian, make sure you're using 
python-pysaml2 = 2.4.0 which I uploaded yesterday, otherwise Keystone 
will be broken.


[1] 
https://qa.debian.org/developer.php?login=openstack-de...@lists.alioth.debian.org


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [surge] Introducing Surge - rapid deploy/scale stream processing systems on OpenStack

2015-05-15 Thread Jay Pipes

I think the surge is in the number of projects that Cisco seems to be 
pushing out recently ;)


On 05/15/2015 01:13 PM, Debojyoti Dutta wrote:

Hi,

It gives me a great pleasure to introduce Surge - a system to rapidly
deploy and scale a stream processing system on OpenStack. It leverages
Vagrant and Ansible, and supports both OpenStack as well as the local
mode (with VirtualBox).

https://github.com/CiscoSystems/surge

Hope to see a lot of pull requests and comments.

thx
-Debo~


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [nova][cinder] can we deprecate the volume CLIs in novaclient?

2015-05-15 Thread Everett Toews

On May 15, 2015, at 10:28 AM, John Griffith 
john.griffi...@gmail.commailto:john.griffi...@gmail.com wrote:



On Thu, May 14, 2015 at 8:29 PM, Matt Riedemann 
mrie...@linux.vnet.ibm.commailto:mrie...@linux.vnet.ibm.com wrote:
This came up while talking about bug 1454369 [1].  This also came up at one 
point in kilo when we found out the volume CLIs in novaclient didn't work at 
one point and we broke the cells devstack exercises job because of it.

python-novaclient uses cinder API to handle the volume CLI rather than going to 
the nova volume API.  There are issues with this because novaclient needs a 
certain endpoint/service_type setup in the service catalog to support cinder 
v1/v2 APIs (whatever devstack sets up today).  novaclient defaults to volume 
(v1) and if you disable that in cinder then novaclient doesn't work because 
it's not using volumev2.

So like anyone might ask, why doesn't novaclient talk to nova volume APIs to do 
volume thingies and the answer is because the nova volume API doesn't handle 
all of the volume thingies like snapshots and volume types.

So I got to to thinking, why the hell are we still supporting volume operations 
via novaclient anyway?  Isn't that cinderclient's job?  Or 
python-openstackclient's job?  Can't we deprecate the volume CLIs in novaclient 
and tell people to use cinderclient instead since it now has version discovery 
[2] so that problem would be handled for us.

Since we have nova volume APIs maybe we can't remove the volume CLIs in 
novaclient, but could they be limited to just operations that the nova API 
supports and then we make novaclient talk to nova volume APIs rather than 
cinder APIs (because the nova API will talk to cinderclient which again has the 
version discovery done for us).

Or assuming we could deprecate the volume CLIs in novaclient, what would the 
timeline on deprecation be since it's not a server project with a 6 month 
release cycle?  I'm assuming we'd still have 6-12 months deprecation on a 
client like this because of all of the tooling potentially written around it.

[1] https://bugs.launchpad.net/python-novaclient/+bug/1454369
[2] https://review.openstack.org/#/c/145613/

I can't speak for the nova folks, however i do think removing the volume calls 
from novaclient seems ok.  It was always sort of left for compat I think, and 
not sure any of us really thought about just removing it.  At this point it 
probably just introduces confusion and as you're running into problems.

Seems like a good plan, and somewhat less confusing.  On a side note, might be 
some other *things* in novaclient that we could look at as well, particularly 
around networking.  

FWIW, this is already underway in jclouds-land. After a lengthy deprecation 
period (still ongoing actually), we’ll be removing the Nova volume calls but 
obviously keeping the volume attachment stuff.

Both the Nova and Cinder calls have coexisted for over a year with 
documentation pointing from Nova to Cinder. The deprecation annotations handle 
emitting warnings for the deprecated calls to increase visibility.

Everett

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [nova][cinder] can we deprecate the volume CLIs in novaclient?

2015-05-15 Thread Jay Pipes


On 05/14/2015 10:29 PM, Matt Riedemann wrote:

This came up while talking about bug 1454369 [1].  This also came up at
one point in kilo when we found out the volume CLIs in novaclient didn't
work at one point and we broke the cells devstack exercises job because
of it.

python-novaclient uses cinder API to handle the volume CLI rather than
going to the nova volume API.  There are issues with this because
novaclient needs a certain endpoint/service_type setup in the service
catalog to support cinder v1/v2 APIs (whatever devstack sets up today).
  novaclient defaults to volume (v1) and if you disable that in cinder
then novaclient doesn't work because it's not using volumev2.

So like anyone might ask, why doesn't novaclient talk to nova volume
APIs to do volume thingies and the answer is because the nova volume API
doesn't handle all of the volume thingies like snapshots and volume types.

So I got to to thinking, why the hell are we still supporting volume
operations via novaclient anyway?  Isn't that cinderclient's job?  Or
python-openstackclient's job?  Can't we deprecate the volume CLIs in
novaclient and tell people to use cinderclient instead since it now has
version discovery [2] so that problem would be handled for us.

Since we have nova volume APIs maybe we can't remove the volume CLIs in
novaclient, but could they be limited to just operations that the nova
API supports and then we make novaclient talk to nova volume APIs rather
than cinder APIs (because the nova API will talk to cinderclient which
again has the version discovery done for us).

Or assuming we could deprecate the volume CLIs in novaclient, what would
the timeline on deprecation be since it's not a server project with a 6
month release cycle?  I'm assuming we'd still have 6-12 months
deprecation on a client like this because of all of the tooling
potentially written around it.


I wrote this on the bug report, too, but I believe deprecating the 
volume commands from novaclient is the appropriate solution. I think a 
single 6 month deprecation cycle is appropriate.


Best,
-jay

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [nova][cinder] can we deprecate the volume CLIs in novaclient?

2015-05-15 Thread Sean Dague

On 05/15/2015 12:28 PM, Everett Toews wrote:
 On May 15, 2015, at 10:28 AM, John Griffith john.griffi...@gmail.com
 mailto:john.griffi...@gmail.com wrote:
 


 On Thu, May 14, 2015 at 8:29 PM, Matt Riedemann
 mrie...@linux.vnet.ibm.com mailto:mrie...@linux.vnet.ibm.com wrote:

 This came up while talking about bug 1454369 [1].  This also came
 up at one point in kilo when we found out the volume CLIs in
 novaclient didn't work at one point and we broke the cells
 devstack exercises job because of it.

 python-novaclient uses cinder API to handle the volume CLI rather
 than going to the nova volume API.  There are issues with this
 because novaclient needs a certain endpoint/service_type setup in
 the service catalog to support cinder v1/v2 APIs (whatever
 devstack sets up today).  novaclient defaults to volume (v1) and
 if you disable that in cinder then novaclient doesn't work because
 it's not using volumev2.

 So like anyone might ask, why doesn't novaclient talk to nova
 volume APIs to do volume thingies and the answer is because the
 nova volume API doesn't handle all of the volume thingies like
 snapshots and volume types.

 So I got to to thinking, why the hell are we still supporting
 volume operations via novaclient anyway?  Isn't that
 cinderclient's job?  Or python-openstackclient's job?  Can't we
 deprecate the volume CLIs in novaclient and tell people to use
 cinderclient instead since it now has version discovery [2] so
 that problem would be handled for us.

 Since we have nova volume APIs maybe we can't remove the volume
 CLIs in novaclient, but could they be limited to just operations
 that the nova API supports and then we make novaclient talk to
 nova volume APIs rather than cinder APIs (because the nova API
 will talk to cinderclient which again has the version discovery
 done for us).

 Or assuming we could deprecate the volume CLIs in novaclient, what
 would the timeline on deprecation be since it's not a server
 project with a 6 month release cycle?  I'm assuming we'd still
 have 6-12 months deprecation on a client like this because of all
 of the tooling potentially written around it.

 [1] https://bugs.launchpad.net/python-novaclient/+bug/1454369
 [2] https://review.openstack.org/#/c/145613/

 I can't speak for the nova folks, however i do think removing the
 volume calls from novaclient seems ok.  It was always sort of left
 for compat I think, and not sure any of us really thought about just
 removing it.  At this point it probably just introduces confusion and
 as you're running into problems.

 Seems like a good plan, and somewhat less confusing.  On a side note,
 might be some other *things* in novaclient that we could look at as
 well, particularly around networking.  
 
 FWIW, this is already underway in jclouds-land. After a lengthy
 deprecation period (still ongoing actually), we’ll be removing the Nova
 volume calls but obviously keeping the volume attachment stuff. 
 
 Both the Nova and Cinder calls have coexisted for over a year with
 documentation pointing from Nova to Cinder. The deprecation annotations
 handle emitting warnings for the deprecated calls to increase visibility.

Everett, this is actually a different thing.

nova volume   does not talk to Nova's volume proxy, it goes
straight to cinder through the service catalog.

Deprecating this part of nova client is probably fine, but it should
have a lengthy deprecation cycle, as it's been like this for a very long
time. It feels like it won't go away before openstack client starts
taking hold anyway.

I think this raises a more important issue of Service Catalog
Standarization. The reason we're in a bind here has as much to do with
the fact that service catalog content isn't standardized for OpenStack
services. If so, having another cinder implementation in novaclient
wouldn't be such a bad thing, and not having to switch cli commands is
pretty handy (all hail our future osc overlords).

Fortunately, we're going to be talking about just this kind of problem
at Summit -
http://libertydesignsummit.sched.org/event/194b2589eca19956cb88ada45e985e29

-Sean

-- 
Sean Dague
http://dague.net

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [neutron][lbaas]LBaaSv2 movies / links

2015-05-15 Thread Samuel Bercovici

Here is an example: https://filepile.radware.com/files/5ba-e02-c39


From: Samuel Bercovici [mailto:samu...@radware.com]
Sent: Friday, May 15, 2015 7:06 PM
To: Vijay Venkatachalam; OpenStack Development Mailing List (not for usage 
questions); Evgeny Fedoruk; Adam Harwell; Kyle Mestery; Brandon Logan; Johnson, 
Michael (HP Cloud - Corvallis); Doug Wiegley
Subject: Re: [openstack-dev] [neutron][lbaas]LBaaSv2 movies / links

Was thinking 1-2 minutes. Tomorrow would be fine.

From: Vijay Venkatachalam [mailto:vijay.venkatacha...@citrix.com]
Sent: Friday, May 15, 2015 6:03 PM
To: OpenStack Development Mailing List (not for usage questions); Evgeny 
Fedoruk; Adam Harwell; Kyle Mestery; Brandon Logan; Johnson, Michael (HP Cloud 
- Corvallis); Doug Wiegley; Samuel Bercovici
Subject: Re: [openstack-dev] [neutron][lbaas]LBaaSv2 movies / links

Hi Sam,
Thanks for probing. How many seconds/mins you have thought per vendor? By when 
do you need this? Will tomorrow work fine?


Thanks,
Vijay V.

Sent from Surface

From: Samuel Bercovicimailto:samu...@radware.com
Sent: ‎Friday‎, ‎15‎ ‎May‎ ‎2015 ‎00‎:‎30
To: OpenStack Development Mailing List (not for usage 
questions)mailto:openstack-dev@lists.openstack.org, Evgeny 
Fedorukmailto:evge...@radware.com, Adam 
Harwellmailto:adam.harw...@rackspace.com, Kyle 
Mesterymailto:mest...@mestery.com, Brandon 
Loganmailto:brandon.lo...@rackspace.com, Johnson, Michael (HP Cloud - 
Corvallis)mailto:michael.d.john...@hp.com, Doug 
Wiegleymailto:do...@a10networks.com, Samuel 
Bercovicimailto:samu...@radware.com

Hi Everyone,

As you may be aware, we have a speaking slot on the Vancouver summit to discuss 
“LBaaS v2, Kilo and beyond” on Monday 
https://openstacksummitmay2015vancouver.sched.org/event/3f1e9e24f36238152749afea9c21a264#.VVTwCPmqqko

We are considering to show vendors demos or list/link such movies.
Please let us know if you have such so we can list/include in the talk.

Regards,
-Sam.

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [nova][heat] sqlalchemy-migrate tool to alembic

2015-05-15 Thread Mike Bayer




On 5/14/15 10:34 PM, Angus Salkeld wrote:


Thanks. Would you suggest we hold off moving to alembic (in Heat) 
until you have this ironed out? I just want to make sure we

don't do this prematurely.


If you need to support running an Alembic-version of Heat against an 
existing database in mid-migration state that has a SQLAlchemy-Migrate 
table in place, you can either do the oslo.db thing, or I can implement 
the feature in Alembic to allow an upgrade for a system that used 
Migrate.If you can survey how big a task moving your migration files 
will be, I can implement the Alembic feature by sometime after the 
summit (if not during the summit).











-Angus






-Angus





__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe:
openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev




__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe:openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
mailto:openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe:
openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev




__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] [tc] A way for Operators/Users to submit feature requests

On 14 May 2015 at 21:55, Boris Pavlovic bo...@pavlovic.me wrote:
 Robert,

 So I think we should explicitly leave room for experimentation and
 divergence, but also encourage a single common path - don't be
 different to be different, be difference because it is important in
 this specific case.


 First of all feature request are the same process as specs (in other
 projects)
 Difference is what we are expecting to get in spec and feature request (and
 auditory)

 By the way feature request in Rally were introduced far far before backlogs
 in other Keystone and Nova.
 It strange from me that those projects are reinventing working mechanism
 from other project=( and not just use it.

I am totally open standardising. Nova added backlogs after feedback
after the cross project session I ran at the last summit. It looked at
standardising the spec process between the different projects using
specs:
https://etherpad.openstack.org/p/kilo-crossproject-specs

The consensus in that session was for projects to adopt the backlog
approach, roughly similar to what keystone was using.

My original email to the operators linked to this web page:
http://specs.openstack.org/openstack/nova-specs/specs/backlog/index.html

It explains backlog specs as:

If you have a problem that needs solving, but you are not currently
planning on implementing it, this is the place we can store your
ideas.
These specifications have the problem description completed, but all
other sections are optional.


I like how it puts queued developer specs and operator requests
through the same process, to keep things simple.

Rally's feature requests and Nova's backlog look very similar to me
(except the name)? Is there something big I am missing here?

Thanks,
John

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [nova][heat] sqlalchemy-migrate tool to alembic

2015-05-15 Thread Henry Gessau

On Fri, May 15, 2015, Mike Bayer mba...@redhat.com wrote:
 
 
 On 5/15/15 9:31 AM, Doug Hellmann wrote:
 This seems more complicated than needed. If we just stop writing the
 sqlalchemy-migrate scripts and don't change them, then for 1 cycle we
 have to run both sets of migrations and after that we can just run
 alembic.
 Then we have a forever-in-perpetuity dependency on SQLAlchemy-Migrate 
 which must be maintained forever for to maintain compatibility with all 
 new SQLAlchemy, oslo.db, etc. releases, despite it never being used for 
 anythine new, because it will be impossible to install an Openstack 
 application without running through the first set of migrate scripts first.
 
 The SQLAlchemy-Migrate dependency must be dropped and the project has to 
 be EOL'ed at some point.   Leaving it in is definitely the more 
 complicated alternative.

Neutron has used alembic since its birth (Folsom release), but during the Juno
cycle we consolidated all the Folsom to Havana migrations into one
havana_initial migration. That is now the start migration for Neutron.

Nova should be able to do something similar in a cycle or two: setting
liberty_initial as the start migration with no legacy before it.


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Openstack-operators] [nova] Can we bump MIN_LIBVIRT_VERSION to 1.2.2 in Liberty?

On 15 May 2015 at 12:28, Daniel P. Berrange berra...@redhat.com wrote:
 One other thing I should have mentioned is that we don't actually have
 one single minimum libvirt version. We actually have a couple of different
 minimum versions based on either the architecture or the hypervisor.

 For example, the Parallels hypervisor support was set to 1.2.12 and
 the S/390 support was /supposed/ to have been set to 1.0.4, but I see
 the devs failed to actally submit that change so I'll be doing that
 shortly.

 I think there is a credible argument that we increase the min required
 libvirt for LXC, because it requires a pretty new libvirt and kernel
 to provide any sensible level of security (ie user namespaces). We're
 rather negligent at the moment to let users deploy LXC with older
 versions as it is trivial for tenants to escape isolation.

 So the current MIN_LIBVIRT_VERSION is really talking about x86 + KVM
 combination.

 We do a really bad job of making this clear anywhere in our docs for
 Nova AFAIK. Likewise we don't make any distinction in our docs about
 the version we have tested with, vs the versions we are capable of
 running with. This is all critical info to people deploying, so they
 have guidance as to how much testing of their specific platform they
 should do at deployment time.

+1

We need to address this as part of the feature classification stuff.

We should certainly document what we are testing for all these combinations.

As I mentioned before, the same goes for other drivers.

We should be clear we don't current test against older versions of
Glance/Cinder/Neutron, for example.

Thanks,
John

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Openstack-operators] [nova] Can we bump MIN_LIBVIRT_VERSION to 1.2.2 in Liberty?

On Fri, May 15, 2015 at 05:27:35PM +0100, John Garbutt wrote:
 On 15 May 2015 at 11:51, Daniel P. Berrange berra...@redhat.com wrote:
  On Thu, May 14, 2015 at 02:23:25PM -0500, Matt Riedemann wrote:
  The minimum required version of libvirt in the driver is 0.9.11 still [1].
  We've been gating against 1.2.2 in Ubuntu Trusty 14.04 since Juno.
 
  The libvirt distro support matrix is here: [2]
 
  Can we safely assume the people aren't going to be running Libvirt compute
  nodes on RHEL  7.1 or Ubuntu Precise?
 
  I don't really think so - at the very least Fedora 20 and RHEL 7.0 are still
  actively supported platforms by their vendors, which both have older libvirt
  versions (1.1.3 and 1.1.1 respectively).
 
  I'm not sure whether SUSE team consider any of the 12.x versions to still be
  actively supported platforms or not, likewise which 13.x versions are under
  active support.
 
 As I understand it, RHEL 5.0 is supported until March 31, 2017.
 https://access.redhat.com/support/policy/updates/errata
 
 Where should we draw the line? Its a tricky question.

No one is seriously deploying KVM on RHEL-5 any more though, nor has done for
quite a while, as RHEL-5 was very immature for KVM.  RHEL-6.x though is a solid
platform for KVM that is still a widely deployed platform for virtualization,
even for brand new deployments.

The cut off line is difficult to set in stone - it takes a bit of analysis
to understand what the real world popularity of the platform is..

  Regarding RHEL, I think this is a safe bet because in Kilo nova dropped
  python 2.6 support and RHEL  6 doesn't have py26 so you'd be in trouble
  running kilo+ nova on RHEL 6.x anyway.
 
  There are add-on repos for RHEL-6 and RHEL-7 that provide newer python
  versions so (py27, various py3x), so it is not unreasonable for people
  to consider sticking with RHEL-6 if that is what their existing deployment
  is based on. Certainly new deployments though I'd expect to be RHEL-7 based.
 
 It seems like we are OK with deprecating support for libvirt  0.10.2 in M 
 then?

Yep.


  In this case, I just don't see compelling benefits to Nova libvirt maint
  to justify increasing the minimum version to the level you suggest, and
  it has a clear negative impact on our users which they will not be able
  to easily deal with. They will be left with 3 options all of which are
  unsatisfactory
 
   - Upgrade from RHEL-6 to RHEL-7 - a major undertaking for most 
  organizations
   - Upgrade libvirt on RHEL-6 - they essentially take on the support burden
 for the hypervisor themselves, loosing support from the vendor
   - Re-add the code we remove from Nova - we've given users a maint burden,
 to rid ourselves of code that was posing no real maint burden on 
  ourselves.
 
 Thats assuming vendors do not backport libvirt and QEMU and support that.

In early updates we do rebase to newer libvirt  QEMU in RHEL, but RHEL-6
has past the point in its lifecycle where we do that, so the versions
are effectively fixed now for remainder of RHEL-6.x.

  As a more general point, I think we are lacking clear guidance on our
  policies around hypervisor platform support and thus have difficulty
  in deciding when it is reasonable for us to drop support for platforms.
 
 +1
 
 Lets try to fixing this in Liberty.
 
 I see this as part of feature classification:
 http://libertydesignsummit.sched.org/event/7ee9be7e3b005880706a914f44b296ed
 
 Honestly, I want us to call out the exact combinations we test. But
 that in logs, or in the release notes , or docs, or some combination
 of all of those.
 
 Partly so we are honest about what we know works, partly so folks
 setup up and help us test more combinations users care about.
 
 This should really be true for everything, VMware, XenServer, Hyper-V
 and others.

Yes indeed. We need to give our users keep information about what is
tested, so that they can undertake their own testing (with tempest
or equiv) when they deploy on platforms that don't 100% match what
upstream tested, or can at least ask their vendors to confirm they
have tested on their behalf.

  The hypervisor platform is very different. While OpenStack does achieve
  some level of testing coverage of the hypervisor platform version used
  in the gate, this testing is inconsequential compared to the level of
  testing that vendors put into their hypervisor platforms.
 
 That depends on the scope right...
 
 Sure, we don't really test the data plane, that is left the
 hypervisor vendor.
 
 But we do heavily test the control plane, and thats useful.

I can't speak for other vendors, but when we test virtualization
in RHEL, we specifically test the combination of libvirt + QEMU + kernel,
so changing any part of that invalidates testing and certification
to some degrees. There are also sometimes fixes to libvirt's RHEL
code to deal with RHEL specific problems, so if deploying a new
upstream libvirt users would loose some of those workarounds.

  Even if users

Re: [openstack-dev] [neutron][lbaas]LBaaSv2 movies / links

2015-05-15 Thread Doug Wiegley

I would say: 1-2 minutes, no audio, mp4, must highlight lbaas v2 or v2/v1. 
Anyone is welcome, even encouraged, to submit, likely up until about Sunday I’m 
guessing.

Thanks,
doug


 On May 15, 2015, at 10:05 AM, Samuel Bercovici samu...@radware.com wrote:
 
 Was thinking 1-2 minutes. Tomorrow would be fine.
  
 From: Vijay Venkatachalam [mailto:vijay.venkatacha...@citrix.com] 
 Sent: Friday, May 15, 2015 6:03 PM
 To: OpenStack Development Mailing List (not for usage questions); Evgeny 
 Fedoruk; Adam Harwell; Kyle Mestery; Brandon Logan; Johnson, Michael (HP 
 Cloud - Corvallis); Doug Wiegley; Samuel Bercovici
 Subject: Re: [openstack-dev] [neutron][lbaas]LBaaSv2 movies / links
  
 Hi Sam,
 Thanks for probing. How many seconds/mins you have thought per vendor? By 
 when do you need this? Will tomorrow work fine?
  
  
 Thanks,
 Vijay V.
  
 Sent from Surface
  
 From: Samuel Bercovici mailto:samu...@radware.com
 Sent: ‎Friday‎, ‎15‎ ‎May‎ ‎2015 ‎00‎:‎30
 To: OpenStack Development Mailing List (not for usage questions) 
 mailto:openstack-dev@lists.openstack.org, Evgeny Fedoruk 
 mailto:evge...@radware.com, Adam Harwell 
 mailto:adam.harw...@rackspace.com, Kyle Mestery 
 mailto:mest...@mestery.com, Brandon Logan 
 mailto:brandon.lo...@rackspace.com, Johnson, Michael (HP Cloud - Corvallis) 
 mailto:michael.d.john...@hp.com, Doug Wiegley 
 mailto:do...@a10networks.com, Samuel Bercovici mailto:samu...@radware.com
  
 Hi Everyone,
  
 As you may be aware, we have a speaking slot on the Vancouver summit to 
 discuss “LBaaS v2, Kilo and beyond” on 
 Mondayhttps://openstacksummitmay2015vancouver.sched.org/event/3f1e9e24f36238152749afea9c21a264#.VVTwCPmqqko
  
 https://openstacksummitmay2015vancouver.sched.org/event/3f1e9e24f36238152749afea9c21a264#.VVTwCPmqqko
  
 We are considering to show vendors demos or list/link such movies.
 Please let us know if you have such so we can list/include in the talk.
  
 Regards,
 -Sam.
  
 __
 OpenStack Development Mailing List (not for usage questions)
 Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [security] Consensus on security guidance license

2015-05-15 Thread Paul McMillan

Works for me.

-Paul
On May 15, 2015 10:03 AM, Murphy, Grant grant.mur...@hp.com wrote:

 At the last OSSG mid-cycle a number of us put together some security
 guidelines and best practices aimed at developers. We are now attempting to
 migrate this content to the https://security.openstack.org page. However
 we need to come to agreement on how to license this content.  I’m proposing
 that we move from an Apache license to CC-BY as per Jeremy’s comments in
 this review https://review.openstack.org/#/c/181123/.  Do any of the
 original authors (listed below) have an objection to this?

 Authors:
   Dave Belcher (HP)
   Doug Chivers (HP)
   Lucas Fisher (Was Nebula)
   Grant Murphy (HP)
   Rob Clarke (HP)
   Travis McPeak (HP)
   Tim Kelsey (HP)
   Eric Brown (VMWare)
   Brant Knudson (IBM)
   Paul McMillan (Was Nebula)
   Rob Fletcher (Uber)
   Nathaniel Dillon (HP)
   Jamie Finnigan (HP)

 - Grant

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [security] Consensus on security guidance license

2015-05-15 Thread Murphy, Grant

At the last OSSG mid-cycle a number of us put together some security guidelines 
and best practices aimed at developers. We are now attempting to migrate this 
content to the https://security.openstack.org page. However we need to come to 
agreement on how to license this content.  I’m proposing that we move from an 
Apache license to CC-BY as per Jeremy’s comments in this review 
https://review.openstack.org/#/c/181123/.  Do any of the original authors 
(listed below) have an objection to this?

Authors:
  Dave Belcher (HP)
  Doug Chivers (HP)
  Lucas Fisher (Was Nebula)
  Grant Murphy (HP)
  Rob Clarke (HP)
  Travis McPeak (HP)
  Tim Kelsey (HP)
  Eric Brown (VMWare)
  Brant Knudson (IBM)
  Paul McMillan (Was Nebula)
  Rob Fletcher (Uber)
  Nathaniel Dillon (HP)
  Jamie Finnigan (HP)

- Grant

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [surge] Introducing Surge - rapid deploy/scale stream processing systems on OpenStack

2015-05-15 Thread Debojyoti Dutta

Hi,

It gives me a great pleasure to introduce Surge - a system to rapidly
deploy and scale a stream processing system on OpenStack. It leverages
Vagrant and Ansible, and supports both OpenStack as well as the local mode
(with VirtualBox).

https://github.com/CiscoSystems/surge

Hope to see a lot of pull requests and comments.

thx
-Debo~
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Openstack-operators] [nova] Can we bump MIN_LIBVIRT_VERSION to 1.2.2 in Liberty?

On 15 May 2015 at 17:41, Daniel P. Berrange berra...@redhat.com wrote:
 On Fri, May 15, 2015 at 05:27:35PM +0100, John Garbutt wrote:
 On 15 May 2015 at 11:51, Daniel P. Berrange berra...@redhat.com wrote:
  On Thu, May 14, 2015 at 02:23:25PM -0500, Matt Riedemann wrote:
  The minimum required version of libvirt in the driver is 0.9.11 still [1].
  We've been gating against 1.2.2 in Ubuntu Trusty 14.04 since Juno.
 
  The libvirt distro support matrix is here: [2]
 
  Can we safely assume the people aren't going to be running Libvirt compute
  nodes on RHEL  7.1 or Ubuntu Precise?
 
  I don't really think so - at the very least Fedora 20 and RHEL 7.0 are 
  still
  actively supported platforms by their vendors, which both have older 
  libvirt
  versions (1.1.3 and 1.1.1 respectively).
 
  I'm not sure whether SUSE team consider any of the 12.x versions to still 
  be
  actively supported platforms or not, likewise which 13.x versions are under
  active support.

 As I understand it, RHEL 5.0 is supported until March 31, 2017.
 https://access.redhat.com/support/policy/updates/errata

 Where should we draw the line? Its a tricky question.

 No one is seriously deploying KVM on RHEL-5 any more though, nor has done for
 quite a while, as RHEL-5 was very immature for KVM.  RHEL-6.x though is a 
 solid
 platform for KVM that is still a widely deployed platform for virtualization,
 even for brand new deployments.

 The cut off line is difficult to set in stone - it takes a bit of analysis
 to understand what the real world popularity of the platform is..

Yeah, makes sense.

  In this case, I just don't see compelling benefits to Nova libvirt maint
  to justify increasing the minimum version to the level you suggest, and
  it has a clear negative impact on our users which they will not be able
  to easily deal with. They will be left with 3 options all of which are
  unsatisfactory
 
   - Upgrade from RHEL-6 to RHEL-7 - a major undertaking for most 
  organizations
   - Upgrade libvirt on RHEL-6 - they essentially take on the support burden
 for the hypervisor themselves, loosing support from the vendor
   - Re-add the code we remove from Nova - we've given users a maint burden,
 to rid ourselves of code that was posing no real maint burden on 
  ourselves.

 Thats assuming vendors do not backport libvirt and QEMU and support that.

 In early updates we do rebase to newer libvirt  QEMU in RHEL, but RHEL-6
 has past the point in its lifecycle where we do that, so the versions
 are effectively fixed now for remainder of RHEL-6.x.

Ah, OK.

It seems I also slightly miss-read the options, you covered it there.

  As a more general point, I think we are lacking clear guidance on our
  policies around hypervisor platform support and thus have difficulty
  in deciding when it is reasonable for us to drop support for platforms.

 +1

 Lets try to fixing this in Liberty.

 I see this as part of feature classification:
 http://libertydesignsummit.sched.org/event/7ee9be7e3b005880706a914f44b296ed

 Honestly, I want us to call out the exact combinations we test. But
 that in logs, or in the release notes , or docs, or some combination
 of all of those.

 Partly so we are honest about what we know works, partly so folks
 setup up and help us test more combinations users care about.

 This should really be true for everything, VMware, XenServer, Hyper-V
 and others.

 Yes indeed. We need to give our users keep information about what is
 tested, so that they can undertake their own testing (with tempest
 or equiv) when they deploy on platforms that don't 100% match what
 upstream tested, or can at least ask their vendors to confirm they
 have tested on their behalf.

Cool, we agree there.

  The hypervisor platform is very different. While OpenStack does achieve
  some level of testing coverage of the hypervisor platform version used
  in the gate, this testing is inconsequential compared to the level of
  testing that vendors put into their hypervisor platforms.

 That depends on the scope right...

 Sure, we don't really test the data plane, that is left the
 hypervisor vendor.

 But we do heavily test the control plane, and thats useful.

 I can't speak for other vendors, but when we test virtualization
 in RHEL, we specifically test the combination of libvirt + QEMU + kernel,
 so changing any part of that invalidates testing and certification
 to some degrees. There are also sometimes fixes to libvirt's RHEL
 code to deal with RHEL specific problems, so if deploying a new
 upstream libvirt users would loose some of those workarounds.

Agreed.

I am just trying to be clear I don't expect OpenStack testing to test
the hypervisor data plane.

That feels like something the upstream projects or vendor, has to do themselves.

  Even if users decide they want to upgrade their hypervisor platform to
  a new version provided officially by the vendor, this is not always a
  quick or easy task. Many organizations have

Re: [openstack-dev] [nova][heat] sqlalchemy-migrate tool to alembic

2015-05-15 Thread Mike Bayer




On 5/15/15 9:31 AM, Doug Hellmann wrote:

This seems more complicated than needed. If we just stop writing the
sqlalchemy-migrate scripts and don't change them, then for 1 cycle we
have to run both sets of migrations and after that we can just run
alembic.
Then we have a forever-in-perpetuity dependency on SQLAlchemy-Migrate 
which must be maintained forever for to maintain compatibility with all 
new SQLAlchemy, oslo.db, etc. releases, despite it never being used for 
anythine new, because it will be impossible to install an Openstack 
application without running through the first set of migrate scripts first.


The SQLAlchemy-Migrate dependency must be dropped and the project has to 
be EOL'ed at some point.   Leaving it in is definitely the more 
complicated alternative.



__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] Barbican : Unable to execute the curl command for uploading/retrieving the secrets with the latest Barbican code.

2015-05-15 Thread Asha Seshagiri

Thanks John for providing the pointer to the fix and also thank all
contributors for providing  the quick fix.

Thanks and Regards,
Asha Seshagiri

On Fri, May 15, 2015 at 3:30 PM, John Vrbanac john.vrba...@rackspace.com
wrote:

 Asha,
 We landed the fix in: https://review.openstack.org/#/c/183391/
 Hopefully, that should address the problem you've been seeing.

 Thanks!

 John Vrbanac


 On Thu, 2015-05-14 at 18:14 -0500, Douglas Mendizábal wrote:
  Hi Asha,
 
  The reason we support an Unauthenticated Context in Barbican is purely
  for development purposes.  We recommend that all production Barbican
  deployments use Keystone or an alternative AuthN/AuthZ service in
  front of Barbican.
 
  Setting up a working Keystone environment just to hack on Barbican is
  a steep requirement, which is why we need the Unauthenticated Context
  to work.
 
  - Douglas Mendizabal
 
  On 5/14/15 6:07 PM, Asha Seshagiri wrote:
   Thanks a lot John for your response. But would like to know why do
   would we have to fix the issue for creating the secret for
   unauthenticated context for Barbican since it would be good to have
   access control mechanism  enforced to access secrets , orders and
   other entities from Barbican.
  
   This should be the expected behavior from security perspective .And
   also we are able to access secrets by providing the right token
   from the Identity service (Keystone ). Looking forward for your
   response.
  
   Thanks and Regards, Asha Seshagiri
  
   On Thu, May 14, 2015 at 4:43 PM, John Vrbanac
   john.vrba...@rackspace.com mailto:john.vrba...@rackspace.com
   wrote:
  
   __ Asha, I spent some time looking into this, It looks to be a
   regression that occurred a few days ago when a CR was merged that
   moved us over to oslo_context. I have reported the issue here:
   https://bugs.launchpad.net/barbican/+bug/1455247
  
   I have a couple ideas on how to fix it, so keep your eyes out for
   a CR to resolve the issue.
  
   John Vrbanac
  
  
  
   On Thu, 2015-05-14 at 12:26 -0500, Asha Seshagiri wrote:
   Hi all ,
  
  
   We are able to execute the curl commands on new barbican code
   provided we integrated it with keystone . I ran into this issue
   because I was trying to configure localhost to actual IP on a
   plain barbican server so that I would get the response and
   request objects with the actual IP rather than the local host .
   This configuration was required for seting up HA proxy for
   Barbican.
  
   And then I thought of integrating with the keystone and
   configure Babrican server to https.
  
   *Its a good learning to know that the latest code drop of
   Barbican enforces the authentication mechanism with the keystone
   which would not allow us to execute the curl command without
   providing the token of Identity service (Keystone ) in the
   request unlike the previous Barbican versions*
  
   Please find the curl command request and responses for
   uploading/reteriving the secets on Barbican Server
  
   root@Clientfor-HAProxy barbican]# curl -X POST -H
   'content-type:application/json' -H 'X-Project-Id:12345' \
   -H X-Auth-Token:c9ac81784e1e4e089fccbca19f862be2 -d
   '{payload: my-secret-here,payload_content_type:
   text/plain}' \
   -k https://localhost:9311/v1/secrets
   {secret_ref:
   https://localhost:9311/v1/secrets/02336016-623b-4deb-bca5-caedc0bf0e
  35}[root@Clientfor-HAProxy
  
  
  barbican]#
  
   [root@Clientfor-HAProxy barbican]# curl -H 'Accept:
   application/json' -H 'X-Project-Id:12345' \
   -H X-Auth-Token:c9ac81784e1e4e089fccbca19f862be2 -k
   https://localhost:9311/v1/secrets {secrets: [{status:
   ACTIVE, secret_type: opaque, updated:
   2015-05-14T16:35:44.109536, name: null, algorithm: null,
   created: 2015-05-14T16:35:44.103982, secret_ref:
   https://localhost:9311/v1/secrets/02336016-623b-4deb-bca5-caedc0bf0e
  35,
  
  
  content_types: {default: text/plain}, creator_id:
   cedd848a8a9e410196793c601c03b99a, mode: null, bit_length:
   null, expiration: null}], total: 1}[root@Clientfor-HAProxy
   barbican]#
  
   Thanks and Regards, Asha Seshagiri
  
   On Wed, May 13, 2015 at 4:26 PM, Asha Seshagiri
   asha.seshag...@gmail.com mailto:asha.seshag...@gmail.com
   wrote:
  
   Hi all ,
  
  
  
   When I started  debugging ,we find that default group  is not
   used instead oslo_policy would be used
  
   Please find the logs below :
  
  
   *2015-05-13 15:59:34.393 13210 WARNING oslo_config.cfg [-] Option
   policy_default_rule from group DEFAULT is deprecated. Use
   option policy_default_rule from group oslo_policy.*
   *2015-05-13 15:59:34.394 13210 WARNING oslo_config.cfg [-] Option
   policy_file from group DEFAULT is deprecated. Use option
   policy_file from group oslo_policy.* 2015-05-13 15:59:34.395
   13210 DEBUG oslo_policy.openstack.common.fileutils
   [req-0c6d2db4-bc15-4752-93ca-5203cf742d79 - 12345 - - -]
   Reloading cached file /etc/barbican/policy.json read_cached_file

Re: [openstack-dev] [neutron] Neutron API rate limiting

2015-05-15 Thread Tidwell, Ryan

The Nova analog in Neutron is specifically what I was interested in.  Makes 
perfect sense.  Thanks!

-Ryan

-Original Message-
From: Russell Bryant [mailto:rbry...@redhat.com] 
Sent: Friday, May 15, 2015 11:54 AM
To: openstack-dev@lists.openstack.org
Subject: Re: [openstack-dev] [neutron] Neutron API rate limiting

On 05/14/2015 08:32 PM, Kevin Benton wrote:
 There isn't anything in neutron at this point that does that. I think 
 the assumption so far is that you could rate limit at your load 
 balancer or whatever distributes requests to neutron servers.

Right, which a lot of sense given the horizontally scalable nature of the API 
workers.  Nova had some rate limiting built-in but I think it may have even 
been disabled by default now because it's basically useless when you run 
multiple API workers.

--
Russell Bryant

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] Barbican : Unable to execute the curl command for uploading/retrieving the secrets with the latest Barbican code.

2015-05-15 Thread John Vrbanac

Asha,
We landed the fix in: https://review.openstack.org/#/c/183391/
Hopefully, that should address the problem you've been seeing.

Thanks!

John Vrbanac


On Thu, 2015-05-14 at 18:14 -0500, Douglas Mendizábal wrote:
 Hi Asha,
 
 The reason we support an Unauthenticated Context in Barbican is purely
 for development purposes.  We recommend that all production Barbican
 deployments use Keystone or an alternative AuthN/AuthZ service in
 front of Barbican.
 
 Setting up a working Keystone environment just to hack on Barbican is
 a steep requirement, which is why we need the Unauthenticated Context
 to work.
 
 - Douglas Mendizabal
 
 On 5/14/15 6:07 PM, Asha Seshagiri wrote:
  Thanks a lot John for your response. But would like to know why do
  would we have to fix the issue for creating the secret for
  unauthenticated context for Barbican since it would be good to have
  access control mechanism  enforced to access secrets , orders and
  other entities from Barbican.
  
  This should be the expected behavior from security perspective .And
  also we are able to access secrets by providing the right token
  from the Identity service (Keystone ). Looking forward for your
  response.
  
  Thanks and Regards, Asha Seshagiri
  
  On Thu, May 14, 2015 at 4:43 PM, John Vrbanac 
  john.vrba...@rackspace.com mailto:john.vrba...@rackspace.com
  wrote:
  
  __ Asha, I spent some time looking into this, It looks to be a
  regression that occurred a few days ago when a CR was merged that
  moved us over to oslo_context. I have reported the issue here: 
  https://bugs.launchpad.net/barbican/+bug/1455247
  
  I have a couple ideas on how to fix it, so keep your eyes out for
  a CR to resolve the issue.
  
  John Vrbanac
  
  
  
  On Thu, 2015-05-14 at 12:26 -0500, Asha Seshagiri wrote:
  Hi all ,
  
  
  We are able to execute the curl commands on new barbican code 
  provided we integrated it with keystone . I ran into this issue
  because I was trying to configure localhost to actual IP on a
  plain barbican server so that I would get the response and
  request objects with the actual IP rather than the local host . 
  This configuration was required for seting up HA proxy for
  Barbican.
  
  And then I thought of integrating with the keystone and
  configure Babrican server to https.
  
  *Its a good learning to know that the latest code drop of
  Barbican enforces the authentication mechanism with the keystone
  which would not allow us to execute the curl command without
  providing the token of Identity service (Keystone ) in the
  request unlike the previous Barbican versions*
  
  Please find the curl command request and responses for 
  uploading/reteriving the secets on Barbican Server
  
  root@Clientfor-HAProxy barbican]# curl -X POST -H 
  'content-type:application/json' -H 'X-Project-Id:12345' \
  -H X-Auth-Token:c9ac81784e1e4e089fccbca19f862be2 -d
  '{payload: my-secret-here,payload_content_type:
  text/plain}' \
  -k https://localhost:9311/v1/secrets
  {secret_ref: 
  https://localhost:9311/v1/secrets/02336016-623b-4deb-bca5-caedc0bf0e
 35}[root@Clientfor-HAProxy
 
  
 barbican]#
  
  [root@Clientfor-HAProxy barbican]# curl -H 'Accept: 
  application/json' -H 'X-Project-Id:12345' \
  -H X-Auth-Token:c9ac81784e1e4e089fccbca19f862be2 -k
  https://localhost:9311/v1/secrets {secrets: [{status:
  ACTIVE, secret_type: opaque, updated:
  2015-05-14T16:35:44.109536, name: null, algorithm: null,
  created: 2015-05-14T16:35:44.103982, secret_ref: 
  https://localhost:9311/v1/secrets/02336016-623b-4deb-bca5-caedc0bf0e
 35,
 
  
 content_types: {default: text/plain}, creator_id:
  cedd848a8a9e410196793c601c03b99a, mode: null, bit_length: 
  null, expiration: null}], total: 1}[root@Clientfor-HAProxy 
  barbican]#
  
  Thanks and Regards, Asha Seshagiri
  
  On Wed, May 13, 2015 at 4:26 PM, Asha Seshagiri 
  asha.seshag...@gmail.com mailto:asha.seshag...@gmail.com
  wrote:
  
  Hi all ,
  
  
  
  When I started  debugging ,we find that default group  is not 
  used instead oslo_policy would be used
  
  Please find the logs below :
  
  
  *2015-05-13 15:59:34.393 13210 WARNING oslo_config.cfg [-] Option
  policy_default_rule from group DEFAULT is deprecated. Use
  option policy_default_rule from group oslo_policy.* 
  *2015-05-13 15:59:34.394 13210 WARNING oslo_config.cfg [-] Option
  policy_file from group DEFAULT is deprecated. Use option
  policy_file from group oslo_policy.* 2015-05-13 15:59:34.395
  13210 DEBUG oslo_policy.openstack.common.fileutils 
  [req-0c6d2db4-bc15-4752-93ca-5203cf742d79 - 12345 - - -] 
  Reloading cached file /etc/barbican/policy.json read_cached_file 
  /usr/lib/python2.7/site-packages/oslo_policy/openstack/common/fileuti
 ls.py:64
 
  
 2015-05-13 15:59:34.398 13210 DEBUG oslo_policy.policy
  [req-0c6d2db4-bc15-4752-93ca-5203cf742d79 - 12345 - - -] Reloaded
  policy file: /etc/barbican/policy.json _load_policy_file 
  /usr/lib/python2.7/site-packages/oslo_policy/policy.py:424

Re: [openstack-dev] [neutron] Neutron API rate limiting

2015-05-15 Thread Tidwell, Ryan

From: Carl Baldwin [c...@ecbaldwin.net]
Sent: Thursday, May 14, 2015 9:10 PM
To: OpenStack Development Mailing List
Subject: Re: [openstack-dev] [neutron] Neutron API rate limiting

@Gal, your proposal sounds like packet or flow rate limiting of data through a 
port.  What Ryan is proposing is rate limiting of api requests to the server.  
They are separate topics, each may be a valid need on its own but should be 
considered separately.

@Ryan, I tend to agree that rate limiting belongs in front of the api servers 
at the load balancer level.  That is not to say we couldn't eventually use our 
own lbaas for this someday and integrate rate limiting there.  Thoughts?

Carl

On May 14, 2015 9:26 PM, Gal Sagie 
gal.sa...@gmail.commailto:gal.sa...@gmail.com wrote:
Hello Ryan,

We have proposed a spec to liberty to add rate limit functionality to security 
groups [1].
We see two big use cases for it, one as you mentioned is DDoS for east-west and 
another
is brute force prevention (for example port scanning).

We are re-writing the spec as an extension to the current API, we also have a 
proposal
to enhance the Security Group / FWaaS implementation in order to make it easily 
extendible by such
new classes of security rules.

We are planning to discuss all of that in the SG/FWaaS future directions 
session [2].
I or Lionel will update you as soon as we have the fixed spec for review, and 
feel free to come to the discussion
as we are more then welcoming everyone to help this effort.

Gal.

[1] https://review.openstack.org/#/c/151247/
[2] https://etherpad.openstack.org/p/YVR-neutron-sg-fwaas-future-direction

On Fri, May 15, 2015 at 2:21 AM, Tidwell, Ryan 
ryan.tidw...@hp.commailto:ryan.tidw...@hp.com wrote:
I was batting around some ideas regarding IPAM functionality, and it occurred 
to me that rate-limiting at an API level might come in handy and as an example 
might help provide one level of defense against DoS for an external IPAM 
provider that Neutron might make calls off to.  I’m simply using IPAM as an 
example here, there are a number of other (ie better) reasons for rate-limiting 
at the API level.  I may just be ignorant (please forgive me if I am ☺ ), but 
I’m not aware of any rate-limiting functionality at the API level in Neutron.  
Does anyone know if such a feature exists that could point me at some 
documentation? If it doesn’t exist, has the Neutron community broached this 
subject before? I have to imagine someone has brought this up before and I just 
was out of the loop.  Anyone have thoughts they care to share? Thanks!

-Ryan

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: 
openstack-dev-requ...@lists.openstack.org?subject:unsubscribehttp://openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

--
Best Regards ,

The G.

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: 
openstack-dev-requ...@lists.openstack.org?subject:unsubscribehttp://openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Certainly you want to do some rate-limiting before a request even hits Neutron. 
 I was asking the question since I believe Nova has a rate-limiting feature 
that is built-in, although it seems to serve a different purpose than just 
keeping generic DoS attacks at bay (which is why you want to put something in 
front of Neutron/Nova/etc.).  I simply wondered if there was any utility to 
per-tenant throttling which is what Nova seems to have.  I shared a very poor 
example and wasn't very clear, my apologies.

-Ryan
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [neutron] Neutron API rate limiting

2015-05-15 Thread Russell Bryant

On 05/14/2015 08:32 PM, Kevin Benton wrote:
 There isn't anything in neutron at this point that does that. I think
 the assumption so far is that you could rate limit at your load balancer
 or whatever distributes requests to neutron servers.

Right, which a lot of sense given the horizontally scalable nature of
the API workers.  Nova had some rate limiting built-in but I think it
may have even been disabled by default now because it's basically
useless when you run multiple API workers.

-- 
Russell Bryant

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [neutron] Neutron API rate limiting

2015-05-15 Thread Rick Jones


On May 14, 2015 9:26 PM, Gal Sagie 
gal.sa...@gmail.commailto:gal.sa...@gmail.com wrote:
Hello Ryan,

We have proposed a spec to liberty to add rate limit functionality to security 
groups [1].
We see two big use cases for it, one as you mentioned is DDoS for east-west and 
another
is brute force prevention (for example port scanning).

We are re-writing the spec as an extension to the current API, we also have a 
proposal
to enhance the Security Group / FWaaS implementation in order to make it easily 
extendible by such
new classes of security rules.

We are planning to discuss all of that in the SG/FWaaS future directions 
session [2].
I or Lionel will update you as soon as we have the fixed spec for review, and 
feel free to come to the discussion
as we are more then welcoming everyone to help this effort.

Gal.

[1] https://review.openstack.org/#/c/151247/
[2] https://etherpad.openstack.org/p/YVR-neutron-sg-fwaas-future-direction


Isn't there already described a way to rate-limit instances (overall) by 
putting qdiscs onto their tap devices?


Having looked only briefly at the spec, I would say you want to have the 
option to MARK that traffic which is EDN enabled the rate limiting 
might have otherwise dropped.


The extant mechanism I mentioned uses HTB in one direction (instance 
inbound/tap outbound) and a policing filter in the other.  I've used it 
(as a mostly end user) and have noticed that as described, one can 
introduce non-trivial bufferbloat inbound to the instance.


And I've always wished (as in if wishes were horses) that the instance 
outbound throttle were actually implemented in a way where it becomes 
immediately apparent to the instance by causing the tx queue in the 
instance to build-up.  That wouldn't be something on a tap device though.


Does there need to be both a packet and bit rate limit?  I've some 
experience with bit rate limits and have seen otherwise rather throttled 
(bitrate) instances cause non-trivial problems with a network node.


rick jones

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [openstackclient] About my blueprint every-time-record-log-in-file

2015-05-15 Thread Dean Troyer

On Fri, May 15, 2015 at 7:04 AM, Fujita, Daisuke
fuzita.dais...@jp.fujitsu.com wrote:

I would like to implement following my buleprint.

https://blueprints.launchpad.net/python-openstackclient/+spec/every-time-record-log-in-file

We don't always follow the blueprint process rigidly, but thanks for
starting out that way.

So, Could you tell me when and how blueprint is approved?
My understanding is that the blueprint is determined to approve or not
in Vancouver. Please let me know about your opinion.

We can talk about this BP in Vancouver due to the timing, but that is not a
requirement. The places to talk about things like this are in the BP
itself, but primarily in the weekly OSC meeting [0].

Regarding the BP content, I have concerns with always writing logs to the
user's disk, so anything added would need to be turned off by default.
Also, the only configuration we have at this time is the relatively new
clouds.yaml referenced by the --os-cloud option. Nothing in there is
currently used to OSC like this, but that is where anything new would have
to go.

I do think you have some good ideas about what the log needs to include,
such as adding the source of the options (cli, environment, clouds.yaml).
OSC's debug output needs some cleaning, and I would like to have an
intermediate level that provides some specifics without all of the details,
such as the REST API endpoints called, etc. I would like to hear if you
have some input on what that might look like.

We can talk about this next week, I understand you will not be in Vancouver
but someone who can speak to this will, correct?

[0] https://wiki.openstack.org/wiki/Meetings/OpenStackClient

Dean Troyer
dtro...@gmail.com
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [api] http guideline outline plus ad-hoc plane sprint

2015-05-15 Thread Sean Dague

After many interesting conversations with jaypipes and cdent the last
couple of weeks, and a few edge case API change patches in Nova, I
decided to try to jump in and help provide context and an outline for
the http guidelines in the spec.

This is a standard that's clocking in on 20 years, there is a lot of
water under this bridge, and a lot of context that if we get it down,
should help projects going forward.

https://etherpad.openstack.org/p/api-wg-http is a draft outline, and
some ideas.

I've been slicing chunks off while packing in per bullet patches -
https://review.openstack.org/#/q/status:open+project:openstack/api-wg+branch:master+topic:http,n,z

But I think that given time on planes, if a few people grabbed a bullet
or two before boarding, we could probably get that plane sprinted by the
end of the summit. Which would be a great step forward.

Commentary welcomed, as always.

-Sean

-- 
Sean Dague
http://dague.net

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [neutron][lbaas]LBaaSv2 movies / links

2015-05-15 Thread Samuel Bercovici

Bad link. Here is the correct one: 
https://filepile.radware.com/files/2cd-953-809


From: Samuel Bercovici [mailto:samu...@radware.com]
Sent: Friday, May 15, 2015 7:48 PM
To: OpenStack Development Mailing List (not for usage questions); Vijay 
Venkatachalam; Evgeny Fedoruk; Adam Harwell; Kyle Mestery; Brandon Logan; 
Johnson, Michael (HP Cloud - Corvallis); Doug Wiegley
Subject: Re: [openstack-dev] [neutron][lbaas]LBaaSv2 movies / links

Here is an example: https://filepile.radware.com/files/5ba-e02-c39


From: Samuel Bercovici [mailto:samu...@radware.com]
Sent: Friday, May 15, 2015 7:06 PM
To: Vijay Venkatachalam; OpenStack Development Mailing List (not for usage 
questions); Evgeny Fedoruk; Adam Harwell; Kyle Mestery; Brandon Logan; Johnson, 
Michael (HP Cloud - Corvallis); Doug Wiegley
Subject: Re: [openstack-dev] [neutron][lbaas]LBaaSv2 movies / links

Was thinking 1-2 minutes. Tomorrow would be fine.

From: Vijay Venkatachalam [mailto:vijay.venkatacha...@citrix.com]
Sent: Friday, May 15, 2015 6:03 PM
To: OpenStack Development Mailing List (not for usage questions); Evgeny 
Fedoruk; Adam Harwell; Kyle Mestery; Brandon Logan; Johnson, Michael (HP Cloud 
- Corvallis); Doug Wiegley; Samuel Bercovici
Subject: Re: [openstack-dev] [neutron][lbaas]LBaaSv2 movies / links

Hi Sam,
Thanks for probing. How many seconds/mins you have thought per vendor? By when 
do you need this? Will tomorrow work fine?


Thanks,
Vijay V.

Sent from Surface

From: Samuel Bercovicimailto:samu...@radware.com
Sent: ‎Friday‎, ‎15‎ ‎May‎ ‎2015 ‎00‎:‎30
To: OpenStack Development Mailing List (not for usage 
questions)mailto:openstack-dev@lists.openstack.org, Evgeny 
Fedorukmailto:evge...@radware.com, Adam 
Harwellmailto:adam.harw...@rackspace.com, Kyle 
Mesterymailto:mest...@mestery.com, Brandon 
Loganmailto:brandon.lo...@rackspace.com, Johnson, Michael (HP Cloud - 
Corvallis)mailto:michael.d.john...@hp.com, Doug 
Wiegleymailto:do...@a10networks.com, Samuel 
Bercovicimailto:samu...@radware.com

Hi Everyone,

As you may be aware, we have a speaking slot on the Vancouver summit to discuss 
“LBaaS v2, Kilo and beyond” on Monday 
https://openstacksummitmay2015vancouver.sched.org/event/3f1e9e24f36238152749afea9c21a264#.VVTwCPmqqko

We are considering to show vendors demos or list/link such movies.
Please let us know if you have such so we can list/include in the talk.

Regards,
-Sam.

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [ceph] Introducing CephEWS

2015-05-15 Thread Debojyoti Dutta

Hi!

Its a pleasure to introduce CephEWS, an awesome dashboard for Ceph with a
twist - it has ceph health checks and warnings built in along with OSD and
CRUSH map viz etc. https://github.com/CiscoSystems/cephEWS

Hope to see a lot of pull requests and feedback!

thx
-Debo~
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Openstack-operators] [nova] Can we bump MIN_LIBVIRT_VERSION to 1.2.2 in Liberty?

On 15 May 2015 at 11:51, Daniel P. Berrange berra...@redhat.com wrote:
 On Thu, May 14, 2015 at 02:23:25PM -0500, Matt Riedemann wrote:
 The minimum required version of libvirt in the driver is 0.9.11 still [1].
 We've been gating against 1.2.2 in Ubuntu Trusty 14.04 since Juno.

 The libvirt distro support matrix is here: [2]

 Can we safely assume the people aren't going to be running Libvirt compute
 nodes on RHEL  7.1 or Ubuntu Precise?

 I don't really think so - at the very least Fedora 20 and RHEL 7.0 are still
 actively supported platforms by their vendors, which both have older libvirt
 versions (1.1.3 and 1.1.1 respectively).

 I'm not sure whether SUSE team consider any of the 12.x versions to still be
 actively supported platforms or not, likewise which 13.x versions are under
 active support.

As I understand it, RHEL 5.0 is supported until March 31, 2017.
https://access.redhat.com/support/policy/updates/errata

Where should we draw the line? Its a tricky question.

 Regarding RHEL, I think this is a safe bet because in Kilo nova dropped
 python 2.6 support and RHEL  6 doesn't have py26 so you'd be in trouble
 running kilo+ nova on RHEL 6.x anyway.

 There are add-on repos for RHEL-6 and RHEL-7 that provide newer python
 versions so (py27, various py3x), so it is not unreasonable for people
 to consider sticking with RHEL-6 if that is what their existing deployment
 is based on. Certainly new deployments though I'd expect to be RHEL-7 based.

It seems like we are OK with deprecating support for libvirt  0.10.2 in M then?


 There are some workarounds in the code [3] I'd like to see removed by
 bumping the minimum required version.

 Sure, its nice to remove workarounds from a cleanliness POV, but I'm generally
 pretty conservative about doing so, because in the majority of case (while it
 looks ugly) it is not really a significant burden on maintainers to keep it
 around.

 snip

 In this case, I just don't see compelling benefits to Nova libvirt maint
 to justify increasing the minimum version to the level you suggest, and
 it has a clear negative impact on our users which they will not be able
 to easily deal with. They will be left with 3 options all of which are
 unsatisfactory

  - Upgrade from RHEL-6 to RHEL-7 - a major undertaking for most organizations
  - Upgrade libvirt on RHEL-6 - they essentially take on the support burden
for the hypervisor themselves, loosing support from the vendor
  - Re-add the code we remove from Nova - we've given users a maint burden,
to rid ourselves of code that was posing no real maint burden on ourselves.

Thats assuming vendors do not backport libvirt and QEMU and support that.

 As a more general point, I think we are lacking clear guidance on our
 policies around hypervisor platform support and thus have difficulty
 in deciding when it is reasonable for us to drop support for platforms.

+1

Lets try to fixing this in Liberty.

I see this as part of feature classification:
http://libertydesignsummit.sched.org/event/7ee9be7e3b005880706a914f44b296ed

Honestly, I want us to call out the exact combinations we test. But
that in logs, or in the release notes , or docs, or some combination
of all of those.

Partly so we are honest about what we know works, partly so folks
setup up and help us test more combinations users care about.

This should really be true for everything, VMware, XenServer, Hyper-V
and others.

 I think it is important to distinguish the hypervisor platform, from
 the openstack platform because there are different support implications
 there for users.

Agreed.

 The hypervisor platform is very different. While OpenStack does achieve
 some level of testing coverage of the hypervisor platform version used
 in the gate, this testing is inconsequential compared to the level of
 testing that vendors put into their hypervisor platforms.

That depends on the scope right...

Sure, we don't really test the data plane, that is left the
hypervisor vendor.

But we do heavily test the control plane, and thats useful.

 Even if users decide they want to upgrade their hypervisor platform to
 a new version provided officially by the vendor, this is not always a
 quick or easy task. Many organizations have non-trivial internal
 testing and certification requirements before upgrading OS and/or hypervisor
 platforms. The hardware they own has to be certified as compatible and
 tested. They often have 3rd party monitoring and security auditing tools
 that need to be upgraded and integrated with the new platform. They may
 need todo long term stress tests to prove the new platform / hardare
 combination is reliable at meeting their uptime requirements, so on.
 So even with an active desire to upgrade their platform, it may take
 anywhere from 3-6 months to actually put that plan into pratice. It may
 seem strange, but at the same time they can be perfectly ok upgrading
 openstack in a matter of weeks, or even doing continuous deployment,
 simply

Re: [openstack-dev] devstack local.conf file fro sriov pci nic passthrough

2015-05-15 Thread Robert Li (baoli)

Hi,

On your controller node,  you can add in local.conf:

[[post-config|$NOVA_CONF]]
[DEFAULT]
scheduler_default_filters = RetryFilter, AvailabilityZoneFilter, RamFilter, 
ComputeFilter, ComputeCapabilitiesFilter, ImagePropertiesFilter, 
ServerGroupAntiAffinityFilter, ServerGroupAffinityFilter, PciPassthroughFilter

On your compute node:
[[post-config|$NOVA_CONF]]
[DEFAULT]
pci_passthrough_whitelist = your whitelist definition 1
…
pci_passthrough_whitelist = your whitelist definition 2

You can get examples from the sr-iov wikis as people has pointed out from other 
threads.

good luck.

—Robert




On 5/15/15, 9:13 AM, Kamsali, RaghavendraChari (Artesyn) 
raghavendrachari.kams...@artesyn.commailto:raghavendrachari.kams...@artesyn.com
 wrote:

Hi,
I am bringing up the single node openstack cloud and having pci sriov supported 
nic controller (intel XL710), created 4virtual functions on top of the nic. My 
goal is to bring up the cloud setup with neutron and nova network services 
using devstack.
How to configure local.conf file  so that when any VM spawns , it used the 
virtual function of the sriov nic and able to communicate .

Could anyone help me ???


Thanks and Regards,
Raghavendrachari kamsali | Software Engineer II  | Embedded Computing
Artesyn Embedded Technologies|5th Floor, Capella Block, The V, Madhapur| 
Hyderabad, AP 500081 India
T +91-40-66747059 | M +919705762153

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [nova][heat] sqlalchemy-migrate tool to alembic

2015-05-15 Thread Mike Bayer




On 5/15/15 1:13 PM, Mike Bayer wrote:



On 5/15/15 9:31 AM, Doug Hellmann wrote:

This seems more complicated than needed. If we just stop writing the
sqlalchemy-migrate scripts and don't change them, then for 1 cycle we
have to run both sets of migrations and after that we can just run
alembic.
Then we have a forever-in-perpetuity dependency on SQLAlchemy-Migrate 
which must be maintained forever for to maintain compatibility with 
all new SQLAlchemy, oslo.db, etc. releases, despite it never being 
used for anythine new, because it will be impossible to install an 
Openstack application without running through the first set of migrate 
scripts first.


The SQLAlchemy-Migrate dependency must be dropped and the project has 
to be EOL'ed at some point.   Leaving it in is definitely the more 
complicated alternative.


Maybe I'm not understanding what you mean.  Maybe you mean, do the 
migrate-alembic compatibility thing in oslo.db, but then on the next 
release cycle, *do* rewrite the migrate files in Alembic and drop the 
migrate dependency?   That is, if a site wants to upgrade from K - N, 
they need to install and migrate each of K, L, M individually 
first...e.g. it's not expected that N can upgrade from an existing K 
install ?






__ 


OpenStack Development Mailing List (not for usage questions)
Unsubscribe: 
openstack-dev-requ...@lists.openstack.org?subject:unsubscribe

http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [surge] Introducing Surge - rapid deploy/scale stream processing systems on OpenStack

On Fri, May 15, 2015 at 10:13 AM, Debojyoti Dutta ddu...@gmail.com wrote:

 Hi,

 It gives me a great pleasure to introduce Surge - a system to rapidly
 deploy and scale a stream processing system on OpenStack. It leverages
 Vagrant and Ansible, and supports both OpenStack as well as the local mode
 (with VirtualBox).

 https://github.com/CiscoSystems/surge


I see you support Storm and Kafka,

How is this different then Sahara's Storm plugin?

https://github.com/openstack/sahara/blob/45045d918f363fa5763cde700561434345017661/setup.cfg#L47

And I See Sahara is exploring Kafka support:
https://blueprints.launchpad.net/sahara/+spec/cdh-kafka-service


 Hope to see a lot of pull requests and comments.

 thx
 -Debo~

 __
 OpenStack Development Mailing List (not for usage questions)
 Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [Fuel][Library] Binary Files Installation Finally Moved Into Packages

2015-05-15 Thread Vladimir Kuklin

Folks

As I promised before we merged the code that builds packages for binary
files like OCF scripts and other things.

Starting from now, if you want to put files like OCF scripts, some binaries
or services onto the node with puppet - do not do it. Simply use
debian/rules or RPM specs to build a package and install them.

We will enable corresponding noop tests shortly that, but I urge reviewers
to not accept patches containing such code.

If you find any bugs, please feel free to file them

Merged reviews are here:

https://review.openstack.org/#/c/172542/
https://review.openstack.org/#/c/178354/


-- 
Yours Faithfully,
Vladimir Kuklin,
Fuel Library Tech Lead,
Mirantis, Inc.
+7 (495) 640-49-04
+7 (926) 702-39-68
Skype kuklinvv
35bk3, Vorontsovskaya Str.
Moscow, Russia,
www.mirantis.com http://www.mirantis.ru/
www.mirantis.ru
vkuk...@mirantis.com
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [nova][all] Architecture Diagrams in ascii art?

On Tue, May 12, 2015 at 5:25 AM, Thierry Carrez thie...@openstack.org
wrote:

 Sean Dague wrote:
  They can be modified if you provide source files, or use a source
  oriented format like SVG, or ISO standard ODG (used by OpenOffice /
  LibreOffice). There is a reason the spider diagram has ended up in
  every single OpenStack presentation I've seen for the last 2 years.

 And there is also a reason the spider diagram was never updated (or
 fixed for accuracy, for that matter).


I tried creating the spider diagram pragmatically with pydot, but it didn't
work very well, it just looks like plate of spaghetti

Diagram: https://i.imgur.com/fDqpB9m.png
source: https://github.com/jogo/graphing-openstack


 I'm not saying we should use ASCII art, I'm saying we should use a
 source-oriented format so that we reduce the likeliness of stale
 information. I was mostly reacting to Matt's mention of real image
 files (taking as an example the pictures in
 https://wiki.openstack.org/wiki/QA/AuthInterface which are PNGs without
 any source).

 If only one person can easily update a picture, it *will* go stale. And
 I think I prefer ugly to plain wrong. With source-oriented formats
 you can take advantage of more dimensions without sacrificing the
 ability to update it.

 --
 Thierry Carrez (ttx)

 __
 OpenStack Development Mailing List (not for usage questions)
 Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [nova][all] Architecture Diagrams in ascii art?