Re: [openstack-dev] [nova] Libguestfs: possibility not to use it, even when installed ?

2015-02-23 Thread Raphael Glon

On 02/23/2015 11:23 AM, Daniel P. Berrange wrote:

The alternative Nova implementation is*not*  using fuse,


yeah you're right - read your page too quickly 
https://www.berrange.com/tags/nbd/


added the term fuse where it should not have been
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [nova] Libguestfs: possibility not to use it, even when installed ?

2015-02-23 Thread Daniel P. Berrange
On Mon, Feb 23, 2015 at 11:52:29AM +0100, Raphael Glon wrote:
 On 02/23/2015 11:23 AM, Daniel P. Berrange wrote:
 The alternative Nova implementation is*not*  using fuse, it is using real
 mounts on the host FS. This is not a potential issue, it is an*actual*
 issue. There have been bugs in Linux filesystem drivers, including ext4,
 that would have allowed a malicous kernel image to crash and/or exploit
 the host kernel if mounted.
 
http://libguestfs.org/guestfs.3.html#security-of-mounting-filesystems
 
 Ok noted - so why is losetup or qemu-nbd still proposed by nova and still
 the default method ?

Libguestfs method takes priority if it is installed on the host, but
the legacy code still exists for benefit of existing deployed setups
and drivers which don't have qemu/kvm available, eg LXC containers.

Regards,
Daniel
-- 
|: http://berrange.com  -o-http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org  -o- http://virt-manager.org :|
|: http://autobuild.org   -o- http://search.cpan.org/~danberr/ :|
|: http://entangle-photo.org   -o-   http://live.gnome.org/gtk-vnc :|

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [horizon][heat]Vote for Openstack L summit topic The Heat Orchestration Template Builder: A demonstration

2015-02-23 Thread Aggarwal, Nikunj
Hi Angus,

I am working Timur and Drago on HOT builder. I am using this - 
https://github.com/rackerlabs/hotbuilder as the base and made some improvements 
over the existing code and wish to give a demonstration on how easy it is to 
create a HOT template from Horizon.

Regards,
Nikunj

From: Angus Salkeld [mailto:asalk...@mirantis.com]
Sent: Monday, February 23, 2015 4:52 AM
To: OpenStack Development Mailing List (not for usage questions)
Subject: Re: [openstack-dev] [horizon][heat]Vote for Openstack L summit topic 
The Heat Orchestration Template Builder: A demonstration

On Sat, Feb 21, 2015 at 4:21 AM, Aggarwal, Nikunj 
nikunj.aggar...@hp.commailto:nikunj.aggar...@hp.com wrote:
Hi,

I have submitted  presentations for Openstack L summit :


The Heat Orchestration Template Builder: A 
demonstrationhttps://www.openstack.org/vote-vancouver/Presentation/the-heat-orchestration-template-builder-a-demonstration



Hi
Nice to see work on a HOT builder progressing, but..
are you planning on integrating this with the other HOT builder efforts?
Is the code public (link)?

This is more of a framework to make these easier to build:
https://github.com/stackforge/merlin
https://wiki.openstack.org/wiki/Merlin

Timur (who works on Merlin) is working with Rackers to build this upstream - I 
am not sure on the progress.
https://github.com/rackerlabs/hotbuilder
https://github.com/rackerlabs/foundry

It would be nice if we could all work together (I am hoping you are already)?
Hopefully some of the others that are working on this can chip in and say where 
they
are.

-Angus


Please cast your vote if you feel it is worth for presentation.

Thanks  Regards,
Nikunj


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: 
openstack-dev-requ...@lists.openstack.org?subject:unsubscribehttp://openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [nova] Libguestfs: possibility not to use it, even when installed ?

2015-02-23 Thread Daniel P. Berrange
On Mon, Feb 23, 2015 at 11:08:31AM +0100, Raphael Glon wrote:
 On 02/19/2015 12:45 PM, Richard W.M. Jones wrote:
 On Wed, Feb 18, 2015 at 07:23:52PM +0100, Raphael Glon wrote:
 I entcountered a similar case more recently on powerkvm 2.1.0
 (defect with the libguestfs)
 What's the actual bug?  We've worked hard, with IBM, to make
 libguestfs work on POWER 7 and POWER 8 systems.  I have full access to
 those systems through Red Hat.  If there's a new bug I'm sure we'll be
 able to fix it.
 
 Rich.
 
 Hi, thank you for all your answers.
 
 Not saying there are actual bugs (anyway I'm stuck here because i would
 need to find time+environment to recheck all/reproduce) - i haven't even
 deployed juno on pkvm yet
 
 We've talked with ibm (and they have likely been working with you) and they
 are really responsive in fixing defects with their distribution
 
 We've entcountered two problems with powerkvm regarding nova + libguestfs.
 
 1. since pkvm 2.1.x is forked from a Fedo 19, we fell back to this Red Hat
 bug you fixed regarding the attach method
 
 Note that one of the workaround proposed was
 
 sudo sysctl -w fs.protected_hardlinks=0 + common user nova/qemu
 
 
 - Not a specialist here, but seems like to be able to use libguestfs to
 avoid potential issues with fuse mounts, we open other potential holes
 somewhere else

The alternative Nova implementation is *not* using fuse, it is using real
mounts on the host FS. This is not a potential issue, it is an *actual*
issue. There have been bugs in Linux filesystem drivers, including ext4,
that would have allowed a malicous kernel image to crash and/or exploit
the host kernel if mounted.

  http://libguestfs.org/guestfs.3.html#security-of-mounting-filesystems

The libguestfs architecture is explicitly designed such that any security
critical tasks take place inside an unprivileged KVM guest. So unless Nova
is using libguestfs in a broken way, the security of libguestfs is effectively
equivalent to the security of KVM in general. This is a faaar better security
architecture design


 2. because pkvm 2.1.x is forked from fedo 19 it embeds rather old versions
 of libguestfs and libvirt.

Fedora 19 is end of life so not really relevant any more as a target.
If there are bugs you find in current versions of Fedora please file
bugs so they can be addressed.

Regards,
Daniel
-- 
|: http://berrange.com  -o-http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org  -o- http://virt-manager.org :|
|: http://autobuild.org   -o- http://search.cpan.org/~danberr/ :|
|: http://entangle-photo.org   -o-   http://live.gnome.org/gtk-vnc :|

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [nova] Libguestfs: possibility not to use it, even when installed ?

2015-02-23 Thread Raphael Glon

On 02/23/2015 11:23 AM, Daniel P. Berrange wrote:

The alternative Nova implementation is*not*  using fuse, it is using real
mounts on the host FS. This is not a potential issue, it is an*actual*
issue. There have been bugs in Linux filesystem drivers, including ext4,
that would have allowed a malicous kernel image to crash and/or exploit
the host kernel if mounted.

   http://libguestfs.org/guestfs.3.html#security-of-mounting-filesystems


Ok noted - so why is losetup or qemu-nbd still proposed by nova and 
still the default method ?


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [nova] Libguestfs: possibility not to use it, even when installed ?

2015-02-23 Thread Raphael Glon

On 02/23/2015 11:23 AM, Daniel P. Berrange wrote:

Fedora 19 is end of life so not really relevant any more as a target.

 powerkvm 2.1.x is not eol

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [nova] Libguestfs: possibility not to use it, even when installed ?

2015-02-23 Thread Daniel P. Berrange
On Mon, Feb 23, 2015 at 11:54:40AM +0100, Raphael Glon wrote:
 On 02/23/2015 11:23 AM, Daniel P. Berrange wrote:
 Fedora 19 is end of life so not really relevant any more as a target.
  powerkvm 2.1.x is not eol

Well Fedora 19 is end of life, so if powerkvm is using Fedora 19, then
it is running on software which has no further update support. So if
powerkvm team find bugs in it, they'll have to take responsibility for
fixing them, as Fedora maintainers won't be doing any fixes for it.

Regards,
Daniel
-- 
|: http://berrange.com  -o-http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org  -o- http://virt-manager.org :|
|: http://autobuild.org   -o- http://search.cpan.org/~danberr/ :|
|: http://entangle-photo.org   -o-   http://live.gnome.org/gtk-vnc :|

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Fuel] Separating granular tasks validator

2015-02-23 Thread Kamil Sambor
Thank you guys for response.
There is no cons, so we migrate validation to separate repo.

Best regards,
Kamil Sambor

On Wed, Feb 18, 2015 at 9:34 AM, Evgeniy L e...@mirantis.com wrote:

 +1 to extract validators for granular deployment tasks

 Dmitry, do you mean that we should create some cli to generate graph
 picture? Or just make it as a module and then use it in Nailgun?

 Thanks,

 On Tue, Feb 17, 2015 at 4:31 PM, Dmitriy Shulyak dshul...@mirantis.com
 wrote:

 +1 for separate tasks/graph validation library

 In my opinion we may even migrate graph visualizer to this library, cause
 it is most usefull during development and to demand installed fuel with
 nailgun feels a bit suboptimal


 On Tue, Feb 17, 2015 at 12:58 PM, Kamil Sambor ksam...@mirantis.com
 wrote:

 Hi all,

 I want to discuss separating validation from our repositories to one. On
 this moment in fuel we have validation for  granular deployment tasks in 3
 separate repositories so we need to maintain very similar code in all of
 them. New idea that we discussed with guys assumes to keep this code in one
 place. Below are more details.

 Schema validator should be in separate repo, we will install validator
 in fuel-plugin, fuel-lib, fuel-nailgun. Validator should support versions
 (return schemas and validate them for selected version).
 Reasons why we should have validation in all three repositories:
 nailgun: we need validation in api because we are able to send our own
 tasks to nailgun and execute them (now we validate type of tasks in
 deployment graph and  during installation of plugin)
 fuel-library: we need to check if tasks schema is correct defined in
 task.yaml files and if tasks not create cycles (actually we do both things)
 fuel-plugin: we need to check if defined tasks are supported by selected
 version of nailgun (now we check if task type are the same with hardcoded
 types in fuel-plugin, we not update this part since a while and now there
 are only 2 type of tasks: shell and puppet)
 With versioning we shouldn't have conflicts between nailgun
 serialization and fuel-plugin because plugin will be able to use schemas
 for specified version of nailgun.

 As a core reviewers of repositories we should keep the same reviewers as
 we have in fuel-core.

 How validator should looks like:
 separate repo, to install using pip
 need to return correct schema for selected version of fuel
 should be able to validate schema for selected version and ignore
 selected fields
 validate graph from selected tasks

 Pros and cons of this solution:
 pros:
 one place to keep validation
 less error prone - we will eliminate errors caused by not updating one
 of the repos, also it will be easy to test if changes are correct and
 compatible with all repos
 easier to develop (less changes in cases when we add new type of task or
 we change schemas of tasks - we edit just one place)
 easy distribution of code between repositories and easy to use by
 external developers
 cons:
 new repository that needs to be managed (and included into CI/QA/release
 cycle)
 new dependency for fuel-library, fuel-web, fuel-plugins (fuel plugin
 builder) of which developer need to be aware of

 Please comments and give opinions.

 Best regards,
 Kamil Sambor


 __
 OpenStack Development Mailing List (not for usage questions)
 Unsubscribe:
 openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



 __
 OpenStack Development Mailing List (not for usage questions)
 Unsubscribe:
 openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



 __
 OpenStack Development Mailing List (not for usage questions)
 Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [horizon][heat]Vote for Openstack L summit topic The Heat Orchestration Template Builder: A demonstration

2015-02-23 Thread Angus Salkeld
On Mon, Feb 23, 2015 at 9:18 PM, Aggarwal, Nikunj nikunj.aggar...@hp.com
wrote:

  Hi Angus,

  I am working Timur and Drago on HOT builder. I am using this -
 https://github.com/rackerlabs/hotbuilder as the base and made some
 improvements over the existing code and wish to give a demonstration on how
 easy it is to create a HOT template from Horizon.


That's awesome, looking forward to see it!

-Angus




 Regards,

 Nikunj



 *From:* Angus Salkeld [mailto:asalk...@mirantis.com]
 *Sent:* Monday, February 23, 2015 4:52 AM
 *To:* OpenStack Development Mailing List (not for usage questions)
 *Subject:* Re: [openstack-dev] [horizon][heat]Vote for Openstack L summit
 topic The Heat Orchestration Template Builder: A demonstration



 On Sat, Feb 21, 2015 at 4:21 AM, Aggarwal, Nikunj nikunj.aggar...@hp.com
 wrote:

  Hi,



 I have submitted  presentations for Openstack L summit :



 The Heat Orchestration Template Builder: A demonstration
 https://www.openstack.org/vote-vancouver/Presentation/the-heat-orchestration-template-builder-a-demonstration





 Hi

 Nice to see work on a HOT builder progressing, but..

 are you planning on integrating this with the other HOT builder efforts?

 Is the code public (link)?


 This is more of a framework to make these easier to build:
 https://github.com/stackforge/merlin
 https://wiki.openstack.org/wiki/Merlin

 Timur (who works on Merlin) is working with Rackers to build this upstream
 - I am not sure on the progress.
 https://github.com/rackerlabs/hotbuilder
 https://github.com/rackerlabs/foundry



 It would be nice if we could all work together (I am hoping you are
 already)?

 Hopefully some of the others that are working on this can chip in and say
 where they

 are.


 -Angus





 Please cast your vote if you feel it is worth for presentation.



 Thanks  Regards,

 Nikunj




 __
 OpenStack Development Mailing List (not for usage questions)
 Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



 __
 OpenStack Development Mailing List (not for usage questions)
 Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [neutron] New version of python-neutronclient release: 2.3.11

2015-02-23 Thread Ihar Hrachyshka
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

On 02/20/2015 11:01 PM, Matt Riedemann wrote:
 
 
 On 2/20/2015 6:23 AM, Alan Pevec wrote:
 https://review.openstack.org/#/c/157535/
 
 Do you know where it ends ? We could set up Depends lines on
 those requirements stable/* reviews and line them up so that
 they are ready to merge when openstackclient is fixed in
 devstack.
 
 Alternative workaround is https://review.openstack.org/157654
 which is blocked on swift-dsvm-functional issue fixed by 
 https://review.openstack.org/157670 which is blocked on
 neutronclient i.e. we got a cyclic dep here which will require
 ninja merge to resolve.
 
 I suggest to start with ninja merging 157670 which looks the
 most innocent.
 
 Once we get icehouse working again we can look at backporting
 venv patch series to devstack icehouse.
 
 
 Cheers, Alan
 
 __


 
OpenStack Development Mailing List (not for usage questions)
 Unsubscribe: 
 openstack-dev-requ...@lists.openstack.org?subject:unsubscribe 
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


 
 We disabled the swift functional job on stable/icehouse since it
 doesn't have the tox target and shouldn't have been running on
 stable/icehouse:
 
 https://review.openstack.org/#/c/157825/
 
 That freed up Adam's devstack workaround change:
 
 https://review.openstack.org/#/c/157654/
 
 So we can now focus on the neutronclient cap:
 
 https://review.openstack.org/#/c/157606/
 
 That hit a problem with an uncapped pyghmi version in the gate so
 we're also now capping that library as well.
 

UPD: pyghmi cap didn't work, and it seems that we hit a bug in
'git-diff' that makes it return wrong exit code when used with --quiet
argument. The workaround is to switch to --exit-code argument:

https://review.openstack.org/158247

Once it's merged, we hit another issue - requirements integration
tools trying to sync stable/icehouse requirements for pycadf that does
not maintain a stable branch. The proper fix for this issue is to
avoid integration run for those projects for stable branches:

https://review.openstack.org/158086

/Ihar
-BEGIN PGP SIGNATURE-
Version: GnuPG v1

iQEcBAEBAgAGBQJU6xpTAAoJEC5aWaUY1u57TkgIAKnys1hfuscwDTyErlbsvu6q
JnKkkHSfnn/so36xCt65r6mq1NeBUF+Wn4CFeoMwh6eIbql4aeXJC0voBwH5ScpL
HUKZtz2eL088wJfwu5yhn0TQleh1C5GCQf2rutQv1bcqdwhRbo1MLf0Dda34dJSg
HL6gNfYTm3TckgNvWujNxGOluEUg7It0a/myf9Hn0xDhyLCvdF9+7DMzvotmEai+
HuoGhaFBGjyEYgS/aCRjg0c0qtBxAk4pITQwuIvUzkDWLYQLuW6S4pzaySGDXxP0
zVbqMoReUtyCv1f6t9k43K+K1nj2yloBtlzBsUUF04n9YbGlfVqlWtHu2yAc86c=
=aMhI
-END PGP SIGNATURE-

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [nova] Temporary freeze on API changes

2015-02-23 Thread Matthew Gilliard
The devref has merged:
http://docs.openstack.org/developer/nova/devref/api_microversions.html

There are a few corner cases being worked on now, but the information
in there should be enough to add a new API change  version.

MG

On Fri, Feb 20, 2015 at 12:33 AM, Christopher Yeoh cbky...@gmail.com wrote:
 Hi,

 The Nova API subgroup is planning on releasing V2.1 on Monday (changing its
 status from experimental to CURRENT) and merging the first patch which uses
 microversions on top of 2.1 on the Wednesday.

 In the meantime we'd appreciate it if reviewers did not +A any patchset
 which changes the REST API (as it potentially means we'd have to sync
 changes to 2.1 as well. Any future api changes should only be applied to the
 v2.1/v3 tree and use the microversions mechanism.

 When https://review.openstack.org/#/c/140313/ is merged, that will signal
 that microversions
 is enabled and we are open for patches to v2.1 microversions. At this point
 no changes should be accepted to the old v2 api code and anything to the
 v2.1 code base should be done using the microversions mechanism.

 There is devref docs on how to use microversions either merged or working
 its way through gerrit.

 Regards,

 Chris

 __
 OpenStack Development Mailing List (not for usage questions)
 Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [nova][vmware][ironic] Configuring active/passive HA Nova compute

2015-02-23 Thread Matthew Booth
On 20/02/15 11:48, Matthew Booth wrote:
 Gary Kotton came across a doozy of a bug recently:
 
 https://bugs.launchpad.net/nova/+bug/1419785
 
 In short, when you start a Nova compute, it will query the driver for
 instances and compare that against the expected host of the the instance
 according to the DB. If the driver is reporting an instance the DB
 thinks is on a different host, it assumes the instance was evacuated
 while Nova compute was down, and deletes it on the hypervisor. However,
 Gary found that you trigger this when starting up a backup HA node which
 has a different `host` config setting. i.e. You fail over, and the first
 thing it does is delete all your instances.
 
 Gary and I both agree on a couple of things:
 
 1. Deleting all your instances is bad
 2. HA nova compute is highly desirable for some drivers
 
 We disagree on the approach to fixing it, though. Gary posted this:
 
 https://review.openstack.org/#/c/154029/
 
 I've already outlined my objections to this approach elsewhere, but to
 summarise I think this fixes 1 symptom of a design problem, and leaves
 the rest untouched. If the value of nova compute's `host` changes, then
 the assumption that instances associated with that compute can be
 identified by the value of instance.host becomes invalid. This
 assumption is pervasive, so it breaks a lot of stuff. The worst one is
 _destroy_evacuated_instances(), which Gary found, but if you scan
 nova/compute/manager for the string 'self.host' you'll find lots of
 them. For example, all the periodic tasks are broken, including image
 cache management, and the state of ResourceTracker will be unusual.
 Worse, whenever a new instance is created it will have a different value
 of instance.host, so instances running on a single hypervisor will
 become partitioned based on which nova compute was used to create them.
 
 In short, the system may appear to function superficially, but it's
 unsupportable.
 
 I had an alternative idea. The current assumption is that the `host`
 managing a single hypervisor never changes. If we break that assumption,
 we break Nova, so we could assert it at startup and refuse to start if
 it's violated. I posted this VMware-specific POC:
 
 https://review.openstack.org/#/c/154907/
 
 However, I think I've had a better idea. Nova creates ComputeNode
 objects for its current configuration at startup which, amongst other
 things, are a map of host:hypervisor_hostname. We could assert when
 creating a ComputeNode that hypervisor_hostname is not already
 associated with a different host, and refuse to start if it is. We would
 give an appropriate error message explaining that this is a
 misconfiguration. This would prevent the user from hitting any of the
 associated problems, including the deletion of all their instances.

I have posted a patch implementing the above for review here:

https://review.openstack.org/#/c/158269/

Matt
-- 
Matthew Booth
Red Hat Engineering, Virtualisation Team

Phone: +442070094448 (UK)
GPG ID:  D33C3490
GPG FPR: 3733 612D 2D05 5458 8A8A 1600 3441 EA19 D33C 3490

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [neutron][cisco][apic] entry point for APIC agent

2015-02-23 Thread Ihar Hrachyshka
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

Ivar,

Thanks! Please add me to review list.

is the agent leaving the tree this cycle? If so, there is no big
reason to introduce a new agent in master just to yank it several
weeks ago.

/Ihar

On 02/21/2015 01:41 AM, Ivar Lazzaro wrote:
 Hi Ihar,
 
 That was missed for the Juno release, I'll post a patch on master
 and then backport it to stable/juno.
 
 Thanks for catching that, Ivar.
 
 On Fri Feb 20 2015 at 11:06:11 AM Ihar Hrachyshka
 ihrac...@redhat.com mailto:ihrac...@redhat.com wrote:
 
 Hi,
 
 does anyone know why we don't maintain an entry point for APIC
 agent in setup.cfg? The code in [1] looks like there is a main()
 function for the agent, but for some reason it's not exposed to
 any console_script during installation of neutron.
 
 Is there any reason not to do it?
 
 [1]: 
 http://git.openstack.org/cgit/openstack/neutron/tree/__neutron__/plugins/ml2/drivers/__cisco/__apic/apic_topology.py#__n320

 
http://git.openstack.org/cgit/openstack/neutron/tree/neutron/plugins/ml2/drivers/cisco/apic/apic_topology.py#n320
 
 /Ihar
 
 __

 
OpenStack Development Mailing List (not for usage questions)
 Unsubscribe: 
 OpenStack-dev-request@lists.__op__enstack.org?subject:__unsubscrib__e
 http://openstack-dev-requ...@lists.openstack.org?subject:unsubscribe

 
http://lists.openstack.org/__cgi__-bin/mailman/listinfo/__openstac__k-dev
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

 
 
 
 __

 
OpenStack Development Mailing List (not for usage questions)
 Unsubscribe:
 openstack-dev-requ...@lists.openstack.org?subject:unsubscribe 
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
 
-BEGIN PGP SIGNATURE-
Version: GnuPG v1

iQEcBAEBAgAGBQJU6xqrAAoJEC5aWaUY1u57GXYIAMAiXzgvK/QCHmaQtp4YWMvJ
SQeG7Ql/Yv/j7ew1a5sLeQvI/9qjsmFJ4l2ePAYPxuw5VMENISguCn/5HVIUPIru
iuuS9jvQXbIM5X0ug8y6OVDpKEiSPYFHG05HZ/atsH5D2UjdFF24SaHa9IWJyo4y
mzH/QIkGuo/hjIxfcwm0vWB7+5axrHGLu5Km8PFxQADGbii7IJcVU3BsFdvjyZld
LWDwKESwN3KTj7a9yGssDIZ8q5/o/8DwHSG/izcBY5C8SHC7iH2tnsY5F0/p56B6
Sy+IFh/pSffAfXevJhrqKawlWH4GfLGytaJ6pHBwI6FOAV6pMDl1bTUY+t96PF0=
=Dach
-END PGP SIGNATURE-

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


[openstack-dev] [Neutron] db-level locks, non-blocking algorithms, active/active DB clusters and IPAM

2015-02-23 Thread Salvatore Orlando
Lazy-Stacker summary:
I am doing some work on Neutron IPAM code for IP Allocation, and I need to
found whether it's better to use db locking queries (SELECT ... FOR UPDATE)
or some sort of non-blocking algorithm.
Some measures suggest that for this specific problem db-level locking is
more efficient even when using multi-master DB clusters, which kind of
counters recent findings by other contributors [2]... but also backs those
from others [7].

The long and boring thing:

In the past few months there's been a fair amount of discussion concerning
the use of locking queries (ie: SELECT ... FOR UPDATE) in various OpenStack
projects, especially in connection with the fact that this kind of queries
is likely to trigger write set certification failures in synchronous db
replication engines such as MySQL Galera - as pointed out in [1] and
thoroughly analysed in [2].
Neutron, in particular, uses this construct often - and the correctness of
the whole IPAM logic depends on it. On this regard Neutron is also guilty
of not implementing any sort of retry mechanism. Eugene, Rossella, and
others have been working towards fixing this issue. Some examples of their
work can be found at [3] and [4].

However, techniques such as compare and swap described in [2] can be used
for implementing non-blocking algorithms which avoid at all the write-set
certification issue.
A neutron example of such technique can be found in [5]. However, a bug in
this process found by Attila [6], and the subsequent gerrit discussion [7],
raised further questions on locking vs non-blocking solutions.
At this stage, we know that using database-level locks triggers write-set
certification failures in synchronous multi master modes. This failures are
reported as deadlock errors. Such errors can be dealt with retrying the
transaction. However, this is claimed as not efficient in [2], whereas
Attila in [7] argues the opposite.

As I'm rewriting Neutron's IPAM logic as a part of [8], which heavily
relies on this construct, I decided to spend some time to put everything to
the test.
To this aim I devised 3 algorithms.

A-1) The classic one which locks the availability ranges for a subnet until
allocation is complete, augmented with a simple and brutal retry mechanism
[9].
A-2) A wait-free 2 step process, in which the first one creates the IP
allocation entry, and the second completes it by adjusting availability
ranges [10]. Primary key violations will trigger retries.
A-3) A wait-free, lock-free 3 step process [11], which adopts
compare-and-swap [2] and the same election process as the bully algorithm
[12].

Unfortunately neutron does not create IP records for every address in a
subnet's CIDR - this would be fairly bad for IPv6 networks. Therefore we
cannot simply apply the simple CAS technique introduced in [2].

I then tested them under arbitrary concurrent load from multiple workers.
The tests were performed first on a single node backend, and then a 3-node
mysql Galera cluster, installed and configured using the official
documentation [13].
The single-node tests showed, as it might have been expected, that the
algorithm leveraging db-level locks is way more efficient [14]
While it's pointless discussing the full results, one important aspect here
is that with db-level locks are a lot more gentle with the database, and
this result in consistently faster execution times.
With 20 concurrent threads, every thread completed in about 0.06 seconds
with db-level locks (A-1). With A-2 it took about 0.45 seconds, and with
A-3 0.32. With A-2 we saw over 500 queries, a little more than 200 with
A-3, and just a bit more than 100 with A-1.
It's worth noting that what makes A-3 faster than A-2 is a random
allocation strategy which drastically reduces collisions. A-3's performance
with sequential allocations are much worse (the avg. thread runtime with 20
concurrent threads is about 0.75 seconds)

With the test on the Galera cluster I was expecting a terrible slowdown in
A-1 because of deadlocks caused by certification failures. I was extremely
disappointed that the slowdown I measured however does not make any of the
other algorithms a viable alternative.
On the Galera cluster I did not run extensive collections for A-2. Indeed
primary key violations seem to triggers db deadlock because of failed write
set certification too (but I have not yet tested this).
I run tests with 10 threads on each node, for a total of 30 workers. Some
results are available at [15]. There was indeed a slow down in A-1 (about
20%), whereas A-3 performance stayed pretty much constant. Regardless, A-1
was still at least 3 times faster than A-3.
As A-3's queries are mostly select (about 75% of them) use of caches might
make it a lot faster; also the algorithm is probably inefficient and can be
optimised in several areas. Still, I suspect it can be made faster than
A-1. At this stage I am leaning towards adoption db-level-locks with
retries for Neutron's IPAM. However, since I never trust 

[openstack-dev] [Fuel][Nova][Cinder] Operations: adding new nodes in disabled state, allowed for test tenant only

2015-02-23 Thread Bogdan Dobrelya
+ [Fuel] tag
+ openstack-operators ML

Joe Gordon joe.gordon0 at gmail.com
Thu Dec 4 13:26:59 UTC 2014

On Wed, Dec 3, 2014 at 3:31 PM, Mike Scherbakov mscherbakov at
mirantis.com
wrote:

 Hi all,
 enable_new_services in nova.conf seems to allow add new compute nodes in
 disabled state:


https://github.com/openstack/nova/blob/master/nova/db/sqlalchemy/api.py#L507-L508,
 so it would allow to check everything first, before allowing production
 workloads host a VM on it. I've filed a bug to Fuel to use this by
default
 when we scale up the env (add more computes) [1].

 A few questions:

1. can we somehow enable compute service for test tenant first? So
cloud administrator would be able to run test VMs on the node, and
after
ensuring that everything is fine - to enable service for all tenants



Although there may be more then one way to set this up in nova, this can
definitely be done via nova host aggregates. Put new compute services into
an aggregate that only specific tenants can access (controlled via
scheduler filter).

Looks reasonable, +1 for Nova host aggregates [0].

There is still a question, though, about an enable_new_services
parameter, cinder and other OpenStack services. It is not clear how to
use this parameter from an operator perspective, for example:
1) While deploying or scaling the OpenStack environment, we should set
enable_new_services=false for all services which support it.
2) Once the deploy/scale is done, re-enable disabled services. But how
exactly that should be done?
* Set enable_new_services=True and restart the schedulers and conductor
services? Or API services as well?
* Keep enable_new_services=false in configs, but issue - for nova
exmaple - 'nova-manage service enable ...' commands for added compute
nodes? And what about cinder and other ones (there are no *-manage
service enable commands)?
* Some another way?

And regarding plans for implementing this improvement in Fuel, I believe
for nova computes it could be done as a workaround for the 6.1 release
with the help of enable_new_services configuration parameter and a
separate Fuel post-deploy granular task which should re-enable the
disabled compute services. But this should be re-implemented later with
separate host aggregate for deployment and health checks, see the
related blueprint [1].

[0]
http://docs.openstack.org/havana/config-reference/content/host-aggregates.html
[1] https://blueprints.launchpad.net/fuel/+spec/disable-new-computes


1.
2. What about Cinder? Is there a similar option / ability?
3. What about other OpenStack projects?

 What is your opinion, how we should approach the problem (if there is a
 problem)?

 [1] https://bugs.launchpad.net/fuel/+bug/1398817
 --
 Mike Scherbakov
 #mihgen


-- 
Best regards,
Bogdan Dobrelya,
Skype #bogdando_at_yahoo.com
Irc #bogdando

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [nova][vmware][ironic] Configuring active/passive HA Nova compute

2015-02-23 Thread Gary Kotton


On 2/23/15, 2:05 PM, Matthew Booth mbo...@redhat.com wrote:

On 20/02/15 11:48, Matthew Booth wrote:
 Gary Kotton came across a doozy of a bug recently:
 
 https://bugs.launchpad.net/nova/+bug/1419785
 
 In short, when you start a Nova compute, it will query the driver for
 instances and compare that against the expected host of the the instance
 according to the DB. If the driver is reporting an instance the DB
 thinks is on a different host, it assumes the instance was evacuated
 while Nova compute was down, and deletes it on the hypervisor. However,
 Gary found that you trigger this when starting up a backup HA node which
 has a different `host` config setting. i.e. You fail over, and the first
 thing it does is delete all your instances.
 
 Gary and I both agree on a couple of things:
 
 1. Deleting all your instances is bad
 2. HA nova compute is highly desirable for some drivers
 
 We disagree on the approach to fixing it, though. Gary posted this:
 
 https://review.openstack.org/#/c/154029/
 
 I've already outlined my objections to this approach elsewhere, but to
 summarise I think this fixes 1 symptom of a design problem, and leaves
 the rest untouched. If the value of nova compute's `host` changes, then
 the assumption that instances associated with that compute can be
 identified by the value of instance.host becomes invalid. This
 assumption is pervasive, so it breaks a lot of stuff. The worst one is
 _destroy_evacuated_instances(), which Gary found, but if you scan
 nova/compute/manager for the string 'self.host' you'll find lots of
 them. For example, all the periodic tasks are broken, including image
 cache management, and the state of ResourceTracker will be unusual.
 Worse, whenever a new instance is created it will have a different value
 of instance.host, so instances running on a single hypervisor will
 become partitioned based on which nova compute was used to create them.
 
 In short, the system may appear to function superficially, but it's
 unsupportable.
 
 I had an alternative idea. The current assumption is that the `host`
 managing a single hypervisor never changes. If we break that assumption,
 we break Nova, so we could assert it at startup and refuse to start if
 it's violated. I posted this VMware-specific POC:
 
 https://review.openstack.org/#/c/154907/
 
 However, I think I've had a better idea. Nova creates ComputeNode
 objects for its current configuration at startup which, amongst other
 things, are a map of host:hypervisor_hostname. We could assert when
 creating a ComputeNode that hypervisor_hostname is not already
 associated with a different host, and refuse to start if it is. We would
 give an appropriate error message explaining that this is a
 misconfiguration. This would prevent the user from hitting any of the
 associated problems, including the deletion of all their instances.

I have posted a patch implementing the above for review here:

https://review.openstack.org/#/c/158269/

I have to look at what you have posted. I think that this topic is
something that we should speak about at the summit and this should fall
under some BP and well defined spec. I really would not like to see
existing installations being broken if and when this patch lands. It may
also affect Ironic as it works on the same model.


Matt
-- 
Matthew Booth
Red Hat Engineering, Virtualisation Team

Phone: +442070094448 (UK)
GPG ID:  D33C3490
GPG FPR: 3733 612D 2D05 5458 8A8A 1600 3441 EA19 D33C 3490

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [horizon][heat]Vote for Openstack L summit topic The Heat Orchestration Template Builder: A demonstration

2015-02-23 Thread Aggarwal, Nikunj
I am also looking forward to present it. Please support this by giving your 
vote ☺

Regards,
Nikunj

From: Angus Salkeld [mailto:asalk...@mirantis.com]
Sent: Monday, February 23, 2015 5:53 PM
To: OpenStack Development Mailing List (not for usage questions)
Subject: Re: [openstack-dev] [horizon][heat]Vote for Openstack L summit topic 
The Heat Orchestration Template Builder: A demonstration

On Mon, Feb 23, 2015 at 9:18 PM, Aggarwal, Nikunj 
nikunj.aggar...@hp.commailto:nikunj.aggar...@hp.com wrote:
Hi Angus,
I am working Timur and Drago on HOT builder. I am using this - 
https://github.com/rackerlabs/hotbuilder as the base and made some improvements 
over the existing code and wish to give a demonstration on how easy it is to 
create a HOT template from Horizon.

That's awesome, looking forward to see it!
-Angus


Regards,
Nikunj

From: Angus Salkeld [mailto:asalk...@mirantis.commailto:asalk...@mirantis.com]
Sent: Monday, February 23, 2015 4:52 AM
To: OpenStack Development Mailing List (not for usage questions)
Subject: Re: [openstack-dev] [horizon][heat]Vote for Openstack L summit topic 
The Heat Orchestration Template Builder: A demonstration

On Sat, Feb 21, 2015 at 4:21 AM, Aggarwal, Nikunj 
nikunj.aggar...@hp.commailto:nikunj.aggar...@hp.com wrote:
Hi,

I have submitted  presentations for Openstack L summit :


The Heat Orchestration Template Builder: A 
demonstrationhttps://www.openstack.org/vote-vancouver/Presentation/the-heat-orchestration-template-builder-a-demonstration



Hi
Nice to see work on a HOT builder progressing, but..
are you planning on integrating this with the other HOT builder efforts?
Is the code public (link)?

This is more of a framework to make these easier to build:
https://github.com/stackforge/merlin
https://wiki.openstack.org/wiki/Merlin

Timur (who works on Merlin) is working with Rackers to build this upstream - I 
am not sure on the progress.
https://github.com/rackerlabs/hotbuilder
https://github.com/rackerlabs/foundry

It would be nice if we could all work together (I am hoping you are already)?
Hopefully some of the others that are working on this can chip in and say where 
they
are.

-Angus


Please cast your vote if you feel it is worth for presentation.

Thanks  Regards,
Nikunj


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: 
openstack-dev-requ...@lists.openstack.org?subject:unsubscribehttp://openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: 
openstack-dev-requ...@lists.openstack.org?subject:unsubscribehttp://openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [nova][vmware][ironic] Configuring active/passive HA Nova compute

2015-02-23 Thread Matthew Booth
On 23/02/15 12:13, Gary Kotton wrote:
 
 
 On 2/23/15, 2:05 PM, Matthew Booth mbo...@redhat.com wrote:
 
 On 20/02/15 11:48, Matthew Booth wrote:
 Gary Kotton came across a doozy of a bug recently:

 https://bugs.launchpad.net/nova/+bug/1419785

 In short, when you start a Nova compute, it will query the driver for
 instances and compare that against the expected host of the the instance
 according to the DB. If the driver is reporting an instance the DB
 thinks is on a different host, it assumes the instance was evacuated
 while Nova compute was down, and deletes it on the hypervisor. However,
 Gary found that you trigger this when starting up a backup HA node which
 has a different `host` config setting. i.e. You fail over, and the first
 thing it does is delete all your instances.

 Gary and I both agree on a couple of things:

 1. Deleting all your instances is bad
 2. HA nova compute is highly desirable for some drivers

 We disagree on the approach to fixing it, though. Gary posted this:

 https://review.openstack.org/#/c/154029/

 I've already outlined my objections to this approach elsewhere, but to
 summarise I think this fixes 1 symptom of a design problem, and leaves
 the rest untouched. If the value of nova compute's `host` changes, then
 the assumption that instances associated with that compute can be
 identified by the value of instance.host becomes invalid. This
 assumption is pervasive, so it breaks a lot of stuff. The worst one is
 _destroy_evacuated_instances(), which Gary found, but if you scan
 nova/compute/manager for the string 'self.host' you'll find lots of
 them. For example, all the periodic tasks are broken, including image
 cache management, and the state of ResourceTracker will be unusual.
 Worse, whenever a new instance is created it will have a different value
 of instance.host, so instances running on a single hypervisor will
 become partitioned based on which nova compute was used to create them.

 In short, the system may appear to function superficially, but it's
 unsupportable.

 I had an alternative idea. The current assumption is that the `host`
 managing a single hypervisor never changes. If we break that assumption,
 we break Nova, so we could assert it at startup and refuse to start if
 it's violated. I posted this VMware-specific POC:

 https://review.openstack.org/#/c/154907/

 However, I think I've had a better idea. Nova creates ComputeNode
 objects for its current configuration at startup which, amongst other
 things, are a map of host:hypervisor_hostname. We could assert when
 creating a ComputeNode that hypervisor_hostname is not already
 associated with a different host, and refuse to start if it is. We would
 give an appropriate error message explaining that this is a
 misconfiguration. This would prevent the user from hitting any of the
 associated problems, including the deletion of all their instances.

 I have posted a patch implementing the above for review here:

 https://review.openstack.org/#/c/158269/
 
 I have to look at what you have posted. I think that this topic is
 something that we should speak about at the summit and this should fall
 under some BP and well defined spec. I really would not like to see
 existing installations being broken if and when this patch lands. It may
 also affect Ironic as it works on the same model.

This patch will only affect installations configured with multiple
compute hosts for a single hypervisor. These are already broken, so this
patch will at least let them know if they haven't already noticed.

It won't affect Ironic, because they configure all compute hosts to have
the same 'host' value. An Ironic user would only notice this patch if
they accidentally misconfigured it, which is the intended behaviour.

Incidentally, I also support more focus on the design here. Until we
come up with a better design, though, we need to do our best to prevent
non-trivial corruption from a trivial misconfiguration. I think we need
to merge this, or something like it, now and still have a summit discussion.

Matt
-- 
Matthew Booth
Red Hat Engineering, Virtualisation Team

Phone: +442070094448 (UK)
GPG ID:  D33C3490
GPG FPR: 3733 612D 2D05 5458 8A8A 1600 3441 EA19 D33C 3490

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [nova] Priority tracking

2015-02-23 Thread Gary Kotton
Just for a few clarifications:

  1.  The link below is for the nova priority tracking
  2.  It was requested that we do not add bug fixes to that list. There are 
other tracking tools for that
  3.  The list does have a Quick/Trivial Hit Bugs. Please read the 
description before you start to add to that list. Feel free to reach out to the 
guys who regularly contribute to this list if you feel that you want to dive in

A luta continua
Gary

From: Gary Kotton gkot...@vmware.commailto:gkot...@vmware.com
Reply-To: OpenStack List 
openstack-dev@lists.openstack.orgmailto:openstack-dev@lists.openstack.org
Date: Monday, February 23, 2015 at 2:19 PM
To: OpenStack List 
openstack-dev@lists.openstack.orgmailto:openstack-dev@lists.openstack.org
Subject: [openstack-dev] [nova] Priority tracking

Hi,
Not sure if everyone is aware of this. There is a ether pad where the priority 
of the project appears: 
https://etherpad.openstack.org/p/kilo-nova-priorities-tracking. This has the 
major bullets for the K cycle.
So in short - if you reviews are not on the page they may not get review cycles 
:)
Thanks
Gary
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


[openstack-dev] [nova] Priority tracking

2015-02-23 Thread Gary Kotton
Hi,
Not sure if everyone is aware of this. There is a ether pad where the priority 
of the project appears: 
https://etherpad.openstack.org/p/kilo-nova-priorities-tracking. This has the 
major bullets for the K cycle.
So in short - if you reviews are not on the page they may not get review cycles 
:)
Thanks
Gary
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [nova] Outcome of the nova FFE meeting for Kilo

2015-02-23 Thread Nikola Đipanov
On 02/20/2015 11:33 PM, Sourabh Patwardhan (sopatwar) wrote:
 Nova core reviewers,
 
 May I request an FFE for Cisco VIF driver:
 https://review.openstack.org/#/c/157616/
 
 This is a small isolated change similar to the vhostuser / open contrail
 vif drivers for which FFE has been granted.
 
 Thanks,
 Sourabh
 
 

Hey Sourabh,

Sorry that you didn't get any responses sooner.

Actually the FFEs get decided by a subset of the Nova core team called
the nova-drivers. You can see it briefly mentioned here [1]. (*)

You can see an ethepad that hosts the minutes of the meeting where
nova-drivers were deciding on FFEs at [2] which may give you more
insight into why your BP did not make the cut.

Once again - apologies for any poor experience you may have had trying
to contribute to Nova.

N.

[1] https://wiki.openstack.org/wiki/Nova
[2] https://etherpad.openstack.org/p/kilo-nova-ffe-requests

(*) On a side note this is an example of us not making the process and
the players clear to our contributors, so we should probably try to
document the role of the nova-drivers better at the very least.

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] A question about strange behavior of oslo.config in eclipse

2015-02-23 Thread Doug Hellmann


On Mon, Feb 23, 2015, at 01:29 AM, Joshua Zhang wrote:
 Hi Doug,
 
  I can't find above error when using the latest codes (2015-02-22),
  but
 new error occur ( see [1] ).  I feel it has no concern with the
 code(ConfigOpts.import_opt), the same code can be run in both bash and
 pycharm, it just didn't work in eclipse+pydev. It looks like there are
 some
 conflicts between oslo/monkey patch and pydev. You are python expert,
 could
 you give me some input by the following traceback ?

Unfortunately, I don't know much about eclipse or pydev, so I'm not sure
how much help I can be. The traceback is saying that the MultiString
class is missing from oslo_config.types. Is that true? Do you somehow
have a version of the file without that class? Is there another version
of oslo.config installed elsewhere that could be interfering with the
imports?

 
  Another question,  'Gevent compatible debugging' feature both in
 eclipse and pycharm doesn't work, changing 'thread=False' for monkey
 patch
 may cause the error [2], so now I have to get back to use pdb to debug
 openstack. Could you have some idea to make oslo/monkey patch is more
 friendly for IDE ? many thanks.

This is outside of my area of expertise. Maybe another pydev user can
share information about how they have configured it?

 
 *[1]*, traceback when running 'neutron-server --config-file
 /etc/neutron/neutron.conf --config-file
 /etc/neutron/plugins/ml2/ml2_conf.ini' in eclipse+pydev env. while we can
 run this command well in bash and pycharm.
 
 Traceback (most recent call last):
   File /usr/local/bin/neutron-server, line 9, in module
 load_entry_point('neutron==2015.1.dev202', 'console_scripts',
 'neutron-server')()
   File
   /usr/local/lib/python2.7/dist-packages/pkg_resources/__init__.py,
 line 521, in load_entry_point
 return get_distribution(dist).load_entry_point(group, name)
   File
   /usr/local/lib/python2.7/dist-packages/pkg_resources/__init__.py,
 line 2632, in load_entry_point
 return ep.load()
   File
   /usr/local/lib/python2.7/dist-packages/pkg_resources/__init__.py,
 line 2312, in load
 return self.resolve()
   File
   /usr/local/lib/python2.7/dist-packages/pkg_resources/__init__.py,
 line 2318, in resolve
 module = __import__(self.module_name, fromlist=['__name__'], level=0)
   File /bak/openstack/neutron/neutron/cmd/eventlet/server/__init__.py,
 line 13, in module
 from neutron import server
   File /bak/openstack/neutron/neutron/server/__init__.py, line 26, in
 module
 from neutron.common import config
   File /bak/openstack/neutron/neutron/common/config.py, line 25, in
 module
 import oslo_messaging
   File
   /usr/local/lib/python2.7/dist-packages/oslo_messaging/__init__.py,
 line 18, in module
 from .notify import *
   File
 /usr/local/lib/python2.7/dist-packages/oslo_messaging/notify/__init__.py,
 line 23, in module
 from .notifier import *
   File
 /usr/local/lib/python2.7/dist-packages/oslo_messaging/notify/notifier.py,
 line 32, in module
 help='Driver or drivers to handle sending notifications.'),
   File /usr/local/lib/python2.7/dist-packages/oslo_config/cfg.py, line
 1108, in __init__
 item_type=types.MultiString(),
 AttributeError: 'module' object has no attribute 'MultiString'
 
 *[2] *
 
 2015-02-22 14:56:24.117 ERROR oslo_messaging.rpc.dispatcher
 [req-65501748-3db5-43b6-b00e-732565d2192a TestNetworkVPNaaS-1393357785
 TestNetworkVPNaaS-1147259963] Exception during message handling:
 _oslo_messaging_localcontext_9bb7d928d1a042e085f354eb118e98a0
 2015-02-22 14:56:24.117 28042 TRACE oslo_messaging.rpc.dispatcher
 Traceback
 (most recent call last):
 2015-02-22 14:56:24.117 28042 TRACE oslo_messaging.rpc.dispatcher   File
 /usr/local/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py,
 line 142, in _dispatch_and_reply
 2015-02-22 14:56:24.117 28042 TRACE oslo_messaging.rpc.dispatcher
 executor_callback))
 2015-02-22 14:56:24.117 28042 TRACE oslo_messaging.rpc.dispatcher   File
 /usr/local/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py,
 line 188, in _dispatch
 2015-02-22 14:56:24.117 28042 TRACE oslo_messaging.rpc.dispatcher
 localcontext.clear_local_context()
 2015-02-22 14:56:24.117 28042 TRACE oslo_messaging.rpc.dispatcher   File
 /usr/local/lib/python2.7/dist-packages/oslo_messaging/localcontext.py,
 line 55, in clear_local_context
 2015-02-22 14:56:24.117 28042 TRACE oslo_messaging.rpc.dispatcher
 delattr(_STORE, _KEY)
 2015-02-22 14:56:24.117 28042 TRACE oslo_messaging.rpc.dispatcher
 AttributeError:
 _oslo_messaging_localcontext_9bb7d928d1a042e085f354eb118e98a0
 
 
 
 On Sat, Feb 14, 2015 at 7:05 AM, Doug Hellmann d...@doughellmann.com
 wrote:
 
 
 
  On Thu, Feb 12, 2015, at 07:19 AM, Joshua Zhang wrote:
   Hi Doug,
  
  Thank you very much for your reply. I don't have any codes, so no any
   special codes as well.
  Only thing I did is that:
  1, use devstack to install a fresh openstack env, all are ok.
  2, 

Re: [openstack-dev] [neutron] New version of python-neutronclient release: 2.3.11

2015-02-23 Thread Doug Hellmann


On Mon, Feb 23, 2015, at 07:17 AM, Ihar Hrachyshka wrote:
 -BEGIN PGP SIGNED MESSAGE-
 Hash: SHA1
 
 On 02/20/2015 11:01 PM, Matt Riedemann wrote:
  
  
  On 2/20/2015 6:23 AM, Alan Pevec wrote:
  https://review.openstack.org/#/c/157535/
  
  Do you know where it ends ? We could set up Depends lines on
  those requirements stable/* reviews and line them up so that
  they are ready to merge when openstackclient is fixed in
  devstack.
  
  Alternative workaround is https://review.openstack.org/157654
  which is blocked on swift-dsvm-functional issue fixed by 
  https://review.openstack.org/157670 which is blocked on
  neutronclient i.e. we got a cyclic dep here which will require
  ninja merge to resolve.
  
  I suggest to start with ninja merging 157670 which looks the
  most innocent.
  
  Once we get icehouse working again we can look at backporting
  venv patch series to devstack icehouse.
  
  
  Cheers, Alan
  
  __
 
 
  
 OpenStack Development Mailing List (not for usage questions)
  Unsubscribe: 
  openstack-dev-requ...@lists.openstack.org?subject:unsubscribe 
  http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
 
 
  
  We disabled the swift functional job on stable/icehouse since it
  doesn't have the tox target and shouldn't have been running on
  stable/icehouse:
  
  https://review.openstack.org/#/c/157825/
  
  That freed up Adam's devstack workaround change:
  
  https://review.openstack.org/#/c/157654/
  
  So we can now focus on the neutronclient cap:
  
  https://review.openstack.org/#/c/157606/
  
  That hit a problem with an uncapped pyghmi version in the gate so
  we're also now capping that library as well.
  
 
 UPD: pyghmi cap didn't work, and it seems that we hit a bug in
 'git-diff' that makes it return wrong exit code when used with --quiet
 argument. The workaround is to switch to --exit-code argument:
 
 https://review.openstack.org/158247
 
 Once it's merged, we hit another issue - requirements integration
 tools trying to sync stable/icehouse requirements for pycadf that does
 not maintain a stable branch. The proper fix for this issue is to
 avoid integration run for those projects for stable branches:
 
 https://review.openstack.org/158086

We will also need to start creating stable branches for all libraries
with caps. It might make sense to do that retroactively for pycadf and
any others for our existing stable branches.

Doug

 
 /Ihar
 -BEGIN PGP SIGNATURE-
 Version: GnuPG v1
 
 iQEcBAEBAgAGBQJU6xpTAAoJEC5aWaUY1u57TkgIAKnys1hfuscwDTyErlbsvu6q
 JnKkkHSfnn/so36xCt65r6mq1NeBUF+Wn4CFeoMwh6eIbql4aeXJC0voBwH5ScpL
 HUKZtz2eL088wJfwu5yhn0TQleh1C5GCQf2rutQv1bcqdwhRbo1MLf0Dda34dJSg
 HL6gNfYTm3TckgNvWujNxGOluEUg7It0a/myf9Hn0xDhyLCvdF9+7DMzvotmEai+
 HuoGhaFBGjyEYgS/aCRjg0c0qtBxAk4pITQwuIvUzkDWLYQLuW6S4pzaySGDXxP0
 zVbqMoReUtyCv1f6t9k43K+K1nj2yloBtlzBsUUF04n9YbGlfVqlWtHu2yAc86c=
 =aMhI
 -END PGP SIGNATURE-
 
 __
 OpenStack Development Mailing List (not for usage questions)
 Unsubscribe:
 openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [keystone] [nova]

2015-02-23 Thread Adam Young

On 02/16/2015 05:00 AM, Nikolay Makhotkin wrote:
Well, if we use trust-scoped token for getting server-list from nova 
(simply use nova.servers.list() ),


Novaclient somehow tries to get another token: 
https://github.com/openstack/python-novaclient/blob/master/novaclient/client.py#L690-L724


Actually, novaclient does this request: (found from debug of novaclient)
So this sounds like a bug in Nova client.  Jamie Lennox has been working 
with the various client teams to get the  using the Auth plugin 
architecture and session management from Keystone client to try and make 
the usage consistent.






REQ: curl -i 'http://my_host:5000/v2.0/tokens' -X POST -H Accept: 
application/json -H Content-Type: application/json -H User-Agent: 
python-novaclient -d '{auth: {token: {id: 
78c71fb549244075b3a5d994baa326b3}, tenantName: 
b0c9bbb541d541b098c3c0a92412720d}}'


I.e., this is the request for another auth token from keystone. 
Keystone here returns 403 because token in request is trust-scoped.


Why I can't do this simple command using trust-scoped token?

Note: Doing the keystone --os-token 5483086d91094a3886ccce1442b538a0 
--os-endpoint http://my_host:5000/v2.0 tenant-list, it returns 
tenant-list (not 403).
Note2: Doing the server-list request directly to api with trust-scoped 
token, it returns 200, not 403:


curl -H X-Auth-Token: 5483086d91094a3886ccce1442b538a0 
http://192.168.0.2:8774/v3/servers


{
servers: [ list_of_servers ]
}

How I can use trust-scoped tokrn via client?

On Fri, Feb 13, 2015 at 9:16 PM, Alexander Makarov 
amaka...@mirantis.com mailto:amaka...@mirantis.com wrote:


Adam, Nova client does it for some reason during a call to
nova.servers.list()


On Thu, Feb 12, 2015 at 10:03 PM, Adam Young ayo...@redhat.com
mailto:ayo...@redhat.com wrote:

On 02/12/2015 10:40 AM, Alexander Makarov wrote:

A trust token cannot be used to get another token:

https://github.com/openstack/keystone/blob/master/keystone/token/controllers.py#L154-L156
You have to make your Nova client use the very same trust
scoped token obtained from authentication using trust without
trying to authenticate with it one more time.



Actually, there have been some recent changes to allow
re-delegation of Trusts, but for older deployments, you are
correct.  I hadn't seen anywhere here that he was trying to
use a trust token to get another token, though.




On Wed, Feb 11, 2015 at 9:10 PM, Adam Young
ayo...@redhat.com mailto:ayo...@redhat.com wrote:

On 02/11/2015 12:16 PM, Nikolay Makhotkin wrote:

No, I just checked it. Nova receives trust token and
raise this error.

In my script, I see:

http://paste.openstack.org/show/171452/

And as you can see, token from trust differs from direct
user's token.


The original user needs to have the appropriate role to
perform the operation on the specified project.  I see
the admin role is created on the trust. If the trustor
did not have that role, the trustee would not be able to
exececute the trust and get a token.  It looks like you
were able to execute the trust and get a token,  but I
would like you to confirm that, and not just trust the
keystone client:  either put debug statements in Keystone
or call the POST to tokens from curl with the appropriate
options to get a trust token.  In short, make sure you
have not fooled yourself.  You can also look in the token
table inside Keystone to see the data for the trust
token, or validate the token  via curl to see the data in
it.  In all cases, there should be an OS-TRUST stanza in
the token data.


If it is still failing, there might be some issue on the
Policy side.  I have been assuming that you are running
with the default policy for Nova.


http://git.openstack.org/cgit/openstack/nova/tree/etc/nova/policy.json

I'm not sure which rule matches for list servers (Nova
developer input would be appreciated)  but I'm guessing
it is executing the rule
|
admin_or_owner: is_admin:True or
project_id:%(project_id)s,

Since that is the default. I am guessing that the
project_id in question comes from the token here, as that
seems to be common, but if not, it might be that the two
values are mismatched. Perhaps there Proejct ID value
from the client env var is sent, and matches what the
trustor normally works as, not the project in question. 
If these two values don't match, then, yes, the rule

would fail.
|





On Wed, Feb 11, 

[openstack-dev] [mistral] Cancelling team meeting today on 02/23/2015

2015-02-23 Thread Renat Akhmerov
Sorry for the late notice. We’re cancelling our team meeting today. Several 
team members are on official holidays.

Renat Akhmerov
@ Mirantis Inc.




__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [nova] Outcome of the nova FFE meeting for Kilo

2015-02-23 Thread Daniel P. Berrange
On Mon, Feb 23, 2015 at 03:47:20PM +0100, Nikola Đipanov wrote:
 On 02/20/2015 11:33 PM, Sourabh Patwardhan (sopatwar) wrote:
  Nova core reviewers,
  
  May I request an FFE for Cisco VIF driver:
  https://review.openstack.org/#/c/157616/
  
  This is a small isolated change similar to the vhostuser / open contrail
  vif drivers for which FFE has been granted.
 
 Actually the FFEs get decided by a subset of the Nova core team called
 the nova-drivers. You can see it briefly mentioned here [1]. (*)
 
 You can see an ethepad that hosts the minutes of the meeting where
 nova-drivers were deciding on FFEs at [2] which may give you more
 insight into why your BP did not make the cut.

Unfortunately the Cisco VIF driver was not on the list of features
considered in the first place, since (AFAICT) this FFE request was
only made after the FFEs had all been decided for Kilo.

Personally I think the Nova FFE process is too inflexible, and thus
I would support inclusion of this (and any) VIF driver into Kilo
regardless of FFE status for pragmatic reasons, but this appears to
be a minority view right now.

 (*) On a side note this is an example of us not making the process and
 the players clear to our contributors, so we should probably try to
 document the role of the nova-drivers better at the very least.

As a member of Nova drivers, I personally think that the team should
never have existed in the first place, and should cease to exist at
the soonest opportunity. That it exists at all is just an accident
of history due the use of launchpad for blueprint approval. IMHO it
makes no sense to restrict which cores can participate in feature
review  approval - any Nova core should be able to be choose to
participate as their time allows, without needing to be granted
access to yet another special group.

Regards,
Daniel
-- 
|: http://berrange.com  -o-http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org  -o- http://virt-manager.org :|
|: http://autobuild.org   -o- http://search.cpan.org/~danberr/ :|
|: http://entangle-photo.org   -o-   http://live.gnome.org/gtk-vnc :|

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [nova] Libguestfs: possibility not to use it, even when installed ?

2015-02-23 Thread Richard W.M. Jones
On Mon, Feb 23, 2015 at 11:08:31AM +0100, Raphael Glon wrote:
 sudo sysctl -w fs.protected_hardlinks=0 + common user nova/qemu

We fixed this a while back (in July 2013 in fact).

I think if you forked at Fedora 19 then you're probably using
libguestfs 1.24 + supermin 4.  I'd definitely recommend updating to at
least libguestfs = 1.26 + supermin 5 since that fixes a bunch of bugs
around building the appliance, and is also faster.

Rich.

-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
virt-df lists disk usage of guests without needing to install any
software inside the virtual machine.  Supports Linux and Windows.
http://people.redhat.com/~rjones/virt-df/

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


[openstack-dev] [Heat] Vote for talk on using Metatemplates for heat

2015-02-23 Thread Pratik Mallya
Hey Folks,

In case you missed it, many of you might find the work we’ve done in maintaing 
catalogs of heat templates interesting.
The proposal is here: 
https://www.openstack.org/vote-vancouver/presentation/one-heat-template-to-rule-them-all

Please take a look if you haven’t already, and vote accordingly :-).

Thanks,
Pratik


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [horizon][heat]Vote for Openstack L summit topic The Heat Orchestration Template Builder: A demonstration

2015-02-23 Thread Yves-Gwenaël Bourhis
We have a library called flame :

https://github.com/cloudwatt/flame
https://pypi.python.org/pypi/python-flameclient/0.1.0

It generates heat templates from existing resources:

http://dev.cloudwatt.com/en/blog/introducing-flame-automatic-heat-template-generation.html

Could it help?

Le 23/02/2015 00:21, Angus Salkeld a écrit :
 On Sat, Feb 21, 2015 at 4:21 AM, Aggarwal, Nikunj
 nikunj.aggar...@hp.com mailto:nikunj.aggar...@hp.com wrote:
 
 Hi, 
 
 __ __
 
 I have submitted  presentations for Openstack L summit :
 
 __ __
 
 The Heat Orchestration Template Builder: A demonstration
 
 https://www.openstack.org/vote-vancouver/Presentation/the-heat-orchestration-template-builder-a-demonstration
 
 __ 
 
 
 Hi
 
 Nice to see work on a HOT builder progressing, but..
 are you planning on integrating this with the other HOT builder efforts?
 Is the code public (link)?
 
 This is more of a framework to make these easier to build:
 https://github.com/stackforge/merlin
 https://wiki.openstack.org/wiki/Merlin
 
 Timur (who works on Merlin) is working with Rackers to build this
 upstream - I am not sure on the progress.
 https://github.com/rackerlabs/hotbuilder
 https://github.com/rackerlabs/foundry
  
 It would be nice if we could all work together (I am hoping you are
 already)?
 Hopefully some of the others that are working on this can chip in and
 say where they
 are.
 
 -Angus
 
 __
 
 __ __
 
 Please cast your vote if you feel it is worth for presentation.
 
 __ __
 
 Thanks  Regards,
 
 Nikunj
 
 __ __
 
 
 __
 OpenStack Development Mailing List (not for usage questions)
 Unsubscribe:
 openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
 http://openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
 
 
 
 
 __
 OpenStack Development Mailing List (not for usage questions)
 Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
 

-- 
Yves-Gwenaël Bourhis

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


[openstack-dev] [neutron] ML2 versus core plugin for OVN

2015-02-23 Thread Ben Pfaff
[branching off a discussion on ovs-dev at this point:
http://openvswitch.org/pipermail/dev/2015-February/051609.html]

On Fri, Feb 20, 2015 at 6:56 PM, Kyle Mestery mest...@mestery.com wrote:
 One thing to keep in mind, this ties somewhat into my response to Russell
 earlier on the decision around ML2 vs. core plugin. If we do ML2, there are
 type drivers for VLAN, VXLAN, and GRE tunnels. There is no TypeDriver for
 STT tunnels upstream now. It's just an item we need on the TODO list if we
 go down the STT tunnel path.

It was suggested to me off-list that this part of the discussion should be on
openstack-dev, so here it is ;-)

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [horizon][keystone]

2015-02-23 Thread Adam Young

On 02/18/2015 12:02 PM, David Chadwick wrote:

I think this GUI is not intuitive to users and therefore should not be
encouraged or supported.
It is a fist hack.  I think you don't mean any gui  just that there 
are some warning flags raised by this design?




If you ask a user what does authenticate via a Discovery Service mean?
I think you will get some very strange answers. The same goes for
Authenticate using Default Protocol. Users will have no idea what that
means.


He's not a native English speaker.  We'll need to craft the text here, 
and also carefully review the nuances.  Then there are the international 
translations




There has been a lot of research into how to support federated
authentication and there is a lot of practical experience across the
academic world from dozens of countries for many years. Most
universities now use federated login on a daily basis. We should use
this experience and follow best practise (which sadly does not involve
the screens that are being proposed here).

If you want to read more you can read a Good Practice Guide here

https://discovery.refeds.org/

It should help you to redesign the login page
THanks for the links.  Very helpful.  Some of that might require some 
changes from Horizon, but Jamie has seen those issues also come up in 
the context of the Kerberos login, and we can work to make this a 
smoother experience.


David, we certainly could use some UX  feedback here. Unfortunately, my 
GO-TO team member has decided to GO-TO a trip around the world...who can 
we pull in to make this flow in with the rest of Horizon?






regards

David

On 18/02/2015 16:06, Dolph Mathews wrote:

On Fri, Feb 6, 2015 at 12:47 PM, Adam Young ayo...@redhat.com
mailto:ayo...@redhat.com wrote:

 On 02/04/2015 03:54 PM, Thai Q Tran wrote:

 Hi all,

 I have been helping with the websso effort and wanted to get some
 feedback.
 Basically, users are presented with a login screen where they can
 select: credentials, default protocol, or discovery service.
 If user selects credentials, it works exactly the same way it
 works today.
 If user selects default protocol or discovery service, they can
 choose to be redirected to those pages.

 Keep in mind that this is a prototype, early feedback will be good.
 Here are the relevant patches:
 https://review.openstack.org/#/c/136177/
 https://review.openstack.org/#/c/136178/
 https://review.openstack.org/#/c/151842/

 I have attached the files and present them below:



 Replace the dropdown with a specific link for each protocol type:

 SAML and OpenID  are the only real contenders at the moment, but we
 will not likely have so many that it will clutter up the page.


Agree, but the likelihood that a single IdP will support multiple
protocols is probably low. Keystone certainly supports that from an API
perspective, but I don't think it should be the default UX. Choose a
remote IdP first, and then if *that* IdP supports multiple federation
protocols, present them.
  



 Thanks for doing this.






 __
 OpenStack Development Mailing List (not for usage questions)
 Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe 
mailto:openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


 __
 OpenStack Development Mailing List (not for usage questions)
 Unsubscribe:
 openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
 http://openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev




__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Heat] Vote for talk on using Metatemplates for heat

2015-02-23 Thread Anita Kuno
On 02/23/2015 11:16 AM, Pratik Mallya wrote:
 Hey Folks,
 
 In case you missed it, many of you might find the work we’ve done in 
 maintaing catalogs of heat templates interesting.
 The proposal is here: 
 https://www.openstack.org/vote-vancouver/presentation/one-heat-template-to-rule-them-all
 
 Please take a look if you haven’t already, and vote accordingly :-).
 
 Thanks,
 Pratik
 
 
 
 
 
 __
 OpenStack Development Mailing List (not for usage questions)
 Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
 
Please don't spam the list with vote for me requests like this.

The community knows how to evaluate talk proposals and vote. If they
don't they are welcome to ask in #openstack-dev if they need support.

Posts like this don't add to our overall community benefit or discussion.

Thank you,
Anita.

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [stable][requirements] External dependency caps introduced in 499db6b

2015-02-23 Thread Joe Gordon
On Mon, Feb 23, 2015 at 8:49 AM, Ihar Hrachyshka ihrac...@redhat.com
wrote:

 -BEGIN PGP SIGNED MESSAGE-
 Hash: SHA1

 On 02/20/2015 07:16 PM, Joshua Harlow wrote:
  Sean Dague wrote:
  On 02/20/2015 12:26 AM, Adam Gandelman wrote:
  Its more than just the naming.  In the original proposal,
  requirements.txt is the compiled list of all pinned deps
  (direct and transitive), while
  requirements.inhttp://requirements.in  reflects what people
  will actually use.  Whatever is in requirements.txt affects the
  egg's requires.txt. Instead, we can keep requirements.txt
  unchanged and have it still be the canonical list of
  dependencies, while
  reqiurements.out/requirements.gate/requirements.whatever is an
  upstream utility we produce and use to keep things sane on our
  slaves.
 
  Maybe all we need is:
 
  * update the existing post-merge job on the requirements repo
  to produce a requirements.txt (as it does now) as well the
  compiled version.
 
  * modify devstack in some way with a toggle to have it process
  dependencies from the compiled version when necessary
 
  I'm not sure how the second bit jives with the existing
  devstack installation code, specifically with the libraries
  from git-or-master but we can probably add something to warm
  the system with dependencies from the compiled version prior to
  calling pip/setup.py/etc http://setup.py/etc
 
  It sounds like you are suggesting we take the tool we use to
  ensure that all of OpenStack is installable together in a unified
  way, and change it's installation so that it doesn't do that any
  more.
 
  Which I'm fine with.
 
  But if we are doing that we should just whole hog give up on the
  idea that OpenStack can be run all together in a single
  environment, and just double down on the devstack venv work
  instead.
 
  It'd be interesting to see what a distribution (canonical,
  redhat...) would think about this movement. I know yahoo! has been
  looking into it for similar reasons (but we are more flexibly then
  I think a packager such as canonical/redhat/debian/... would/culd
  be). With a move to venv's that seems like it would just offload
  the work to find the set of dependencies that work together (in a
  single-install) to packagers instead.
 
  Is that ok/desired at this point?
 

 Honestly, I failed to track all the different proposals. Just saying
 from packager perspective: we absolutely rely on requirements.txt not
 being a list of hardcoded values from pip freeze, but present us a
 reasonable freedom in choosing versions we want to run in packaged
 products.


in short the current proposal for stable branches is:

keep requirements.txt as is, except maybe put some upper bounds on the
requirements.

Add requirements.gate to specify the *exact* versions we are gating against
(this would be a full list including all transitive dependencies).


 That's why I asked before we should have caps and not pins.

 /Ihar
 -BEGIN PGP SIGNATURE-
 Version: GnuPG v1

 iQEcBAEBAgAGBQJU61oJAAoJEC5aWaUY1u57T7cIALySnlpLV0tjrsTH2gZxskH+
 zY+L6E/DukFNZsWxB2XSaOuVdVaP3Oj4eYCZ2iL8OoxLrBotiOYyRFH29f9vjNSX
 h++dErBr0SwIeUtcnEjbk9re6fNP6y5Hqhk1Ac+NSxwL75KlS3bgKnJAhLA08MVB
 5xkGRR7xl2cuYf9eylPlQaAy9rXPCyyRdxZs6mNjZ2vlY6hZx/w/Y7V28R/V4gO4
 qsvMg6Kv+3urDTRuJdEsV6HbN/cXr2+o543Unzq7gcPpDYXRFTLkpCRV2k8mnmA1
 pO9W10F1FCQZiBnLk0c6OypFz9rQmKxpwlNUN5MTMF15Et6DOxGBxMcfr7TpRaQ=
 =WHOH
 -END PGP SIGNATURE-

 __
 OpenStack Development Mailing List (not for usage questions)
 Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [horizon][keystone]

2015-02-23 Thread David Chadwick
Hi Adam

there is some work being done on this by HP, Intel and IBM, and they
have some designs at

http://invis.io

pieter.c.kruithof...@hp.com  can send you the details as he invited me
to comment on the designs, which I have done.

As you know, we already have our own federated Horizon login screen, and
this summer I am hoping to have another student integrate this into the
current release, as well as make some enhancements to it (like
type-ahead searching for IdPs)

regards

David

On 23/02/2015 15:54, Adam Young wrote:
 On 02/18/2015 12:02 PM, David Chadwick wrote:
 I think this GUI is not intuitive to users and therefore should not be
 encouraged or supported.
 It is a fist hack.  I think you don't mean any gui  just that there
 are some warning flags raised by this design?
 

 If you ask a user what does authenticate via a Discovery Service mean?
 I think you will get some very strange answers. The same goes for
 Authenticate using Default Protocol. Users will have no idea what that
 means.
 
 He's not a native English speaker.  We'll need to craft the text here,
 and also carefully review the nuances.  Then there are the international
 translations
 

 There has been a lot of research into how to support federated
 authentication and there is a lot of practical experience across the
 academic world from dozens of countries for many years. Most
 universities now use federated login on a daily basis. We should use
 this experience and follow best practise (which sadly does not involve
 the screens that are being proposed here).

 If you want to read more you can read a Good Practice Guide here

 https://discovery.refeds.org/

 It should help you to redesign the login page
 THanks for the links.  Very helpful.  Some of that might require some
 changes from Horizon, but Jamie has seen those issues also come up in
 the context of the Kerberos login, and we can work to make this a
 smoother experience.
 
 David, we certainly could use some UX  feedback here. Unfortunately, my
 GO-TO team member has decided to GO-TO a trip around the world...who can
 we pull in to make this flow in with the rest of Horizon?
 
 
 

 regards

 David

 On 18/02/2015 16:06, Dolph Mathews wrote:
 On Fri, Feb 6, 2015 at 12:47 PM, Adam Young ayo...@redhat.com
 mailto:ayo...@redhat.com wrote:

  On 02/04/2015 03:54 PM, Thai Q Tran wrote:
  Hi all,

  I have been helping with the websso effort and wanted to get some
  feedback.
  Basically, users are presented with a login screen where they can
  select: credentials, default protocol, or discovery service.
  If user selects credentials, it works exactly the same way it
  works today.
  If user selects default protocol or discovery service, they can
  choose to be redirected to those pages.

  Keep in mind that this is a prototype, early feedback will be
 good.
  Here are the relevant patches:
  https://review.openstack.org/#/c/136177/
  https://review.openstack.org/#/c/136178/
  https://review.openstack.org/#/c/151842/

  I have attached the files and present them below:


  Replace the dropdown with a specific link for each protocol type:

  SAML and OpenID  are the only real contenders at the moment, but we
  will not likely have so many that it will clutter up the page.


 Agree, but the likelihood that a single IdP will support multiple
 protocols is probably low. Keystone certainly supports that from an API
 perspective, but I don't think it should be the default UX. Choose a
 remote IdP first, and then if *that* IdP supports multiple federation
 protocols, present them.
  

  Thanks for doing this.





 
 __

  OpenStack Development Mailing List (not for usage questions)
  Unsubscribe:
 openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
 mailto:openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
  http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

 
 __

  OpenStack Development Mailing List (not for usage questions)
  Unsubscribe:
  openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
 
 http://openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
  http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev




 __

 OpenStack Development Mailing List (not for usage questions)
 Unsubscribe:
 openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

 __

 OpenStack Development Mailing List (not for usage questions)
 Unsubscribe:
 openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
 

Re: [openstack-dev] [nova] bp serial-ports *partly* implemented?

2015-02-23 Thread Sahid Orentino Ferdjaoui
On Fri, Feb 20, 2015 at 06:03:46PM +0100, Markus Zoeller wrote:
 It seems to me that the blueprint serial-ports[1] didn't implement
 everything which was described in its spec. If one of you could have a 
 look at the following examples and help me to understand if these 
 observations are right/wrong that would be great.
 
 Example 1:
 The flavor provides the extra_spec hw:serial_port_count and the image
 the property hw_serial_port_count. This is used to decide how many
 serial devices (with different ports) should be defined for an instance.
 But the libvirt driver returns always only the *first* defined port 
 (see method get_serial_console [2]). I didn't find anything in the 
 code which uses the other defined ports.

The method you are referencing [2] is used to return the first well
binded and not connected port in the domain.

When defining the domain '{hw_|hw:}serial_port_count' are well take
into account as you can see:

   
https://github.com/openstack/nova/blob/master/nova/virt/libvirt/driver.py#L3702

(The method looks to have been refactored and include several parts
not related to serial-console)

 Example 2:
 If a user is already connected, then reject the attempt of a second
 user to access the console, but have an API to forceably disconnect
 an existing session. This would be particularly important to cope
 with hung sessions where the client network went away before the
 console was cleanly closed. [1]
 I couldn't find the described API. If there is a hung session one cannot
 gracefully recover from that. This could lead to a bad UX in horizons
 serial console client implementation[3].

This API is not implemented, I will see what I can do on that
part. Thanks for this.

 [1] nova bp serial-ports;
 
 https://github.com/openstack/nova-specs/blob/master/specs/juno/implemented/serial-ports.rst
 [2] libvirt driver; return only first port; 
 
 https://github.com/openstack/nova/blob/master/nova/virt/libvirt/driver.py#L2518
 [3] horizon bp serial-console; 
 https://blueprints.launchpad.net/horizon/+spec/serial-console
 
 
 __
 OpenStack Development Mailing List (not for usage questions)
 Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [nova] Outcome of the nova FFE meeting for Kilo

2015-02-23 Thread Dan Smith
 What's the alternative proposed release model?

I deleted a couple paragraphs of the above before sending this, thinking
(like Joe) that there probably needs to be a specific discussion aimed
at this topic. But:

 What's the compatibility story with Glance / Neutron / Cinder in
 whatever that model is?

I think we end up making the tight integrations where we need to.
Presumably Nova and Neutron still need to coordinate tightly. I'm not
sure Nova and Glance do at this point.

 What's the sliding upgrade window look like?

Right now we're maintaining some level of RPC compatibility to a point
far earlier than Juno. The six month releases give us the *only* points
at which we can (currently, by our existing rules) break compatibility.
Presumably we just need to define the minimum timeframe, and then
iterate our interfaces when it makes sense (and fits a policy). Maybe
that's six months; maybe longer, maybe shorter.

 What's the stable maint story look like? the security fix story?

Aren't you the one that wants to get rid of stable-maint? How we handle
security issues is certainly something worthy of a lot of discussion. If
the community only maintains a single stream, then the answer is move
forward. If the only people maintaining slower-moving streams are
vendors or distros then I'd say the answer is ask your vendor.

 I get being frustrated with a thing, but there are a lot of
 considerations before redoing the release cycle.

Well, I'll refer to my original words:

 On 02/23/2015 03:45 PM, Dan Smith wrote:
 I've been wondering this myself for quite a while now. I'm really
 interested to hear what things would look like in a no-release model.

That's all. Just interested. I do think it's the sane way forward,
long-term.

--Dan



signature.asc
Description: OpenPGP digital signature
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [nova] Outcome of the nova FFE meeting for Kilo

2015-02-23 Thread Jay Pipes

On 02/23/2015 04:02 PM, Sean Dague wrote:

On 02/23/2015 03:45 PM, Dan Smith wrote:

Seriously, what is the point of 6-month releases again? We are a
free-form open source set of projects, with a lot of intelligent
engineers. Why are we stuck using an outdated release model?


I've been wondering this myself for quite a while now. I'm really
interested to hear what things would look like in a no-release model.
I'm sure it would be initially met with a lot of resistance, but I think
that in the end, it makes more sense to move to that sort of model and
let vendors/deployers more flexibly decide when to roll out new stuff
based on what has changed and what they value.


What's the alternative proposed release model?


Release at-will. No pre-determined date-based releases. Smaller list of 
features in each tagged release. Focus on API versioning, not releases.



What's the compatibility story with Glance / Neutron / Cinder in
whatever that model is?


Exactly the same as it is now: everything is based on the server 
supported API version and the python-xxxclient release that understands 
that server API version, and integrating support for that client release 
into Nova. There's really very little different at this point between 
our usage of python-neutronclient and our usage of the requests Python 
library. We should pin to a particular library version that we make use 
of the features from, and upgrade the pinning when we utilize 
functionality that is a newer release.



What's the sliding upgrade window look like?


Whatever we want to make it. :) With our RPC and object versioning 
functionality, we can drop support for some old RPC API and object 
versions after some arbitrary amount of time (check with operators and 
get sign off, then just do it).



What's the stable maint story look like? the security fix story?


stable maintenance should be entirely left up to the distributions, IMO.

Security fix story should be up to the distributions to track when their 
products are patched. Upstream should be concerned with quick fixes into 
the upstream branches and coordinated communication with distributions.



I get being frustrated with a thing, but there are a lot of
considerations before redoing the release cycle.


Understood completely. But I'm curious to see just how much grief we put 
ourselves through in trying to do date-based releases when our 
contributor community just isn't assembled in a way that makes those 
releases seem reasonable.


Best,
-jay

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Fuel] tracking bugs superseded by blueprints

2015-02-23 Thread Mike Scherbakov
Bogdan,
I think we should keep bugs open and not supersed them by blueprint. I
see following reasons for it.

Often, we can find workaround in order to fix the bug. Even if bug
naturally seems to be falling into some blueprint's scope. Then problem is
that when you close the bug, you don't even try to think about workaround -
and project gets shipped with some serious gaps from release to release.

Another issue is that you lose real technical requirements for blueprints.
If you keep bugs open and associated with blueprint, you will pass by bugs
a few times before you deliver blueprint's functionality, and finally close
bugs if code covers bug's case. At least, I'd like it to be so.

Finally, QA and users will keep opening duplicates, as no one will be happy
with won't fix. You can vote for bug (by affecting it), and you can't for
blueprint in LP, unfortunately. This just keeps door open for getting
feedback.

I don't really see any cons of moving bugs into Won't fix state instead.

Examples of bugs which I would certainly avoid putting into Won't fix:
https://bugs.launchpad.net/bugs/1398817 - disable computes by default
during scale up
https://bugs.launchpad.net/fuel/+bug/1422856 - separate /var  /var/log on
master node

Thanks,

On Wed, Feb 18, 2015 at 8:46 PM, Andrew Woodward xar...@gmail.com wrote:

 Bogdan,

 Yes I think tracking the bugs like this would be beneficial. We should
 also link them from the BP so that the imperilmenter can track them. It
 adds related blueprints in the bottom of the right column under the
 subscribers so we probably should also edit the description so that the
 data is easy to see

 On Wed, Feb 18, 2015 at 8:12 AM, Bogdan Dobrelya bdobre...@mirantis.com
 wrote:

 Hello.
 There is inconsistency in the triage process for Fuel bugs superseded by
 blueprints.
 The current approach is to set won't fix status for such bugs.
 But there are some cases we should clarify [0], [1].

 I vote to not track superseded bugs separately and keep them as won't
 fix but update the status back to confirmed in case of regression
 discovered. And if we want to backport an improvement tracked by a
 blueprint (just for an exceptional case) let's assign milestones for
 related bugs.

 If we want to change the triage rules, let's announce that so the people
 won't get confused.

 [0] https://bugs.launchpad.net/fuel/+bug/1383741
 [1] https://bugs.launchpad.net/fuel/+bug/1422856

 --
 Best regards,
 Bogdan Dobrelya,
 Skype #bogdando_at_yahoo.com http://bogdando_at_yahoo.com
 Irc #bogdando



 __
 OpenStack Development Mailing List (not for usage questions)
 Unsubscribe:
 openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev




 --
 Andrew
 Mirantis
 Fuel community ambassador
 Ceph community

 __
 OpenStack Development Mailing List (not for usage questions)
 Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev




-- 
Mike Scherbakov
#mihgen
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Fuel] Python code in fuel-library

2015-02-23 Thread Sebastian Kalinowski
Hi,

I've created a patchset that introduces a script to run Python tests in
fuel-library [1] and a
Jenkins job [2]

I also would like to backport those tests to stable branches to assure that
any fix backported to stable branches will be also
checked.

Best,
Sebastian

[1] https://review.openstack.org/#/c/157319/
[2] https://review.fuel-infra.org/#/c/3810/

2015-02-18 16:01 GMT+01:00 Aleksandr Didenko adide...@mirantis.com:

 Hi,

 I agree that we need a better testing for python tasks/code. There should
 be no problems adding py.test tests into fuel-library CI, we already have
 one [1] up and running. So I'm all in and ready to help with such testing
 implementation.

 [1] https://fuel-jenkins.mirantis.com/job/fuellib_tasks_graph_check/

 Regards,
 Aleksandr

 On Wed, Feb 18, 2015 at 4:02 PM, Vladimir Kuklin vkuk...@mirantis.com
 wrote:

 Hi, Seb

 Very fair point, thank you. We need to add this to our jobs for unittests
 run and syntax check. I am adding Aleksandr Didenko into the loop as he is
 currently working on the similar task.

 On Wed, Feb 18, 2015 at 4:53 PM, Jay Pipes jaypi...@gmail.com wrote:

 On 02/18/2015 04:57 AM, Sebastian Kalinowski wrote:

 Hello Fuelers,

 There is more and more Python code appearing in fuel-library [1] that is
 used in our Puppet manifests. Now, with introduction of Granular
 Deployment feature it could appear more often as
 writing some tasks as a Python script is a nice option.

 First problem that I see is that in some cases this code is getting
 merged without a positive review from a Python developer from Fuel team.
 My proposition of the solution is simple:
 fuel-library core reviewers shouldn't merge such code if there is no a
 +1 from a Python developer from fuel-core group [2].

 Second problem is that there are no automatic tests for this code.
 Testing it manually and by running deployment when that code is used is
 not enough since those scripts could be quite large and complicated and
 some of them are executed in specific situations so it is hard for
 reviewers to check how they will work.
 In fuel-library we already have tests for Puppet modules: [3].
 I suggest that we should introduce similar checks for Python code:
   - there will be one global 'test-requirements.txt' file (if there will
 be a need to, we could introduce more granular split, like per module)
   - py.test [4] will be used as a test runner
   - (optional, but advised) flake8+hacking checks [5] (could be limited
 to just run flake8/pyflakes checks)

 Looking forward to your opinions on those two issues.


 Hi Seba,

 All those suggestions look fine to me. I'd also add to improve the
 documentation on how to write and run Python tests to help out those
 developers who are not as familiar with Python as Ruby or other languages.

 Best,
 -jay

 
 __
 OpenStack Development Mailing List (not for usage questions)
 Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:
 unsubscribe
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev




 --
 Yours Faithfully,
 Vladimir Kuklin,
 Fuel Library Tech Lead,
 Mirantis, Inc.
 +7 (495) 640-49-04
 +7 (926) 702-39-68
 Skype kuklinvv
 45bk3, Vorontsovskaya Str.
 Moscow, Russia,
 www.mirantis.com http://www.mirantis.ru/
 www.mirantis.ru
 vkuk...@mirantis.com



 __
 OpenStack Development Mailing List (not for usage questions)
 Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [nova] Libguestfs: possibility not to use it, even when installed ?

2015-02-23 Thread Raphael Glon

On 02/19/2015 12:45 PM, Richard W.M. Jones wrote:

On Wed, Feb 18, 2015 at 07:23:52PM +0100, Raphael Glon wrote:

I entcountered a similar case more recently on powerkvm 2.1.0
(defect with the libguestfs)

What's the actual bug?  We've worked hard, with IBM, to make
libguestfs work on POWER 7 and POWER 8 systems.  I have full access to
those systems through Red Hat.  If there's a new bug I'm sure we'll be
able to fix it.

Rich.


Hi, thank you for all your answers.

Not saying there are actual bugs (anyway I'm stuck here because i 
would need to find time+environment to recheck all/reproduce) - i 
haven't even deployed juno on pkvm yet


We've talked with ibm (and they have likely been working with you) and 
they are really responsive in fixing defects with their distribution


We've entcountered two problems with powerkvm regarding nova + libguestfs.

1. since pkvm 2.1.x is forked from a Fedo 19, we fell back to this Red 
Hat bug you fixed regarding the attach method


Note that one of the workaround proposed was

sudo sysctl -w fs.protected_hardlinks=0 + common user nova/qemu


- Not a specialist here, but seems like to be able to use libguestfs to 
avoid potential issues with fuse mounts, we open other potential 
holes somewhere else


2. because pkvm 2.1.x is forked from fedo 19 it embeds rather old 
versions of libguestfs and libvirt.


We also entcountered the following issue(as you see from the dates, it 
is rather old). About this, i was not perfectly accurate saying it was 
a libguestfs defect, it was a pkvm defect embedding an old version of 
libguestfs and anyway it also has been fixed quickly


http://paste.ubuntu.com/8465699/

Sep 30 14:25:33 host-power8-1 libvirtd[15894]: Domain id=2 
name='guestfs-xv6mh1nvhr17ktj6' 
uuid=341b09bc-6583-4b72-9df8-dc9b18116303 is tainted: custom-argv
Sep 30 14:25:33 host-power8-1 libvirtd[15894]: Unable to read from 
monitor: Connection reset by peer
Sep 30 14:25:33 host-power8-1 kernel: [  878.869394] 
qemu-system-ppc[16250]: unhandled signal 11 at 00d8 nip 
3070c0ac lr 3070bff4 code 30001
Sep 30 14:25:33 host-power8-1 libvirtd[15894]: cannot lookup default 
selinux label for /tmp/libguestfsrOhcjP/console.sock
Sep 30 14:25:33 host-power8-1 libvirtd[15894]: cannot lookup default 
selinux label for /tmp/libguestfsrOhcjP/guestfsd.sock
Sep 30 14:25:33 host-power8-1 libvirtd[15894]: cannot lookup default 
selinux label for /var/tmp/.guestfs-0/kernel.16152
Sep 30 14:25:33 host-power8-1 libvirtd[15894]: cannot lookup default 
selinux label for /var/tmp/.guestfs-0/initrd.16152



3. Had no time to investigate on this but when using libguestfs with 
nova, the ghost was not always destroyed after file injection. 
Sometimes, you could find an instance spawned with the libguestfs ghost 
still running in the same time. Anyway, I've got no logs to detailthis 
issue, i'll try to get some one day


So summing up this patch was about:

File injection with libguestfs not working in some specific environment 
(dist pkvm 2.1.0 + libguestfs pkvm packaged version + nova havana) and i 
supposed i was not the only one concerned


On our side we had to temporarily disable file injection using libguestfs

Since nova still considers fuse mounts as acceptableit would have been 
practical if, at the time, it had been flexible on the fact to use or 
not libguestfs when this one is installed. (By the way, correct me if 
wrong, but there are no current open issues with fuse mounts, and if 
there are, why is it still proposed in nova ? would even say this is the 
default/most used method because there is no dist considering libguestfs 
is a dependency for the nova-compute package)


Get your reluctance. Giving up with the patch.

regards

raphael


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [nova] The strange case of osapi_compute_unique_server_name_scope

2015-02-23 Thread Matthew Booth
On 20/02/15 20:15, Andrew Bogott wrote:
 On 2/20/15 9:06 AM, Mike Dorman wrote:
 I can report that we do use this option (‘global' setting.)  We have to
 enforce name uniqueness for instances’ integration with some external
 systems (namely AD and Spacewalk) which require unique naming.

 However, we also do some external name validation which I think
 effectively enforces uniqueness as well.  So if this were deprecated, I
 don’t know if it would directly affect us for our specific situation.

 Other operators, anyone else using
 osapi_compute_unique_server_name_scope?
 I use it!  And, in fact, added it in the first place :(
 
 I have no real recall of what we concluded when originally discussing
 the associated race.  The feature is useful to me and I'd love it if it
 could be moved into the db to fix the race, but I concede that it's a
 pretty big can o' worms, so if no one else cries out in pain I can live
 with it being deprecated.

Ok, with 2 identified users I'm going to abandon my patch to deprecate
with no replacement. My current plan is to leave the race there with a
comment in the code and the config documentation. Perhaps we can fix
this properly at some point when we get the online schema changes, but
for the moment it seems like a lot of complication for a relatively
small problem.

Do you use the global or project scope, btw?

Matt
-- 
Matthew Booth
Red Hat Engineering, Virtualisation Team

Phone: +442070094448 (UK)
GPG ID:  D33C3490
GPG FPR: 3733 612D 2D05 5458 8A8A 1600 3441 EA19 D33C 3490

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


[openstack-dev] [Glance] [all] python-glanceclient release 0.16.0

2015-02-23 Thread Nikhil Komawar
The python-glanceclient release management team is pleased to announce:
python-glanceclient version 0.16.0 has been released on Tuesday, Feb 24th 
around 05:50 UTC.

For more information, please find the details at:

https://launchpad.net/python-glanceclient/+milestone/v0.16.0

Please report the issues through launchpad:

https://bugs.launchpad.net/python-glanceclient

Thanks,
-Nikhil
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [infra] [cinder] CI via infra for the DRBD Cinder driver

2015-02-23 Thread Clark Boylan
On Fri, Feb 20, 2015, at 11:24 AM, Philipp Marek wrote:
 Hi all,
 
 this is a reflection of the discussion I just had on #openstack-infra;
 it's 
 about (re-)using the central CI infrastructure for our Open-Source DRBD 
 driver too.
 
 
 The current status is:
  * The DRBD driver is already in Cinder, so DRBD-replicated Cinder
  storage
using iSCSI to the hypervisors does work out-of-the-box.
  * The Nova-parts didn't make it in for Kilo; we'll try to get them into
  L.
  * I've got a lib/backends/drbd for devstack, that together with a
  matching 
local.conf can set up a node - at least for a limited set of 
distributions (as DRBD needs a kernel module, Ubuntu/Debian via DKMS
are 
the easy way).
[Please note that package installation is not yet done in this script 
yet - I'm not sure whether I can/may/should simply add an 
apt-repository.]
 
 
 Now, clarkb told me about two caveats:
 
   «Yup, so the two things I will start with is that multinode testing is 
still really rudimentary, we only just got tempest sort of working
with 
it. So I might suggest running on a single node first to get the
general 
thing working.
You can follow the current state of multinode work at
https://review.openstack.org/#/c/141530/
 
The other thing is that we don't have the zuul code to vote with 
a different account deployed/merged yet. So initially you could run
your 
job but it wouldn't vote against, say, cinder.»
The stack that adds the necessary zuul stuff ends with
https://review.openstack.org/#/c/121528/
 
 
 Cinder has a deadline for CI: March 19th; upon relaying that fact (resp. 
 nearly correct date) clarkb said
 
   «thats about 3 weeks... probably at least for the zuul thing.»
 
 So, actually it's nearly 4 weeks, let's hope that it all works out.
 
 
 Actually, the multi-node testing will only be needed when we get the Nova 
 parts in, because then it would make sense to test (Nova) via both iSCSI 
 and the DRBD transport; for Cinder CI a single-node setup is sufficient.
 
 
 My remaining questions are:
  * Is it possible to have our driver tested via the common
  infrastructure?
I think it should be, initially just reporting against your devstack
plugin while things get sorted out then once zuul has the appropriate
bits reporting like a third party test account but running upstream.
  * Is it okay to setup another apt-repository during the devstack run,
to install the needed packages? I'm not sure whether our servers
would simply be accessible, some firewall or filtering proxy could
break such things easily.
It is not ok in a gating job to do that but that isn't your intention so
should be fine. That said, isn't drbd available in existing apt/yum
repos? What special sauce do you need to pull in?
  * Apart from the cinder-backend script in devstack (which I'll have to 
finish first, see eg. package installation), is any other information 
needed from us?
Other than following this thread and writing that devstack plugin
probably not.
 
 
Hope this helps,
Clark

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [stable][requirements] External dependency caps introduced in 499db6b

2015-02-23 Thread Joe Gordon
On Mon, Feb 23, 2015 at 11:04 AM, Doug Hellmann d...@doughellmann.com
wrote:



 On Mon, Feb 23, 2015, at 12:26 PM, Joe Gordon wrote:
  On Mon, Feb 23, 2015 at 8:49 AM, Ihar Hrachyshka ihrac...@redhat.com
  wrote:
 
   -BEGIN PGP SIGNED MESSAGE-
   Hash: SHA1
  
   On 02/20/2015 07:16 PM, Joshua Harlow wrote:
Sean Dague wrote:
On 02/20/2015 12:26 AM, Adam Gandelman wrote:
Its more than just the naming.  In the original proposal,
requirements.txt is the compiled list of all pinned deps
(direct and transitive), while
requirements.inhttp://requirements.in  reflects what people
will actually use.  Whatever is in requirements.txt affects the
egg's requires.txt. Instead, we can keep requirements.txt
unchanged and have it still be the canonical list of
dependencies, while
reqiurements.out/requirements.gate/requirements.whatever is an
upstream utility we produce and use to keep things sane on our
slaves.
   
Maybe all we need is:
   
* update the existing post-merge job on the requirements repo
to produce a requirements.txt (as it does now) as well the
compiled version.
   
* modify devstack in some way with a toggle to have it process
dependencies from the compiled version when necessary
   
I'm not sure how the second bit jives with the existing
devstack installation code, specifically with the libraries
from git-or-master but we can probably add something to warm
the system with dependencies from the compiled version prior to
calling pip/setup.py/etc http://setup.py/etc
   
It sounds like you are suggesting we take the tool we use to
ensure that all of OpenStack is installable together in a unified
way, and change it's installation so that it doesn't do that any
more.
   
Which I'm fine with.
   
But if we are doing that we should just whole hog give up on the
idea that OpenStack can be run all together in a single
environment, and just double down on the devstack venv work
instead.
   
It'd be interesting to see what a distribution (canonical,
redhat...) would think about this movement. I know yahoo! has been
looking into it for similar reasons (but we are more flexibly then
I think a packager such as canonical/redhat/debian/... would/culd
be). With a move to venv's that seems like it would just offload
the work to find the set of dependencies that work together (in a
single-install) to packagers instead.
   
Is that ok/desired at this point?
   
  
   Honestly, I failed to track all the different proposals. Just saying
   from packager perspective: we absolutely rely on requirements.txt not
   being a list of hardcoded values from pip freeze, but present us a
   reasonable freedom in choosing versions we want to run in packaged
   products.
  
  
  in short the current proposal for stable branches is:
 
  keep requirements.txt as is, except maybe put some upper bounds on the
  requirements.
 
  Add requirements.gate to specify the *exact* versions we are gating
  against
  (this would be a full list including all transitive dependencies).

 The gate syncs requirements into projects before installing them. Would
 we change the sync script for the gate to work from the
 requirements.gate file, or keep it pulling from requirements.txt?


We would only add requirements.gate for stable branches (because we don't
want to cap/pin  things on master). So I think the answer is sync script
should work for both.  I am not sure on the exact mechanics of how this
would work. Whoever ends up driving this bit of work (I think Adam G), will
have to sort out the details.


 Doug

 
 
   That's why I asked before we should have caps and not pins.
  
   /Ihar
   -BEGIN PGP SIGNATURE-
   Version: GnuPG v1
  
   iQEcBAEBAgAGBQJU61oJAAoJEC5aWaUY1u57T7cIALySnlpLV0tjrsTH2gZxskH+
   zY+L6E/DukFNZsWxB2XSaOuVdVaP3Oj4eYCZ2iL8OoxLrBotiOYyRFH29f9vjNSX
   h++dErBr0SwIeUtcnEjbk9re6fNP6y5Hqhk1Ac+NSxwL75KlS3bgKnJAhLA08MVB
   5xkGRR7xl2cuYf9eylPlQaAy9rXPCyyRdxZs6mNjZ2vlY6hZx/w/Y7V28R/V4gO4
   qsvMg6Kv+3urDTRuJdEsV6HbN/cXr2+o543Unzq7gcPpDYXRFTLkpCRV2k8mnmA1
   pO9W10F1FCQZiBnLk0c6OypFz9rQmKxpwlNUN5MTMF15Et6DOxGBxMcfr7TpRaQ=
   =WHOH
   -END PGP SIGNATURE-
  
  
 __
   OpenStack Development Mailing List (not for usage questions)
   Unsubscribe:
 openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
   http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
  
 
 __
  OpenStack Development Mailing List (not for usage questions)
  Unsubscribe:
  openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
  http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

 __
 OpenStack Development Mailing List (not for usage