date:20150710

Robert Collins wrote:
 Good news everybody, mock 1.1.0 is now out. This backports all the
 improvements over the last couple of years, making it fully
 synchronised with cPython master. Yay.
 
 Bad news. Lots of unit tests jobs have suffered falled from this.
 
 But - none of the things I've looked into so far are bugs in mock 1.1.0.
 
 One of the improvements in mock is to fail when a bad method is called.
 
 Consider this: 
 https://review.openstack.org/#/c/200384/1/taskflow/tests/unit/test_engine_helpers.py
 
 Note the old method: mock_import.assert_called_onec_with(name)
 
 That method never existed. onec is a typo :).
 
 mock 1.0.1 silently accepts that - thats part of its job. But, its a
 very fragile API.
 
 1.1.0 makes that an error, for methods with assert prefixes - unless
 unsafe is specifically requested. So a big chunk of the failing tests
 are tests that were not testing anything *at all*.
 One common exampled of that is 'assert_called' - another method that
 never ever existed. All our tests using that were testing nothing at
 all.

It's still non-trivial to fix, especially as fixing it may expose a real
bug in the never-actually-tested thing.

Releasing this on a Friday sounds like bad timing, especially without
advance notice that we'd have to rush to fix the dozens of new issues it
would expose.

Now that we have data[1] showing what fails, it's not completely crazy
to pin it and buy us some time ?

[1]
http://lists.openstack.org/pipermail/openstack-stable-maint/2015-July/thread.html

-- 
Thierry Carrez (ttx)

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all][tests] Fix it friday! [mock failure in CI]

2015-07-10 Thread Kevin Benton

How do we test to see what is failing in each project with the new version?

Also, I'm responsible for the reference to the private mock method in
Neutron. That particular reference is to prevent people from patching the
same target twice because mock.patch.stopall() unwinds patches in a
non-deterministic order in  py34 which leads to leftover monkey patches.
These caused completely unrelated unit tests to randomly and inexplicably
explode later. More details here:
https://github.com/openstack/neutron/commit/1b60df85ba3ad442c2e4e7e52538e1b9a1bf9378

Cheers,
Kevin Benton

On Fri, Jul 10, 2015 at 12:45 AM, Robert Collins robe...@robertcollins.net
wrote:

 Good news everybody, mock 1.1.0 is now out. This backports all the
 improvements over the last couple of years, making it fully
 synchronised with cPython master. Yay.

 Bad news. Lots of unit tests jobs have suffered falled from this.

 But - none of the things I've looked into so far are bugs in mock 1.1.0.

 One of the improvements in mock is to fail when a bad method is called.

 Consider this:
 https://review.openstack.org/#/c/200384/1/taskflow/tests/unit/test_engine_helpers.py

 Note the old method: mock_import.assert_called_onec_with(name)

 That method never existed. onec is a typo :).

 mock 1.0.1 silently accepts that - thats part of its job. But, its a
 very fragile API.

 1.1.0 makes that an error, for methods with assert prefixes - unless
 unsafe is specifically requested. So a big chunk of the failing tests
 are tests that were not testing anything *at all*.
 One common exampled of that is 'assert_called' - another method that
 never ever existed. All our tests using that were testing nothing at
 all.


 Neutron is failing on a bunch of tests that accessed a private
 function inside mock. I'm surprised reviewers didn't spot this, but
 _patch isn't part of the public API, and never was.

 Tempest seems to be failing due to a different object being returned -
 I haven't dug deep enough to describe the cause in more detail.

 We can probably pin mock back to 1.0.1 locally within projects to gain
 breathing room, but given the API improvements in 1.1.0 (see below[1]
 as readthedocs is failing to build it due to having an old pip in
 their virtualenvs :/) - I think we'll be much better off adopting it
 as quickly as we can. NB: with this release we don't need to use
 'unittest.mock' for Python3.4 - we can just standardise on using
 'mock' across the board, which is much simpler and easier to reason
 about (same codebase everywhere[2])

 [2]: Python 2.6 support was dropped in 1.1.0, so we need to use
 markers to select 1.0.1 for the remaining 2.6 gate jobs. (We should
 kill those of asap).
 [1]:
 - Issue #23310: Fix MagicMock's initializer to work with __methods__, just
   like configure_mock().  Patch by Kasia Jachim.

 - Issue #23568: Add rdivmod support to MagicMock() objects.
   Patch by Håkan Lövdahl.

 - Issue #23581: Add matmul support to MagicMock. Patch by Håkan Lövdahl.

 - Issue #23326: Removed __ne__ implementations.  Since fixing default
 __ne__
   implementation in issue #21408 they are redundant. *** NOT BACKPORTED ***

 - Issue #21270: We now override tuple methods in mock.call objects so that
   they can be used as normal call attributes.

 - Issue #21256: Printout of keyword args should be in deterministic order
 in
   a mock function call. This will help to write better doctests.

 - Issue #21262: New method assert_not_called for Mock.
   It raises AssertionError if the mock has been called.

 - Issue #21238: New keyword argument `unsafe` to Mock. It raises
   `AttributeError` incase of an attribute startswith assert or assret.

 - Issue #21239: patch.stopall() didn't work deterministically when the same
   name was patched more than once.

 - Issue #21222: Passing name keyword argument to mock.create_autospec now
   works.

 - Issue #17826: setting an iterable side_effect on a mock function created
 by
   create_autospec now works. Patch by Kushal Das.

 - Issue #17826: setting an iterable side_effect on a mock function created
 by
   create_autospec now works. Patch by Kushal Das.

 - Issue #20968: unittest.mock.MagicMock now supports division.
   Patch by Johannes Baiter.

 - Issue #20189: unittest.mock now no longer assumes that any object for
   which it could get an inspect.Signature is a callable written in Python.
   Fix courtesy of Michael Foord.

 - Issue #17467: add readline and readlines support to mock_open in
   unittest.mock.

 - Issue #17015: When it has a spec, a Mock object now inspects its
 signature
   when matching calls, so that arguments can be matched positionally or
   by name.

 - Issue #15323: improve failure message of Mock.assert_called_once_with

 - Issue #14857: fix regression in references to PEP 3135 implicit __class__
   closure variable (Reopens issue #12370)

 - Issue #14295: Add unittest.mock


 -Rob



 --
 Robert Collins rbtcoll...@hp.com
 Distinguished Technologist
 HP Converged Cloud

Re: [openstack-dev] [all][tests] Fix it friday! [mock failure in CI]

On 10 July 2015 at 20:23, Kevin Benton blak...@gmail.com wrote:
 How do we test to see what is failing in each project with the new version?

Look at any CI failure in the last 5 hours or so.

Or run tox :).

 Also, I'm responsible for the reference to the private mock method in
 Neutron. That particular reference is to prevent people from patching the
 same target twice because mock.patch.stopall() unwinds patches in a
 non-deterministic order in  py34 which leads to leftover monkey patches.
 These caused completely unrelated unit tests to randomly and inexplicably
 explode later. More details here:
 https://github.com/openstack/neutron/commit/1b60df85ba3ad442c2e4e7e52538e1b9a1bf9378

Thats fixed in 1.1.0. So you should be able to unwind that. If you
need a short term workaround, its in mock.mock now.

-Rob

-- 
Robert Collins rbtcoll...@hp.com
Distinguished Technologist
HP Converged Cloud

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all][tests] Fix it friday! [mock failure in CI]

On 10 July 2015 at 20:18, Thierry Carrez thie...@openstack.org wrote:
 Robert Collins wrote:
 Good news everybody, mock 1.1.0 is now out. This backports all the
...
 1.1.0 makes that an error, for methods with assert prefixes - unless
 unsafe is specifically requested. So a big chunk of the failing tests
 are tests that were not testing anything *at all*.
 One common exampled of that is 'assert_called' - another method that
 never ever existed. All our tests using that were testing nothing at
 all.

 It's still non-trivial to fix, especially as fixing it may expose a real
 bug in the never-actually-tested thing.

 Releasing this on a Friday sounds like bad timing, especially without
 advance notice that we'd have to rush to fix the dozens of new issues it
 would expose.

There would never be a good time to release such things. The whole
point of the constraints system we're rolling out is to make us not
sensitive to the release schedule of upstreams: we can't dictate the
release schedule of the world to be convenient to our cycles.

*Further to that* we've already rolled out the constraints system for
devstack, so this is not a gate wedge. Its per-project in unit tests,
and it can be fixed one project at a time. There's no rush that makes
this worse for anyone than if it came out on Monday. We don't need [at
least so far as I can tell - maybe the neutron AAS's will be
interlocked - but again, that would happen any day] infra intervention
to force-push stuff in anywhere.

The one thing that we may need, for py26 projects, is for this review:
https://review.openstack.org/#/c/200344/ - to get in asap, for any
projects with python2.6 jobs, since 1.1.0 isn't compatible with 2.6.
That will let folk with 26 jobs update their requirements locally and
easily, without tox.ini hacks.

 Now that we have data[1] showing what fails, it's not completely crazy
 to pin it and buy us some time ?

Pinning will be per repo, because its unittests not devstack. So its
doable if we have to, but since we have to land a patch in the
repositories anyway, trying for a fix first would be good IMO.

-Rob

-- 
Robert Collins rbtcoll...@hp.com
Distinguished Technologist
HP Converged Cloud

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] Setting epoch in setup.cfg??

Joshua Harlow wrote:
 Hi all,
 
 I was thinking about those who are packaging with venvs (and using the
 version of a project in the venv file name); like what anvil[1] is now
 capable of (and does this in the gate now[2]) or packaging via rpms
 (which anvil also does) and they are going to be affected by the version
 shrinkage that just recently happened this cycle.
 
 Soo that got me around to thinking, why aren't all the projects
 setting an epoch[3] in there setup.cfg (for the ones that had a version
 shrinkage from 201X.Y to something small) to make it less painful for
 all these people (and to make it obvious to those reading setup.cfg and
 looking at the version number that a epoch change just happened)...
 
 So I proposed https://review.openstack.org/#/c/200187/ (to cinder) but
 didn't want to push something similar out everywhere before getting more
 input, is there any reason we should not do that? Is there a better way
 to do that? (Or something else I am missing?)

As Robert posted on the review:

a) pbr doesn't support epochs [yet]
b) the release team specifically decided not to epoch things.

Also you'll find that the various distros use different epoch values for
the same software, because epoch are also used to cover local blunders
in packaging and historical artifacts. That is why epochs should be
local to each packaging system and specifying it upstream is imho
counter-productive.

-- 
Thierry Carrez (ttx)

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [openstack][cinder]A discussion about quota update lower than current usage.

2015-07-10 Thread Duncan Thomas

Ah, I apologise, I missed the but where it defaults to force=true. I
withdraw the objection.

I've no strung feelings about the change either way, in that case.
On 10 Jul 2015 10:58, Gorka Eguileor gegui...@redhat.com wrote:

 On Fri, Jul 10, 2015 at 10:28:06AM +0300, Duncan Thomas wrote:
  That is a semantic change to the api that will break anybody who has
  tooling expecting the current behavior. Since there are perfectly
 sensible
  uses of the current behavior, that is not a good thing.

 Hi Duncan,

 I don't think that will be the case, if it's an optional argument that
 by default preserves current behavior (force = True), then it shouldn't
 break anything for all callers that don't use that new option.

 And for those that want the new behavior, they can always pass force set
 to false.

 Cheers,
 Gorka.

  On 10 Jul 2015 07:33, hao wang sxmatch1...@gmail.com wrote:
 
   Cinder now doesn't check the existing resource when user lower the
 quota.
   It's reasonable for admin can adjust the quota limit to lower level
 than
   current usage.
   But it also bring confusion that I have received to end user, they saw
 the
   current usage
   was more than limit, but they can't create resources any more.
  
   So there have been 'bug' reported[1] and code patch[2] committed, I
 knew
   it may be
   inappropriate as 'bug fix', but just want to optimize this API of
   updating quota.
  
   We are proposing to add an option argument which is named 'force' in
   request body.
   Of course the default value is True that means admin can adjust the
 quota
   lower then
   current usage as same as what we did now. When the force is False, that
   will occur
   a Validation and return 400 Bad Request if the update value is lower
 than
   current usage.
  
   I wonder to know folks' opinions and suggestions about this change to
 see
   if this is value to merge this patch.
  
   [1]https://bugs.launchpad.net/neutron/+bug/1304234
   [2]https://review.openstack.org/#/c/197938/
  
   Thanks~
  
   --
  
   Best Wishes For You!
  
  
  
 __
   OpenStack Development Mailing List (not for usage questions)
   Unsubscribe:
 openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
   http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
  
  

 
 __
  OpenStack Development Mailing List (not for usage questions)
  Unsubscribe:
 openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
  http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


 __
 OpenStack Development Mailing List (not for usage questions)
 Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Cinder] [Taskflow]Review help to cinder bp:Implement function to manage/unmanage snapshots

2015-07-10 Thread Dulko, Michal

Feel free to start the work. Right now I’m working on c-vol’s create_volume 
flow. Review is there: https://review.openstack.org/#/c/193167/

From: hao wang [mailto:sxmatch1...@gmail.com]
Sent: Friday, July 10, 2015 4:47 AM
To: OpenStack Development Mailing List (not for usage questions)
Subject: Re: [openstack-dev] [Cinder] [Taskflow]Review help to cinder 
bp:Implement function to manage/unmanage snapshots

Sorry for the long delay, I'd like to help to review those patches, and I don't 
find the patch about refactoring manage_existing flow in manager yet, so maybe 
I can have a try.

2015-06-02 17:45 GMT+08:00 Dulko, Michal 
michal.du...@intel.commailto:michal.du...@intel.com:
Right now we’re working on refactoring current TaskFlow implementations in 
Cinder to make them more readable and clean. Then we’ll be able to decide if we 
want to get more TaskFlow into Cinder or step back from using it. Deadline for 
refactoring work is around 1 of July.

Here’s related patch for scheduler’s create_volume workflow: 
https://review.openstack.org/#/c/186439/

Currently I’m working on a patch for API’s create_volume and John Griffith 
agreed to work on manager’s one (I don’t know current status). If you want to 
help with these efforts – reviews are always welcomed. You may also take a shot 
at refactoring manage_existing flow in manager. It seem simple enough but maybe 
there are some improvement that we can do to make it more readable.

From: hao wang [mailto:sxmatch1...@gmail.commailto:sxmatch1...@gmail.com]
Sent: Tuesday, June 2, 2015 11:30 AM
To: openstack-dev@lists.openstack.orgmailto:openstack-dev@lists.openstack.org
Subject: [openstack-dev] [Cinder] [Taskflow]Review help to cinder bp:Implement 
function to manage/unmanage snapshots

Hi, folks,

There is a cinder bp:Implement function to manage/unmanage 
snapshots(https://review.openstack.org/#/c/144590/), that we use taskflow to 
implement this feature.

So I need your guys' help(cinder  taskflow) to push this forward.

Thanks.

--

Best Wishes For You!

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: 
openstack-dev-requ...@lists.openstack.org?subject:unsubscribehttp://openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

--

Best Wishes For You!
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all][tests] Fix it friday! [mock failure in CI]

Kevin Benton wrote:
 How do we test to see what is failing in each project with the new version?

Just watch one of the thousands tests currently failing in zuul:
http://status.openstack.org/zuul/

Or see the recent periodic stable maint jobs fail reports at:
http://lists.openstack.org/pipermail/openstack-stable-maint/2015-July/thread.html

(the stable/kilo stuff is relatively recent and likely to apply to cover
95% of the master issues as well)

-- 
Thierry Carrez (ttx)

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [Cinder]Restrict volume creation based on type

2015-07-10 Thread Eduard Matei

Hi,

We've been testing a multi-node setup with our driver and ended up in a
strange situation:
- having a driver configured on multiple cinder nodes (But not all)
- having a volume type (available)
- creating a volume with specified volume type causes the volume to be
created (attempted) on a node where the driver is not configured.

We found something called storage_availability_zone but not quite
understood it, and it looks like an extra choice in the Create volume
wizard (which can be skipped/forgotten).

So, question:
- can we force a volume type to be tied to specific nodes? (something
like availability zones, but determined based on volume type, not a
separate setting)


Thanks,

-- 

*Eduard Biceri Matei, Senior Software Developer*
www.cloudfounders.com
 | eduard.ma...@cloudfounders.com
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all][tests] Fix it friday! [mock failure in CI]

2015-07-10 Thread Kevin Benton

Thanks. I didn't realize it was already breaking everything. I thought it
might have been stuck in requirements bump patch somewhere.

Thats fixed in 1.1.0. So you should be able to unwind that. If you need a
short term workaround, its in mock.mock now.

Yeah, I know it's fixed, I reported the upstream bug quite some time ago.
:) I just felt the need to explain because you implied that the Neutron
reviewers missed something when the patch that introduced that was very
intentional due to the many times that bug has caused non-deterministic
failures.

On Fri, Jul 10, 2015 at 1:28 AM, Robert Collins robe...@robertcollins.net
wrote:

 On 10 July 2015 at 20:23, Kevin Benton blak...@gmail.com wrote:
  How do we test to see what is failing in each project with the new
 version?

 Look at any CI failure in the last 5 hours or so.

 Or run tox :).

  Also, I'm responsible for the reference to the private mock method in
  Neutron. That particular reference is to prevent people from patching the
  same target twice because mock.patch.stopall() unwinds patches in a
  non-deterministic order in  py34 which leads to leftover monkey patches.
  These caused completely unrelated unit tests to randomly and inexplicably
  explode later. More details here:
 
 https://github.com/openstack/neutron/commit/1b60df85ba3ad442c2e4e7e52538e1b9a1bf9378

 Thats fixed in 1.1.0. So you should be able to unwind that. If you
 need a short term workaround, its in mock.mock now.

 -Rob

 --
 Robert Collins rbtcoll...@hp.com
 Distinguished Technologist
 HP Converged Cloud

 __
 OpenStack Development Mailing List (not for usage questions)
 Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev




-- 
Kevin Benton
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [nova][nova-docker]

2015-07-10 Thread Daniel Depaoli

Hi all.
I have a brief question about nova and nova-docker.
In nova-docker I add a piece of code when I raise an exception, in
particular in 'novadocker/virt/docker/driver.py', function
'_start_container':

*
exitcode=self.docker.inspect_container(container_id)['State']['ExitCode']*
*if exitcode  0:*
*raise exception.NoValidHost(reason=Error in docker exit
code)*

In nova cli or horizon I expect to see my error and not the general error
raised after my code.
Can you help me?

Daniel
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all][tests] Fix it friday! [mock failure in CI]

On 10 July 2015 at 20:50, Kevin Benton blak...@gmail.com wrote:
 Thanks. I didn't realize it was already breaking everything. I thought it
 might have been stuck in requirements bump patch somewhere.

Thats fixed in 1.1.0. So you should be able to unwind that. If you need a
 short term workaround, its in mock.mock now.

 Yeah, I know it's fixed, I reported the upstream bug quite some time ago. :)
 I just felt the need to explain because you implied that the Neutron
 reviewers missed something when the patch that introduced that was very
 intentional due to the many times that bug has caused non-deterministic
 failures.

Fair enough; I was feeling a little miffed at breaking so many
projects; sorry :)

-Rob


-- 
Robert Collins rbtcoll...@hp.com
Distinguished Technologist
HP Converged Cloud

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Openstack-operators] [Openstack] Rescinding the M name decision

Adam Lawson wrote:
 The alternative of course is to just number the releases since names
 ultimately don't mean anything but it seems there are problems with that
 level of simplicity. I personally prefer Tristan's suggestion to keep it
 as simple as possible. In a few years we'll run out of letters anyway.

Part of the confusion here is that we are not naming releases. We are
naming release *cycles*. We are giving a name to a period of time,
basically. In that period of time, various version numbers for various
components will be released. Saying Glance 12.0.0 was released in
OpenStack 13 cycle is not really helping.

We won't run out of letters, because the names can cycle back to A
(potentially using a new theme, away from geographic features near
where the corresponding design summit happened).

So while we could technically name a release cycle 14, I feel it's a
bit more difficult to rally around a number than a name. Also, numbers
wouldn't really solve the perceived issues with names: numbers happen to
also be culturally meaningful. You don't have a 13th floor in many US
buildings. In China, building miss the 4th floor instead. 9 is feared in
Japan. And don't talk about 39 to Afghans.

I think growing up is accepting the pain that comes with picking a
good name, rather than sidestepping the issue.

-- 
Thierry Carrez (ttx)

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [qa][tempest] kwargs of service clients for POST/PUT methods

2015-07-10 Thread Ken'ichi Ohmichi

Hi Anne,

2015-07-09 12:22 GMT+09:00 Anne Gentle annegen...@justwriteclick.com:
 On Wed, Jul 8, 2015 at 9:48 PM, GHANSHYAM MANN ghanshyamm...@gmail.com
 wrote:
 On Thu, Jul 9, 2015 at 9:39 AM, Ken'ichi Ohmichi ken1ohmi...@gmail.com
 wrote:
  2015-07-08 16:42 GMT+09:00 Ken'ichi Ohmichi ken1ohmi...@gmail.com:
  2015-07-08 14:07 GMT+09:00 GHANSHYAM MANN ghanshyamm...@gmail.com:
  On Wed, Jul 8, 2015 at 12:27 PM, Ken'ichi Ohmichi
  ken1ohmi...@gmail.com wrote:
 
  By defining all parameters on each method like update_quota_set(), it
  is easy to know what parameters are available from caller/programer
  viewpoint.
 
  I think this can be achieved with former approach also by defining all
  expected param in doc string properly.
 
  You are exactly right.
  But current service clients contain 187 methods *only for Nova* and
  most methods don't contain enough doc string.
  So my previous hope which was implied was we could avoid writing doc
  string with later approach.
 
  I am thinking it is very difficult to maintain doc string of REST APIs
  in tempest-lib because APIs continue changing.
  So instead of doing it, how about putting the link of official API
  document[1] in tempest-lib and concentrating on maintaining official
  API document?
  OpenStack APIs are huge now and It doesn't seem smart to maintain
  these docs at different places.
 

 ++, this will be great. Even API links can be provided in both class
 doc string as well as each method doc string (link to specific API).
 This will improve API ref docs quality and maintainability.


 Agreed, though I also want to point you to this doc specification. We hope
 it will help with the maintenance of the API docs.

 https://review.openstack.org/#/c/177934/

 I also want Tempest maintainers to start thinking about how a diff
 comparison can help with reviews of any changes to the API itself. We have a
 proof of concept and need to do additional work to ensure it works for
 multiple OpenStack APIs.

Thanks for your feedback,
That will be a big step for improving the API docs, I also like to
join for working together.

Thanks
Ken Ohmichi

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [all][tests] Fix it friday! [mock failure in CI]

Good news everybody, mock 1.1.0 is now out. This backports all the
improvements over the last couple of years, making it fully
synchronised with cPython master. Yay.

Bad news. Lots of unit tests jobs have suffered falled from this.

But - none of the things I've looked into so far are bugs in mock 1.1.0.

One of the improvements in mock is to fail when a bad method is called.

Consider this: 
https://review.openstack.org/#/c/200384/1/taskflow/tests/unit/test_engine_helpers.py

Note the old method: mock_import.assert_called_onec_with(name)

That method never existed. onec is a typo :).

mock 1.0.1 silently accepts that - thats part of its job. But, its a
very fragile API.

1.1.0 makes that an error, for methods with assert prefixes - unless
unsafe is specifically requested. So a big chunk of the failing tests
are tests that were not testing anything *at all*.
One common exampled of that is 'assert_called' - another method that
never ever existed. All our tests using that were testing nothing at
all.


Neutron is failing on a bunch of tests that accessed a private
function inside mock. I'm surprised reviewers didn't spot this, but
_patch isn't part of the public API, and never was.

Tempest seems to be failing due to a different object being returned -
I haven't dug deep enough to describe the cause in more detail.

We can probably pin mock back to 1.0.1 locally within projects to gain
breathing room, but given the API improvements in 1.1.0 (see below[1]
as readthedocs is failing to build it due to having an old pip in
their virtualenvs :/) - I think we'll be much better off adopting it
as quickly as we can. NB: with this release we don't need to use
'unittest.mock' for Python3.4 - we can just standardise on using
'mock' across the board, which is much simpler and easier to reason
about (same codebase everywhere[2])

[2]: Python 2.6 support was dropped in 1.1.0, so we need to use
markers to select 1.0.1 for the remaining 2.6 gate jobs. (We should
kill those of asap).
[1]:
- Issue #23310: Fix MagicMock's initializer to work with __methods__, just
  like configure_mock().  Patch by Kasia Jachim.

- Issue #23568: Add rdivmod support to MagicMock() objects.
  Patch by Håkan Lövdahl.

- Issue #23581: Add matmul support to MagicMock. Patch by Håkan Lövdahl.

- Issue #23326: Removed __ne__ implementations.  Since fixing default __ne__
  implementation in issue #21408 they are redundant. *** NOT BACKPORTED ***

- Issue #21270: We now override tuple methods in mock.call objects so that
  they can be used as normal call attributes.

- Issue #21256: Printout of keyword args should be in deterministic order in
  a mock function call. This will help to write better doctests.

- Issue #21262: New method assert_not_called for Mock.
  It raises AssertionError if the mock has been called.

- Issue #21238: New keyword argument `unsafe` to Mock. It raises
  `AttributeError` incase of an attribute startswith assert or assret.

- Issue #21239: patch.stopall() didn't work deterministically when the same
  name was patched more than once.

- Issue #21222: Passing name keyword argument to mock.create_autospec now
  works.

- Issue #17826: setting an iterable side_effect on a mock function created by
  create_autospec now works. Patch by Kushal Das.

- Issue #17826: setting an iterable side_effect on a mock function created by
  create_autospec now works. Patch by Kushal Das.

- Issue #20968: unittest.mock.MagicMock now supports division.
  Patch by Johannes Baiter.

- Issue #20189: unittest.mock now no longer assumes that any object for
  which it could get an inspect.Signature is a callable written in Python.
  Fix courtesy of Michael Foord.

- Issue #17467: add readline and readlines support to mock_open in
  unittest.mock.

- Issue #17015: When it has a spec, a Mock object now inspects its signature
  when matching calls, so that arguments can be matched positionally or
  by name.

- Issue #15323: improve failure message of Mock.assert_called_once_with

- Issue #14857: fix regression in references to PEP 3135 implicit __class__
  closure variable (Reopens issue #12370)

- Issue #14295: Add unittest.mock


-Rob



-- 
Robert Collins rbtcoll...@hp.com
Distinguished Technologist
HP Converged Cloud

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [openstack][cinder]A discussion about quota update lower than current usage.

2015-07-10 Thread Gorka Eguileor

On Fri, Jul 10, 2015 at 10:28:06AM +0300, Duncan Thomas wrote:
 That is a semantic change to the api that will break anybody who has
 tooling expecting the current behavior. Since there are perfectly sensible
 uses of the current behavior, that is not a good thing.

Hi Duncan,

I don't think that will be the case, if it's an optional argument that
by default preserves current behavior (force = True), then it shouldn't
break anything for all callers that don't use that new option.

And for those that want the new behavior, they can always pass force set
to false.

Cheers,
Gorka.

 On 10 Jul 2015 07:33, hao wang sxmatch1...@gmail.com wrote:
 
  Cinder now doesn't check the existing resource when user lower the quota.
  It's reasonable for admin can adjust the quota limit to lower level than
  current usage.
  But it also bring confusion that I have received to end user, they saw the
  current usage
  was more than limit, but they can't create resources any more.
 
  So there have been 'bug' reported[1] and code patch[2] committed, I knew
  it may be
  inappropriate as 'bug fix', but just want to optimize this API of
  updating quota.
 
  We are proposing to add an option argument which is named 'force' in
  request body.
  Of course the default value is True that means admin can adjust the quota
  lower then
  current usage as same as what we did now. When the force is False, that
  will occur
  a Validation and return 400 Bad Request if the update value is lower than
  current usage.
 
  I wonder to know folks' opinions and suggestions about this change to see
  if this is value to merge this patch.
 
  [1]https://bugs.launchpad.net/neutron/+bug/1304234
  [2]https://review.openstack.org/#/c/197938/
 
  Thanks~
 
  --
 
  Best Wishes For You!
 
 
  __
  OpenStack Development Mailing List (not for usage questions)
  Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
  http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
 
 

 __
 OpenStack Development Mailing List (not for usage questions)
 Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [nova] Switching the bug status of ~200 bugs at once; Problem?

2015-07-10 Thread Markus Zoeller

I'd like to switch the status of ~200 bugs at once to adjust the
bookkeeping in LaunchPad. This will concern Confirmed and Triaged
bugs which have an assignee [1]. AFAIK they should be In Progress.

I'm concerned if this would affect some reports I'm not aware of in
a bad way. So, if you think this is a bad thing, please speak up.

FWIW, I'll use an enhanced version of [2] which I will provide in the
next days. A few days later I'll have a version ready which can
update stale In Progress bugs by querying Gerrit.

[1] http://bit.ly/1GbhJWu
[2] https://review.openstack.org/#/c/197060/

Regards,
Markus Zoeller (markus_z)


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Fuel] wrong network for keystone endpoint in 6.1 ?

2015-07-10 Thread Daniel Comnea

I know about the flow but what i'm questioning is:

admin endpoint is mapped to br-mgmt subnet (you do have the HAproxy as
below defined in 6.1. In 6.0 and before you had no HAproxy)

listen keystone-2
  bind 192.168.20.3:35357
  option  httpchk
  option  httplog
  option  httpclose
  server node-17 192.168.20.20:35357   check inter 10s fastinter 2s
downinter 3s rise 3 fall 3
  server node-18 192.168.20.21:35357   check inter 10s fastinter 2s
downinter 3s rise 3 fall 3
  server node-23 192.168.20.26:35357   check inter 10s fastinter 2s
downinter 3s rise 3 fall 3

public endpoint is mapped to br-ex

So with this behavior you are saying the bt-mgmt subnet (which i thought is
only for controller  compute traffic, isolated network) should be
routable in the same way br-ex is?

Dani

On Thu, Jul 9, 2015 at 11:30 PM, Stanislaw Bogatkin sbogat...@mirantis.com
wrote:

 Hi Daniel,

 answer is no - actually there is no strong dependency between public and
 internal/admin endpoints. In your case keystone client ask keystone on
 address 10.52.71.39 (which, I think, was provided by system
 variable OS_AUTH_URL), auth on it and then keystone give endpoints list to
 client. Client selected admin endpoint from this list (192.168.20.3
 address) and tried to get information you asked. It's a normal behavior.

 So, in Fuel by default we have 3 different endpoints for keystone - public
 on public VIP, port 5000; internal on management VIP, port 5000, admin on
 management VIP, port 35357.

 On Thu, Jul 9, 2015 at 4:59 PM, Daniel Comnea comnea.d...@gmail.com
 wrote:

 Hi,

 I'm running Fuel 6.1 and i've seen an interesting behavior which i think
 match bug [1]

 Basically the adminUrl  publicUrl part of keystone endpoint are
 different

 And the result of that is that you can't run keystone cli - i.e
 create/list tenants etc

 keystone --debug tenant-list
 /usr/local/lib/python2.7/site-packages/keystoneclient/shell.py:65:
 DeprecationWarning: The keystone CLI is deprecated in favor of python-
 openstackclient. For a Python library, continue using python-keys
 toneclient.
   'python-keystoneclient.', DeprecationWarning)
 DEBUG:keystoneclient.auth.identity.v2:Making authentication request to
 http://10.20.71.39:5000/v2.0/tokens
 INFO:requests.packages.urllib3.connectionpool:Starting new HTTP
 connection (1): 10.52.71.39
 DEBUG:requests.packages.urllib3.connectionpool:POST /v2.0/tokens
 HTTP/1.1 200 3709
 DEBUG:keystoneclient.session:REQ: curl -g -i -X GET
 http://192.168.20.3:35357/v2.0/tenants -H User-Agent: python-
 keystoneclient -H Accept: application/json -H X-Auth-Token:
 {SHA1}cc918b89c2dca563edda43e01964b1f1979c552b

 shouldn't adminURL = publicURL = br-ex for keystone?


 Dani


 [1] https://bugs.launchpad.net/fuel/+bug/1441855

 __
 OpenStack Development Mailing List (not for usage questions)
 Unsubscribe:
 openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



 __
 OpenStack Development Mailing List (not for usage questions)
 Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Tricircle]Tricircle Weekly Team Meeting 2015.07.08 Roundup

2015-07-10 Thread joehuang

Hello,

The scenario I mentioned in the weekly meeting is described in the following 
google doc:

https://docs.google.com/presentation/d/1s0tRI3GtzSR5Dcl0YGSZ2upEwdqPsqU0d6-rq0CRMQo/edit#slide=id.gb6e146160_0_25

To reduce the complexity, I suggest we focus on the scenario 1 in the current 
phase.

Best Regards
Chaoyi Huang ( Joe Huang )

From: Irena Berezovsky [mailto:irenab@gmail.com]
Sent: Wednesday, July 08, 2015 11:27 PM
To: OpenStack Development Mailing List (not for usage questions); 
zhipengh...@gmail.com; Eran Gampel
Subject: Re: [openstack-dev] [Tricircle]Tricircle Weekly Team Meeting 
2015.07.08 Roundup

Hi,
Thank you very much for posting the log. I had to leave this meeting earlier.
I would love to participate and collaborate on any related item use 
cases/requirements/design.

BR,
Irena

On Wed, Jul 8, 2015 at 5:49 PM, Zhipeng Huang 
zhipengh...@gmail.commailto:zhipengh...@gmail.com wrote:
Hi Team,

Thanks a lot for attending today's meeting. The meeting minutes could be found 
at

http://eavesdrop.openstack.org/meetings/tricircle/2015/tricircle.2015-07-08-13.05.html

And please find in the attachment for chatlog that had not been logged by 
meetbot.

--
Zhipeng (Howard) Huang

Standard Engineer
IT Standard  Patent/IT Prooduct Line
Huawei Technologies Co,. Ltd
Email: huangzhip...@huawei.commailto:huangzhip...@huawei.com
Office: Huawei Industrial Base, Longgang, Shenzhen

(Previous)
Research Assistant
Mobile Ad-Hoc Network Lab, Calit2
University of California, Irvine
Email: zhipe...@uci.edumailto:zhipe...@uci.edu
Office: Calit2 Building Room 2402

OpenStack, OPNFV, OpenDaylight, OpenCompute Aficionado

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: 
openstack-dev-requ...@lists.openstack.org?subject:unsubscribehttp://openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [openstack][cinder]A discussion about quota update lower than current usage.

2015-07-10 Thread Duncan Thomas

That is a semantic change to the api that will break anybody who has
tooling expecting the current behavior. Since there are perfectly sensible
uses of the current behavior, that is not a good thing.
On 10 Jul 2015 07:33, hao wang sxmatch1...@gmail.com wrote:

 Cinder now doesn't check the existing resource when user lower the quota.
 It's reasonable for admin can adjust the quota limit to lower level than
 current usage.
 But it also bring confusion that I have received to end user, they saw the
 current usage
 was more than limit, but they can't create resources any more.

 So there have been 'bug' reported[1] and code patch[2] committed, I knew
 it may be
 inappropriate as 'bug fix', but just want to optimize this API of
 updating quota.

 We are proposing to add an option argument which is named 'force' in
 request body.
 Of course the default value is True that means admin can adjust the quota
 lower then
 current usage as same as what we did now. When the force is False, that
 will occur
 a Validation and return 400 Bad Request if the update value is lower than
 current usage.

 I wonder to know folks' opinions and suggestions about this change to see
 if this is value to merge this patch.

 [1]https://bugs.launchpad.net/neutron/+bug/1304234
 [2]https://review.openstack.org/#/c/197938/

 Thanks~

 --

 Best Wishes For You!


 __
 OpenStack Development Mailing List (not for usage questions)
 Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Openstack-operators] [Openstack] Rescinding the M name decision

2015-07-10 Thread Neil Jerram


On 10/07/15 10:19, Thierry Carrez wrote:


Part of the confusion here is that we are not naming releases. We are
naming release *cycles*. We are giving a name to a period of time,
basically. In that period of time, various version numbers for various
components will be released. Saying Glance 12.0.0 was released in
OpenStack 13 cycle is not really helping.

We won't run out of letters, because the names can cycle back to A
(potentially using a new theme, away from geographic features near
where the corresponding design summit happened).

So while we could technically name a release cycle 14, I feel it's a
bit more difficult to rally around a number than a name. Also, numbers
wouldn't really solve the perceived issues with names: numbers happen to
also be culturally meaningful. You don't have a 13th floor in many US
buildings. In China, building miss the 4th floor instead. 9 is feared in
Japan. And don't talk about 39 to Afghans.

I think growing up is accepting the pain that comes with picking a
good name, rather than sidestepping the issue.


+1 to all that.  Nicely put.

Neil

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [openstack][cinder]A discussion about quota update lower than current usage.

2015-07-10 Thread hao wang

Hi, Duncan

As Gorka said, we are trying not to impact default API behavior, just give
a choice to client that it can restrict cloud admin to update quota lower
than current usage.

2015-07-10 16:47 GMT+08:00 Duncan Thomas duncan.tho...@gmail.com:

 Ah, I apologise, I missed the but where it defaults to force=true. I
 withdraw the objection.

 I've no strung feelings about the change either way, in that case.
 On 10 Jul 2015 10:58, Gorka Eguileor gegui...@redhat.com wrote:

 On Fri, Jul 10, 2015 at 10:28:06AM +0300, Duncan Thomas wrote:
  That is a semantic change to the api that will break anybody who has
  tooling expecting the current behavior. Since there are perfectly
 sensible
  uses of the current behavior, that is not a good thing.

 Hi Duncan,

 I don't think that will be the case, if it's an optional argument that
 by default preserves current behavior (force = True), then it shouldn't
 break anything for all callers that don't use that new option.

 And for those that want the new behavior, they can always pass force set
 to false.

 Cheers,
 Gorka.

  On 10 Jul 2015 07:33, hao wang sxmatch1...@gmail.com wrote:
 
   Cinder now doesn't check the existing resource when user lower the
 quota.
   It's reasonable for admin can adjust the quota limit to lower level
 than
   current usage.
   But it also bring confusion that I have received to end user, they
 saw the
   current usage
   was more than limit, but they can't create resources any more.
  
   So there have been 'bug' reported[1] and code patch[2] committed, I
 knew
   it may be
   inappropriate as 'bug fix', but just want to optimize this API of
   updating quota.
  
   We are proposing to add an option argument which is named 'force' in
   request body.
   Of course the default value is True that means admin can adjust the
 quota
   lower then
   current usage as same as what we did now. When the force is False,
 that
   will occur
   a Validation and return 400 Bad Request if the update value is lower
 than
   current usage.
  
   I wonder to know folks' opinions and suggestions about this change to
 see
   if this is value to merge this patch.
  
   [1]https://bugs.launchpad.net/neutron/+bug/1304234
   [2]https://review.openstack.org/#/c/197938/
  
   Thanks~
  
   --
  
   Best Wishes For You!
  
  
  
 __
   OpenStack Development Mailing List (not for usage questions)
   Unsubscribe:
 openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
   http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
  
  

 
 __
  OpenStack Development Mailing List (not for usage questions)
  Unsubscribe:
 openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
  http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


 __
 OpenStack Development Mailing List (not for usage questions)
 Unsubscribe:
 openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


 __
 OpenStack Development Mailing List (not for usage questions)
 Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev




-- 

Best Wishes For You!
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Cinder] [Taskflow]Review help to cinder bp:Implement function to manage/unmanage snapshots

2015-07-10 Thread hao wang

Sure, I have got this review link. :)

2015-07-10 16:10 GMT+08:00 Dulko, Michal michal.du...@intel.com:

  Feel free to start the work. Right now I’m working on c-vol’s
 create_volume flow. Review is there:
 https://review.openstack.org/#/c/193167/



 *From:* hao wang [mailto:sxmatch1...@gmail.com]
 *Sent:* Friday, July 10, 2015 4:47 AM
 *To:* OpenStack Development Mailing List (not for usage questions)
 *Subject:* Re: [openstack-dev] [Cinder] [Taskflow]Review help to cinder
 bp:Implement function to manage/unmanage snapshots



 Sorry for the long delay, I'd like to help to review those patches, and I
 don't find the patch about refactoring manage_existing flow in manager
 yet, so maybe I can have a try.



 2015-06-02 17:45 GMT+08:00 Dulko, Michal michal.du...@intel.com:

  Right now we’re working on refactoring current TaskFlow implementations
 in Cinder to make them more readable and clean. Then we’ll be able to
 decide if we want to get more TaskFlow into Cinder or step back from using
 it. Deadline for refactoring work is around 1 of July.



 Here’s related patch for scheduler’s create_volume workflow:
 https://review.openstack.org/#/c/186439/



 Currently I’m working on a patch for API’s create_volume and John Griffith
 agreed to work on manager’s one (I don’t know current status). If you want
 to help with these efforts – reviews are always welcomed. You may also take
 a shot at refactoring manage_existing flow in manager. It seem simple
 enough but maybe there are some improvement that we can do to make it more
 readable.



 *From:* hao wang [mailto:sxmatch1...@gmail.com]
 *Sent:* Tuesday, June 2, 2015 11:30 AM
 *To:* openstack-dev@lists.openstack.org
 *Subject:* [openstack-dev] [Cinder] [Taskflow]Review help to cinder
 bp:Implement function to manage/unmanage snapshots



 Hi, folks,



 There is a cinder bp:Implement function to manage/unmanage snapshots(
 https://review.openstack.org/#/c/144590/), that we use taskflow to
 implement this feature.



 So I need your guys' help(cinder  taskflow) to push this forward.



 Thanks.







 --

 Best Wishes For You!


 __
 OpenStack Development Mailing List (not for usage questions)
 Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev





 --

 Best Wishes For You!


 __
 OpenStack Development Mailing List (not for usage questions)
 Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev




-- 

Best Wishes For You!
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all][tests] Fix it friday! [mock failure in CI]

2015-07-10 Thread Kevin Benton

No prob. The fixes for Neutron were relatively trivial.
https://review.openstack.org/#/c/200420/

The only one that was a bit surprising was the failure of autospec in this
file:
https://review.openstack.org/#/c/200420/4/neutron/tests/unit/services/metering/agents/test_metering_agent.py

It was coming back with argument count mismatch failures. It seems like the
log decorator[1] trips up autospec in the new version.

1.
https://github.com/openstack/neutron/blob/master/neutron/services/metering/drivers/noop/noop_driver.py#L22

On Fri, Jul 10, 2015 at 2:16 AM, Robert Collins robe...@robertcollins.net
wrote:

On 10 July 2015 at 20:50, Kevin Benton blak...@gmail.com wrote:
Thanks. I didn't realize it was already breaking everything. I thought it
might have been stuck in requirements bump patch somewhere.

Thats fixed in 1.1.0. So you should be able to unwind that. If you need a
short term workaround, its in mock.mock now.

Yeah, I know it's fixed, I reported the upstream bug quite some time
ago. :)
I just felt the need to explain because you implied that the Neutron
reviewers missed something when the patch that introduced that was very
intentional due to the many times that bug has caused non-deterministic
failures.

Fair enough; I was feeling a little miffed at breaking so many
projects; sorry :)

-Rob

--
Robert Collins rbtcoll...@hp.com
Distinguished Technologist
HP Converged Cloud

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

--
Kevin Benton
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] Suggestion - github - Puppet-openstack repository.

2015-07-10 Thread cool dharma06

Hi all,

i trying to install openstack with puppet. i saw two repositories are
active related to puppet-openstack. i need some advice to go on with which
repository.

1. https://github.com/stackforge/puppet-openstack-cloud

2. https://github.com/puppetlabs/puppetlabs-openstack

i just confused which one to go.?

give some suggestion regarding this. i am newbie for this things.

regards,
cooldharma06
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all][tests] Fix it friday! [mock failure in CI]

I'm looking at Nova unit tests, there are at least 3 issues.

The first one is assert_has_calls *used* to work with a single value

m.assert_has_calls(foo)

The documentation says m.assert_has_calls([foo]) is what you should use,
but the other form used to work. That appears to be tightened up.

It would have been polite to call that out in release notes. Bug filed
here - https://github.com/testing-cabal/mock/issues/263

The second is an issue around the use of autospec (which doesn't do
what's expected any more). Robert figured out a work around -
https://github.com/testing-cabal/mock/issues/264

The last one is still unknown:
https://review.openstack.org/#/c/200500/1/nova/tests/unit/virt/hyperv/test_vhdutils.py,cm

And we're going to just skip that one test until it's resolved. Nova bug
is here - https://bugs.launchpad.net/nova/+bug/1473401 however that
upstream bug should be filed.

Hopefully this set of changes will be helpful with other projects
rushing to try to get tests working again.

-Sean


On 07/10/2015 03:45 AM, Robert Collins wrote:
 Good news everybody, mock 1.1.0 is now out. This backports all the
 improvements over the last couple of years, making it fully
 synchronised with cPython master. Yay.
 
 Bad news. Lots of unit tests jobs have suffered falled from this.
 
 But - none of the things I've looked into so far are bugs in mock 1.1.0.
 
 One of the improvements in mock is to fail when a bad method is called.
 
 Consider this: 
 https://review.openstack.org/#/c/200384/1/taskflow/tests/unit/test_engine_helpers.py
 
 Note the old method: mock_import.assert_called_onec_with(name)
 
 That method never existed. onec is a typo :).
 
 mock 1.0.1 silently accepts that - thats part of its job. But, its a
 very fragile API.
 
 1.1.0 makes that an error, for methods with assert prefixes - unless
 unsafe is specifically requested. So a big chunk of the failing tests
 are tests that were not testing anything *at all*.
 One common exampled of that is 'assert_called' - another method that
 never ever existed. All our tests using that were testing nothing at
 all.
 
 
 Neutron is failing on a bunch of tests that accessed a private
 function inside mock. I'm surprised reviewers didn't spot this, but
 _patch isn't part of the public API, and never was.
 
 Tempest seems to be failing due to a different object being returned -
 I haven't dug deep enough to describe the cause in more detail.
 
 We can probably pin mock back to 1.0.1 locally within projects to gain
 breathing room, but given the API improvements in 1.1.0 (see below[1]
 as readthedocs is failing to build it due to having an old pip in
 their virtualenvs :/) - I think we'll be much better off adopting it
 as quickly as we can. NB: with this release we don't need to use
 'unittest.mock' for Python3.4 - we can just standardise on using
 'mock' across the board, which is much simpler and easier to reason
 about (same codebase everywhere[2])
 
 [2]: Python 2.6 support was dropped in 1.1.0, so we need to use
 markers to select 1.0.1 for the remaining 2.6 gate jobs. (We should
 kill those of asap).
 [1]:
 - Issue #23310: Fix MagicMock's initializer to work with __methods__, just
   like configure_mock().  Patch by Kasia Jachim.
 
 - Issue #23568: Add rdivmod support to MagicMock() objects.
   Patch by Håkan Lövdahl.
 
 - Issue #23581: Add matmul support to MagicMock. Patch by Håkan Lövdahl.
 
 - Issue #23326: Removed __ne__ implementations.  Since fixing default __ne__
   implementation in issue #21408 they are redundant. *** NOT BACKPORTED ***
 
 - Issue #21270: We now override tuple methods in mock.call objects so that
   they can be used as normal call attributes.
 
 - Issue #21256: Printout of keyword args should be in deterministic order in
   a mock function call. This will help to write better doctests.
 
 - Issue #21262: New method assert_not_called for Mock.
   It raises AssertionError if the mock has been called.
 
 - Issue #21238: New keyword argument `unsafe` to Mock. It raises
   `AttributeError` incase of an attribute startswith assert or assret.
 
 - Issue #21239: patch.stopall() didn't work deterministically when the same
   name was patched more than once.
 
 - Issue #21222: Passing name keyword argument to mock.create_autospec now
   works.
 
 - Issue #17826: setting an iterable side_effect on a mock function created by
   create_autospec now works. Patch by Kushal Das.
 
 - Issue #17826: setting an iterable side_effect on a mock function created by
   create_autospec now works. Patch by Kushal Das.
 
 - Issue #20968: unittest.mock.MagicMock now supports division.
   Patch by Johannes Baiter.
 
 - Issue #20189: unittest.mock now no longer assumes that any object for
   which it could get an inspect.Signature is a callable written in Python.
   Fix courtesy of Michael Foord.
 
 - Issue #17467: add readline and readlines support to mock_open in
   unittest.mock.
 
 - Issue #17015: When it has a spec, a Mock object now

Re: [openstack-dev] [neutron] Reading an updated port's fixed IPs in mech driver update_port_postcommit

2015-07-10 Thread Neil Jerram


Sorry to leave this unanswered.

It happens every time (as far as we've tested so far).

A pragmatic fix appears to be to explicitly requery the IPAllocation 
table, as you can see in the two commits here:

https://github.com/Metaswitch/calico/commit/5512ce7dd50db414f161bddcef17b0846a1466ac
https://github.com/Metaswitch/calico/commit/4ecaf3568af52ef9cc29662a3b94672540056f05

But still it seems a shame if this is needed.

Neil


On 07/07/15 22:32, Kevin Benton wrote:

How often does this happen? Is it on every call? If not, is it possible
the forking logic in require_state is messing up the DB handle when it's
invoked?

One way to make sure there aren't open transactions for testing is to
just remove the subtransactions=True statement from update_port in the
ML2 plugin.

On Jul 7, 2015 8:33 AM, Neil Jerram neil.jer...@metaswitch.com
mailto:neil.jer...@metaswitch.com wrote:

Thanks, Kevin, but I believe we're already doing what you advise;
please see

https://github.com/Metaswitch/calico/blob/master/calico/openstack/mech_calico.py#L346-348

Is there a way of checking that there aren't still any open
transactions, when update_port_postcommit is called?

Thanks,
 Neil


On 07/07/15 15:57, Kevin Benton wrote:

It sounds like something is starting a transaction before calling
update_port on the core plugin. This will prevent the
transaction from
being completely closed even though ML2 is in the post_commit phase.

In your db.get_port call, make sure you are using the same DB
session
from the context that was passed into update_port_postcommit and
that
will make sure you always have access to the current state even
if the
transaction isn't closed.

On Tue, Jul 7, 2015 at 5:35 AM, Neil Jerram
neil.jer...@metaswitch.com mailto:neil.jer...@metaswitch.com
mailto:neil.jer...@metaswitch.com
mailto:neil.jer...@metaswitch.com wrote:

 I think there's something I'm not understanding about how
Neutron's
 DB tables are related, and when one can safely read
 believed-to-be-committed information from them...

 My project's mechanism driver is handling a port update in
which the
 fixed IPs are changing; specifically, a second fixed IP has
been
 added to the port.  It does this (for a reason I can explain if
 needed) by calling db.get_port(), in the
update_port_postcommit hook.

 But we observe that the result of db.get_port() does not
include the
 new fixed IP.  Even though we're in the postcommit hook,
and hence
 we assume that all the changes for that port have by now
been committed.

 What are we misunderstanding here?  My guess is that this is
 something to do with 'fixed_ips' not being a column
directly in the
 Ports table, but instead calculated from relationships
between the
 port ID and another (IPAllocation) table.  We've seen a similar
 problem in the past with binding:host_id, for which the same is
 true, i.e. info is in the separate portbindings table.

 Or could it be that the transaction hasn't really been
closed yet,
 when update_port_postcommit hook is called?

 (This is with Icehouse-level code, so it could be something
that's
 been fixed...)

 Many thanks,
  Neil



__
 OpenStack Development Mailing List (not for usage questions)
 Unsubscribe:
openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://openstack-dev-requ...@lists.openstack.org?subject:unsubscribe

http://openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev




--
Kevin Benton



__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe:
openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe:
openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [all] [tests] Considering mock alternatives?

2015-07-10 Thread Dmitry Tantsur


Hi all,

Recent breakage makes me finally raise the question that bothered me for 
some time: are there possible alternatives to mock library we could use?


A couple reasons for that:

1. Devs don't seem to care about semver, backward compatibility and all 
this boring stuff. Releasing a minor version that breaks all or vast 
majority of users is not nice at all.


2. side_effect syntax is no longer sane. Previously it was awesome:

 side_effect = Exception()
 side_effect = [value, Exception()]

etc. Now it's a big typing disaster:

 side_effect = iter([Exception()])

ok, I can live with [], I understand that it may be required for some 
corner cases. I can't understand why mock can't call iter() internally.
And seriously, it's a breaking change, and should have been 
communicated/issued a warning for some time.


If someone has contacts within mock team, are there any chances that 
will provide a convenient alternative to side_effect (though any new 
attribute would be a breaking change for Mock class)?


Any ideas?
Dmitry.

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] [tests] Considering mock alternatives?

On 10 July 2015 at 23:04, Dmitry Tantsur dtant...@redhat.com wrote:
 Hi all,

 Recent breakage makes me finally raise the question that bothered me for
 some time: are there possible alternatives to mock library we could use?

There are. Personally, mock is probably about equal evil-wise to any
mocking library - I find mocking leads to a very heavy dependence on
large functional test suites. But thats a different discussion.

 A couple reasons for that:

 1. Devs don't seem to care about semver, backward compatibility and all this
 boring stuff. Releasing a minor version that breaks all or vast majority of
 users is not nice at all.

Fortunately thats not what happened. Openstack had a lot of tests that
were broken, mock 1.1.0 held up a mirror and let us see that. Its the
same mock thats in the standard library - unittest.mock - backported;
any use of unittest.mock from Python 3.5 would have shown the same
thing. There are no breaks to the actual API as far as I can tell[1].

 2. side_effect syntax is no longer sane. Previously it was awesome:

  side_effect = Exception()
  side_effect = [value, Exception()]

 etc. Now it's a big typing disaster:

  side_effect = iter([Exception()])

 ok, I can live with [], I understand that it may be required for some corner
 cases. I can't understand why mock can't call iter() internally.
 And seriously, it's a breaking change, and should have been
 communicated/issued a warning for some time.

It doesn't show up in the test suite - and I put specific code in
place to deal with the potentially related case where Python 2.7's
next() is different to the 3.x next() [2.6 didn't have next()]. So -
if you've found something that works in unittest.most on Python 3.6
[or 3.5beta3] then its a genuine backport bug and I can fix that up
asap. I haven't seen a bug filed from you about it, so perhaps thats
the first step.

 If someone has contacts within mock team, are there any chances that will
 provide a convenient alternative to side_effect (though any new attribute
 would be a breaking change for Mock class)?

Any new features should be discussed in either the Python bug tracker
(bugs.python.org) or the python-ideas mailing list. But first lets
make sure this isn't just a bug in the backports.

 Any ideas?
 Dmitry.

[1]: There is one under investigation from Neutron, but it certainly
wasn't a known issue, and the fairly extensive test suite doesn't show
it.

-Rob


-- 
Robert Collins rbtcoll...@hp.com
Distinguished Technologist
HP Converged Cloud

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] [tests] Considering mock alternatives?

On 10 July 2015 at 23:34, Dmitry Tantsur dtant...@redhat.com wrote:
 On 07/10/2015 01:13 PM, Robert Collins wrote:

 I see that a lot of code started breaking due to

  side_effect = Exception()

 no longer working. Which was a declared way for some time, at least to my
 best knowledge. And I do realize it's just a back port, but as a user of a
 _library_ I'd like such breakages to be treated with care.

 Maybe it happened and I was just not looking at a proper place.

Well, it works in the test suite, and it works by hand for me. So its
not as simple as 'oh look someone didn't care'.

 from mock import Mock
 m = Mock(side_effect=KeyError('foo'))
 m()
Traceback (most recent call last):
  File stdin, line 1, in module
  File mock/mock.py, line 1055, in __call__
return _mock_self._mock_call(*args, **kwargs)
  File mock/mock.py, line , in _mock_call
raise effect
KeyError: 'foo'

I accept that this is affecting Openstack, it sounds like the same
thing affecting Neutron and Nova, which turned out to be a bug in
autospec: try changing from 'autospec=True' to 'spec=True' [the nova
code didn't use set_spec=True, you may want/need to turn that off as
well]. Please capture any learnings to
https://github.com/testing-cabal/mock/issues/264


 2. side_effect syntax is no longer sane. Previously it was awesome:

   side_effect = Exception()
   side_effect = [value, Exception()]

 etc. Now it's a big typing disaster:

   side_effect = iter([Exception()])

 ok, I can live with [], I understand that it may be required for some
 corner
 cases. I can't understand why mock can't call iter() internally.
 And seriously, it's a breaking change, and should have been
 communicated/issued a warning for some time.


 It doesn't show up in the test suite - and I put specific code in
 place to deal with the potentially related case where Python 2.7's
 next() is different to the 3.x next() [2.6 didn't have next()]. So -
 if you've found something that works in unittest.most on Python 3.6
 [or 3.5beta3] then its a genuine backport bug and I can fix that up
 asap. I haven't seen a bug filed from you about it, so perhaps thats
 the first step.


 People say it's stopped working in Python 3.4.2 and it did work in Python
 3.4.0. I didn't verify this sentence.

So, if its stopped working in cPython, then we're going to want to
follow that (and bugfixes should go straight into cPython too). Once
its in 3.6 we can backport it very rapidly.

-Rob


-- 
Robert Collins rbtcoll...@hp.com
Distinguished Technologist
HP Converged Cloud

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Fuel] wrong network for keystone endpoint in 6.1 ?

2015-07-10 Thread Vladimir Kuklin

Daniel

Yes, if you want to do some administrative stuff you need to have access to
management network to be able to work with internal and admin endpoints.

On Fri, Jul 10, 2015 at 9:58 AM, Daniel Comnea comnea.d...@gmail.com
wrote:

 I know about the flow but what i'm questioning is:

 admin endpoint is mapped to br-mgmt subnet (you do have the HAproxy as
 below defined in 6.1. In 6.0 and before you had no HAproxy)

 listen keystone-2
   bind 192.168.20.3:35357
   option  httpchk
   option  httplog
   option  httpclose
   server node-17 192.168.20.20:35357   check inter 10s fastinter 2s
 downinter 3s rise 3 fall 3
   server node-18 192.168.20.21:35357   check inter 10s fastinter 2s
 downinter 3s rise 3 fall 3
   server node-23 192.168.20.26:35357   check inter 10s fastinter 2s
 downinter 3s rise 3 fall 3

 public endpoint is mapped to br-ex

 So with this behavior you are saying the bt-mgmt subnet (which i thought
 is only for controller  compute traffic, isolated network) should be
 routable in the same way br-ex is?

 Dani


 On Thu, Jul 9, 2015 at 11:30 PM, Stanislaw Bogatkin 
 sbogat...@mirantis.com wrote:

 Hi Daniel,

 answer is no - actually there is no strong dependency between public and
 internal/admin endpoints. In your case keystone client ask keystone on
 address 10.52.71.39 (which, I think, was provided by system
 variable OS_AUTH_URL), auth on it and then keystone give endpoints list to
 client. Client selected admin endpoint from this list (192.168.20.3
 address) and tried to get information you asked. It's a normal behavior.

 So, in Fuel by default we have 3 different endpoints for keystone -
 public on public VIP, port 5000; internal on management VIP, port 5000,
 admin on management VIP, port 35357.

 On Thu, Jul 9, 2015 at 4:59 PM, Daniel Comnea comnea.d...@gmail.com
 wrote:

 Hi,

 I'm running Fuel 6.1 and i've seen an interesting behavior which i
 think match bug [1]

 Basically the adminUrl  publicUrl part of keystone endpoint are
 different

 And the result of that is that you can't run keystone cli - i.e
 create/list tenants etc

 keystone --debug tenant-list
 /usr/local/lib/python2.7/site-packages/keystoneclient/shell.py:65:
 DeprecationWarning: The keystone CLI is deprecated in favor of python-
 openstackclient. For a Python library, continue using python-keys
 toneclient.
   'python-keystoneclient.', DeprecationWarning)
 DEBUG:keystoneclient.auth.identity.v2:Making authentication request to
 http://10.20.71.39:5000/v2.0/tokens
 INFO:requests.packages.urllib3.connectionpool:Starting new HTTP
 connection (1): 10.52.71.39
 DEBUG:requests.packages.urllib3.connectionpool:POST /v2.0/tokens
 HTTP/1.1 200 3709
 DEBUG:keystoneclient.session:REQ: curl -g -i -X GET
 http://192.168.20.3:35357/v2.0/tenants -H User-Agent: python-
 keystoneclient -H Accept: application/json -H X-Auth-Token:
 {SHA1}cc918b89c2dca563edda43e01964b1f1979c552b

 shouldn't adminURL = publicURL = br-ex for keystone?


 Dani


 [1] https://bugs.launchpad.net/fuel/+bug/1441855


 __
 OpenStack Development Mailing List (not for usage questions)
 Unsubscribe:
 openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



 __
 OpenStack Development Mailing List (not for usage questions)
 Unsubscribe:
 openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



 __
 OpenStack Development Mailing List (not for usage questions)
 Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev




-- 
Yours Faithfully,
Vladimir Kuklin,
Fuel Library Tech Lead,
Mirantis, Inc.
+7 (495) 640-49-04
+7 (926) 702-39-68
Skype kuklinvv
35bk3, Vorontsovskaya Str.
Moscow, Russia,
www.mirantis.com http://www.mirantis.ru/
www.mirantis.ru
vkuk...@mirantis.com
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [nova] Switching the bug status of ~200 bugs at once; Problem?

On 07/10/2015 04:09 AM, Markus Zoeller wrote:
 I'd like to switch the status of ~200 bugs at once to adjust the
 bookkeeping in LaunchPad. This will concern Confirmed and Triaged
 bugs which have an assignee [1]. AFAIK they should be In Progress.
 
 I'm concerned if this would affect some reports I'm not aware of in
 a bad way. So, if you think this is a bad thing, please speak up.
 
 FWIW, I'll use an enhanced version of [2] which I will provide in the
 next days. A few days later I'll have a version ready which can
 update stale In Progress bugs by querying Gerrit.
 
 [1] http://bit.ly/1GbhJWu
 [2] https://review.openstack.org/#/c/197060/

I would not do that. Bugs with assignee tend to get ignored by other
people looking at the problem, and in *most* cases the assignee is not
actually working on the bug. I would instead go the other direction and
pop any In Progress bug that has seen no activity in the last 60 days
back to Confirmed.

-Sean

-- 
Sean Dague
http://dague.net

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [neutron][qos] how we get back into master

2015-07-10 Thread Ihar Hrachyshka

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA256

Hi all,

as you probably know, QoS feature is on horizon for Liberty and uses a
separate feature/qos branch. It's expected that after it's complete,
the branch is merged back into master.

Note that it will probably be the first (or the second, if
feature/pecan goes in before feature/qos) time when a feature branch
is actually merged into master, so we're in an unknown land here, and
I hope everyone involved understands we want to set a good example in
terms of the process. If we fail on that road, it may wrongfully
suggest that feature branches do not work.

There are two things that QoS subteam and broader Neutron community
should solve to make the merge happen.

1. for QoS subteam, complete the feature, get the code into good shape
that does not lower the quality bar set for master (meaning, it's
clean, it's idiomatic, it's covered by misc types of tests, it's
properly documented, etc.)

2. for QoS subteam, arrange everything needed to make review process
for merge-back *meaningful*.

3. for broad community, provide meaningful feedback for the feature
and accommodate merge-back if the feature branch is considered ready
for that.

QoS subteam believes that review process for merge-back should not be
shallow rubber stamping, so active participation from core reviewers
external to the subteam is highly desired. We appreciate any trust
someone may put into the subteam :), but we would still like others to
keep us honest. In the end, we are as interested in the overall
Neutron quality as other contributors. Of course, the feature will be
considered experimental for Liberty, but still we don't want to merge
anything completely wrong or something that would break others. :)
Here I will note that the subteam paid significant attention to
decouple the feature from other parts of Neutron, so there should be
few places where its integration in master could break other stuff.
That said... we'll see. ;)

To handle 1., I suggest QoS subteam follows the priority list:
- - complete PoC implementation (some agent side pieces are still not in
the tree);
- - cover the existing code with meaningful tests, if we see no or
limited coverage for any new code introduced;
- - close gaps that are currently marked as TODO(QoS) in the branch
(there are a lot of them);
- - after we consider it's done, do another walk thru each new piece of
code to see whether there is something we may clean up (typos,
comments, wording) to avoid (some of) nitpicking when we get to the
merge-back review process.

It's hard to expect that review for a thousands lines of code
merge-back patch can be meaningful and not loose reviewer attention.
So I suggest we take specific steps to avoid such outcome.

To accommodate 2. and 3., the following is proposed:
2.1 the feature is described in high to mid level details in devref,
with every separate piece of code being described: what it's for, how
it sticks into the whole picture, where the code is found, how it's
tested, etc.
2.2 the document is provided to broad community as part of the
merge-back review process. Reviewers are (self-)assigned to specific
pieces of the whole feature. The size of those pieces should be
consumable, small enough not to take risk of loosing too much review
attention while we go thru them. From QoS subteam side, people are
also assigned to those pieces to accommodate specific issues raised
during review in timely manner.
2.3 Once we get all the pieces ready for review (feature branch is
considered ready by the subteam; feature is described in devref and
split into logical separately reviewable parts; people from external
core reviewer team are on board with getting pieces to review; people
from QoS subteam are available for responses to feedback; ...), we may
want to dedicate a single day just to review the proposed feature. It
may be something like a virtual review sprint.

We may need another iteration of the latest entry point if feedback
requires significant time to handle. After one or more iterations of
feedback, assuming the branch gets to the point where everyone
involved in review process agrees that it's indeed ready for
merge-back, we finally accept it into master.

I think Miguel already signed up to provide some form of devref
description of the system till Wed. It will need attention from all
QoS subteam members to make sure it correctly reflects the desired
state of the tree, so keep an eye on his devref proposal.

Once in master, QoS subteam will polish the feature based on feedback
collected during review process and any bugs that are revealed after
the merge. For that, we should make sure that we have time in the
cycle to apply those late changes. It naturally goes into expected
schedule for the process.

It is desired that we complete the process till L-2 or the start of
L-3. Internally for QoS subteam, let's set end of L-2 as a goal to get
the branch ready for merge-back, and set start of L-3 as the expected

Re: [openstack-dev] [tox/pep8] different result between running tox and pep8

On 2015-07-10 03:15:08 + (+), Gareth wrote:
 My problem looks simple:
 
 Running tox -e pep8, the result says ok, not pep8 mistakes.
 Running pep8 ., a lot of pep8 mistkes come out.
 
 But in your project's tox.ini, there isn't such a long ignore list. So what
 makes this difference happen?

It depends a lot on what repository you're running pep8 against, but
there are a couple of fundamental differences between running pep8
directly and invoking a pep8 environment through tox:

1. In most cases tox is not set to use system site packages, and so
will potentially install a different version of pep8 into the
virtualenv it creates than you have installed system-wide.

2. Most of our repos have the pep8 environment in their tox.ini
configured to invoke flake8 instead, which also runs the pyflakes
checker and possibly other plugins like the hacking checker.
Further, flake8 reads an exclusion list from tox.ini and so may not
report some PEP-8 compliance violations that you'll see when running
the pep8 tool directly.

So, I hope that explains the difference. When you run `tox -e pep8`
you're not asking tox to invoke the pep8 tool; rather you're asking
tox to invoke a set of commands associated with an environment
definition named pep8 in its tox.ini file.
-- 
Jeremy Stanley

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] [tests] Considering mock alternatives?

2015-07-10 Thread Dmitry Tantsur


On 07/10/2015 02:00 PM, Robert Collins wrote:

On 10 July 2015 at 23:34, Dmitry Tantsur dtant...@redhat.com wrote:

On 07/10/2015 01:13 PM, Robert Collins wrote:



I see that a lot of code started breaking due to

  side_effect = Exception()

no longer working. Which was a declared way for some time, at least to my
best knowledge. And I do realize it's just a back port, but as a user of a
_library_ I'd like such breakages to be treated with care.

Maybe it happened and I was just not looking at a proper place.


Well, it works in the test suite, and it works by hand for me. So its
not as simple as 'oh look someone didn't care'.


Yeah, sorry. I'm happy to hear that it was not intentional.




from mock import Mock
m = Mock(side_effect=KeyError('foo'))
m()

Traceback (most recent call last):
   File stdin, line 1, in module
   File mock/mock.py, line 1055, in __call__
 return _mock_self._mock_call(*args, **kwargs)
   File mock/mock.py, line , in _mock_call
 raise effect
KeyError: 'foo'

I accept that this is affecting Openstack, it sounds like the same
thing affecting Neutron and Nova, which turned out to be a bug in
autospec: try changing from 'autospec=True' to 'spec=True' [the nova
code didn't use set_spec=True, you may want/need to turn that off as
well]. Please capture any learnings to
https://github.com/testing-cabal/mock/issues/264


spec=True is not a complete replacement (does not seem to check method 
signature) and is not documented to be used this way (well, at least I 
didn't find it). But yeah, it kind of works.







2. side_effect syntax is no longer sane. Previously it was awesome:

   side_effect = Exception()
   side_effect = [value, Exception()]

etc. Now it's a big typing disaster:

   side_effect = iter([Exception()])

ok, I can live with [], I understand that it may be required for some
corner
cases. I can't understand why mock can't call iter() internally.
And seriously, it's a breaking change, and should have been
communicated/issued a warning for some time.



It doesn't show up in the test suite - and I put specific code in
place to deal with the potentially related case where Python 2.7's
next() is different to the 3.x next() [2.6 didn't have next()]. So -
if you've found something that works in unittest.most on Python 3.6
[or 3.5beta3] then its a genuine backport bug and I can fix that up
asap. I haven't seen a bug filed from you about it, so perhaps thats
the first step.



People say it's stopped working in Python 3.4.2 and it did work in Python
3.4.0. I didn't verify this sentence.


So, if its stopped working in cPython, then we're going to want to
follow that (and bugfixes should go straight into cPython too). Once
its in 3.6 we can backport it very rapidly.

-Rob





__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Cinder]Restrict volume creation based on type

2015-07-10 Thread Erlon Cruz

Hi Eduard,

What extra_specs do you configured in this volume_type?
Yes you can force a volume to an specific node, or backend. You must
configure the volume_backend_name on cinder.conf, and add an extra spec
volume_backend_name='same_as_in_cinder_conf' on the volume type  you
created.

For example:
[lvm1]
volume_driver = cinder.volume.drivers.lvm.LVMVolumeDriver
volume_backend_name = lvm

[lvm2]
volume_driver = cinder.volume.drivers.lvm.LVMVolumeDriver
volume_backend_name = lvm

Then:
cinder type-create lvm
$ cinder type-key lvm_node set volume_backend_name=lvm


On Fri, Jul 10, 2015 at 5:36 AM, Eduard Matei 
eduard.ma...@cloudfounders.com wrote:

 Hi,

 We've been testing a multi-node setup with our driver and ended up in a
 strange situation:
 - having a driver configured on multiple cinder nodes (But not all)
 - having a volume type (available)
 - creating a volume with specified volume type causes the volume to be
 created (attempted) on a node where the driver is not configured.

 We found something called storage_availability_zone but not quite
 understood it, and it looks like an extra choice in the Create volume
 wizard (which can be skipped/forgotten).

 So, question:
 - can we force a volume type to be tied to specific nodes? (something
 like availability zones, but determined based on volume type, not a
 separate setting)


 Thanks,

 --

 *Eduard Biceri Matei, Senior Software Developer*
 www.cloudfounders.com
  | eduard.ma...@cloudfounders.com



 __
 OpenStack Development Mailing List (not for usage questions)
 Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all][tests] Fix it friday! [mock failure in CI]

On 10 July 2015 at 22:07, Kevin Benton blak...@gmail.com wrote:
 No prob. The fixes for Neutron were relatively trivial.
 https://review.openstack.org/#/c/200420/

 The only one that was a bit surprising was the failure of autospec in this
 file:
 https://review.openstack.org/#/c/200420/4/neutron/tests/unit/services/metering/agents/test_metering_agent.py

 It was coming back with argument count mismatch failures. It seems like the
 log decorator[1] trips up autospec in the new version.

 1.
 https://github.com/openstack/neutron/blob/master/neutron/services/metering/drivers/noop/noop_driver.py#L22

Hmm, that might be a flaw in funcsigs, the inspect.signature backport.

We should figure it out - does it get tripped up in Python 3.5 with
the unittest.mock variant? If so, bugs.python.org for the bug. If not,
then please file the bug (and how you tested - the tighter the
reproduction the better) at
https://github.com/testing-cabal/mock/issues.

Thanks!
-Rob

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Fuel] New Criteria for UX bugs

2015-07-10 Thread Jay Pipes


On 07/09/2015 06:14 PM, Andrew Woodward wrote:

We often have bugs which create really poor User eXperience (UX) but our
current bug priority criteria prevent nearly all of them from being
higher than medium (as they nearly always have workarounds). We need to
identify what should qualify as a critical, or high UX defect so that
they can receive appropriate attention.

We discussed what this may look like on the IRC meeting, the general
idea here is that the complexity of effort to work around the UX issue
should be related to the priority.

Critical: requires massive effort to work around, including [un|under]
documented commands and edits to config files

High: requires modification of config files, interfaces that users
aren't expected to use (ie the API when it's _intended_ to work in the
CLI / UI (exclusive of interfaces that are intended to only be available
via API) or requires custom node yaml (again except when it should
exclusively be available)

Medium: Straight forward commands in the CLI


Above criteria look excellent to me, thanks Andrew!
-jay


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all][tests] Fix it friday! [mock failure in CI]

On 07/10/2015 03:45 AM, Robert Collins wrote:
 Good news everybody, mock 1.1.0 is now out. This backports all the
 improvements over the last couple of years, making it fully
 synchronised with cPython master. Yay.
 
 Bad news. Lots of unit tests jobs have suffered falled from this.
 
 But - none of the things I've looked into so far are bugs in mock 1.1.0.
 
 One of the improvements in mock is to fail when a bad method is called.
 
 Consider this: 
 https://review.openstack.org/#/c/200384/1/taskflow/tests/unit/test_engine_helpers.py
 
 Note the old method: mock_import.assert_called_onec_with(name)
 
 That method never existed. onec is a typo :).
 
 mock 1.0.1 silently accepts that - thats part of its job. But, its a
 very fragile API.
 
 1.1.0 makes that an error, for methods with assert prefixes - unless
 unsafe is specifically requested. So a big chunk of the failing tests
 are tests that were not testing anything *at all*.
 One common exampled of that is 'assert_called' - another method that
 never ever existed. All our tests using that were testing nothing at
 all.
 
 
 Neutron is failing on a bunch of tests that accessed a private
 function inside mock. I'm surprised reviewers didn't spot this, but
 _patch isn't part of the public API, and never was.
 
 Tempest seems to be failing due to a different object being returned -
 I haven't dug deep enough to describe the cause in more detail.

The Tempest fix is here - https://review.openstack.org/#/c/200449/ and
was pretty straight forward. It was another one of those assert_*
doesn't exist issues. I just approved it though.

-Sean

-- 
Sean Dague
http://dague.net

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [nova][cinder][qa] encrypted volumes tests don't actually test encrypted volumes for most backends

2015-07-10 Thread Deepak Shetty

Thanks Mike for the heads up

I fixed it for GlusterFS CI [1]
Post the fix, glusterfs CI jobs are running fine [2] See Jul 10, 10:43 AM
onwards

thanx,
deepak

[1]: https://review.openstack.org/#/c/200399/2
[2]:
https://jenkins07.openstack.org/job/check-tempest-dsvm-full-glusterfs-nv/

On Fri, Jul 10, 2015 at 12:52 AM, Mike Perez thin...@gmail.com wrote:

 On 16:47 Jun 30, Mike Perez wrote:
  On 12:24 Jun 26, Matt Riedemann wrote:
  snip
   So the question is, is everyone OK with this and ready to make that
 change?
 
  Thanks for all your work on this Matt.
 
  I'm fine with this. I say bite the bullet and we'll see the CI's surface
 that
  aren't skipping or failing this test.
 
  I will communicate with CI maintainers on the CI list about failures as
 I've
  been doing, and reference this thread and the meeting discussion.

 This landed.

 If your Cinder CI is now failing, set
 ATTACH_ENCRYPTED_VOLUME_AVAILABLE=False
 [1] as explained earlier in this thread.

 [1] - https://review.openstack.org/#/c/199709/

 --
 Mike Perez

 __
 OpenStack Development Mailing List (not for usage questions)
 Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [tox/pep8] different result between running tox and pep8

2015-07-10 Thread Ihar Hrachyshka

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA256

On 07/10/2015 05:15 AM, Gareth wrote:
 Hi guys
 
 My problem looks simple:
 
 Running tox -e pep8, the result says ok, not pep8 mistakes. 
 Running pep8 ., a lot of pep8 mistkes come out.
 
 But in your project's tox.ini, there isn't such a long ignore list.
 So what makes this difference happen?
 

Maybe your pep8/hacking/flake8 versions are different inside pep8 tox
environment and in your root environment where you execute pep8
directly from.

Ihar
-BEGIN PGP SIGNATURE-
Version: GnuPG v2

iQEcBAEBCAAGBQJVn6C3AAoJEC5aWaUY1u57+NwIAIy31gTO6tFKhiERACoPnvj7
I5WvlV1lI2yv6/OlKuqHaQR6eFcehIe6JoT1H+qQpDNkVm/a+zX0CIEhXUJkjH26
z3itWkZJQDgFAgR1QoBesZfiZ1BJDP5b3rFomJjfSr/P6TzeeLmtQHaX54Eb8A/X
F86DetN4nA4Gwwaj7Tzsu30ZOVup3harWGxlzQcl8QLrL6sz/2Ti5+ZhYCg50S1j
CZ6w7TbigCdwq8FcAD4A7dttUbjPilZISYh4lNjcVn89ENoPz2NYMNY007UjjLMw
TaJB3tx0kGA/ztpWwgZnDRsnlL73TYraXPZKAY7VsUxwgY2lnkiC7RiVxMgxLlI=
=75km
-END PGP SIGNATURE-

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [Cinder] Quobyte CI Reporting, procedure for driver readdition

2015-07-10 Thread Silvan Kaiser


Hello Cinder Community!
The Quobyte Cinder CI [1] is back reporting since tuesday late afternoon UTC.
It is working stably with 8 test failing because of a Nova [2] and a 
Driver/Cinder bug [3].
Result logs can be reviewed at [4] (FYI: these also contain sandbox test logs).

Over the last weekend our team was able to find a solution to the networking 
issue that previously gave the CI failures in scenario tests. We implemented 
the CI changes and tested that on monday and tuesday, subsequently reactivating 
the CI. Today i deactivated encrypted volume testing as proposed on the mailing 
list [6].

With the CI working for the better part of this week now I'd like to have 
feedback on how to prepare the revert of the Quobyte Cinder driver removal [6]. 
My guess is a FFE type request to this list and an according gerrit change, any 
help or advice welcome.

Best regards
Silvan Kaiser
(kaisers or casusbelli on IRC)


[1] https://wiki.openstack.org/wiki/ThirdPartySystems/Quobyte_CI
[2] https://bugs.launchpad.net/nova/+bug/1465416
[3] https://bugs.launchpad.net/cinder/+bug/1473116
[4] http://176.9.127.22:8081/?C=M;O=D
[5] http://lists.openstack.org/pipermail/openstack-dev/2015-July/069099.html
[6] http://lists.openstack.org/pipermail/openstack-dev/2015-July/068609.html


-- 

--
*Quobyte* GmbH
Hardenbergplatz 2 - 10623 Berlin - Germany
+49-30-814 591 800 - www.quobyte.com
Amtsgericht Berlin-Charlottenburg, HRB 149012B
management board: Dr. Felix Hupfeld, Dr. Björn Kolbeck, Dr. Jan Stender

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all][tests][swift] Fix it friday! [mock failure in CI] - fix for Swift

2015-07-10 Thread Victor Stinner


Here is a fix for Swift:
https://review.openstack.org/200474

Victor

On 10/07/2015 09:45, Robert Collins wrote:

Good news everybody, mock 1.1.0 is now out. This backports all the
improvements over the last couple of years, making it fully
synchronised with cPython master. Yay.

Bad news. Lots of unit tests jobs have suffered falled from this.

But - none of the things I've looked into so far are bugs in mock 1.1.0.

One of the improvements in mock is to fail when a bad method is called.

Consider this: 
https://review.openstack.org/#/c/200384/1/taskflow/tests/unit/test_engine_helpers.py

Note the old method: mock_import.assert_called_onec_with(name)

That method never existed. onec is a typo :).

mock 1.0.1 silently accepts that - thats part of its job. But, its a
very fragile API.

1.1.0 makes that an error, for methods with assert prefixes - unless
unsafe is specifically requested. So a big chunk of the failing tests
are tests that were not testing anything *at all*.
One common exampled of that is 'assert_called' - another method that
never ever existed. All our tests using that were testing nothing at
all.


Neutron is failing on a bunch of tests that accessed a private
function inside mock. I'm surprised reviewers didn't spot this, but
_patch isn't part of the public API, and never was.

Tempest seems to be failing due to a different object being returned -
I haven't dug deep enough to describe the cause in more detail.

We can probably pin mock back to 1.0.1 locally within projects to gain
breathing room, but given the API improvements in 1.1.0 (see below[1]
as readthedocs is failing to build it due to having an old pip in
their virtualenvs :/) - I think we'll be much better off adopting it
as quickly as we can. NB: with this release we don't need to use
'unittest.mock' for Python3.4 - we can just standardise on using
'mock' across the board, which is much simpler and easier to reason
about (same codebase everywhere[2])

[2]: Python 2.6 support was dropped in 1.1.0, so we need to use
markers to select 1.0.1 for the remaining 2.6 gate jobs. (We should
kill those of asap).
[1]:
- Issue #23310: Fix MagicMock's initializer to work with __methods__, just
   like configure_mock().  Patch by Kasia Jachim.

- Issue #23568: Add rdivmod support to MagicMock() objects.
   Patch by Håkan Lövdahl.

- Issue #23581: Add matmul support to MagicMock. Patch by Håkan Lövdahl.

- Issue #23326: Removed __ne__ implementations.  Since fixing default __ne__
   implementation in issue #21408 they are redundant. *** NOT BACKPORTED ***

- Issue #21270: We now override tuple methods in mock.call objects so that
   they can be used as normal call attributes.

- Issue #21256: Printout of keyword args should be in deterministic order in
   a mock function call. This will help to write better doctests.

- Issue #21262: New method assert_not_called for Mock.
   It raises AssertionError if the mock has been called.

- Issue #21238: New keyword argument `unsafe` to Mock. It raises
   `AttributeError` incase of an attribute startswith assert or assret.

- Issue #21239: patch.stopall() didn't work deterministically when the same
   name was patched more than once.

- Issue #21222: Passing name keyword argument to mock.create_autospec now
   works.

- Issue #17826: setting an iterable side_effect on a mock function created by
   create_autospec now works. Patch by Kushal Das.

- Issue #17826: setting an iterable side_effect on a mock function created by
   create_autospec now works. Patch by Kushal Das.

- Issue #20968: unittest.mock.MagicMock now supports division.
   Patch by Johannes Baiter.

- Issue #20189: unittest.mock now no longer assumes that any object for
   which it could get an inspect.Signature is a callable written in Python.
   Fix courtesy of Michael Foord.

- Issue #17467: add readline and readlines support to mock_open in
   unittest.mock.

- Issue #17015: When it has a spec, a Mock object now inspects its signature
   when matching calls, so that arguments can be matched positionally or
   by name.

- Issue #15323: improve failure message of Mock.assert_called_once_with

- Issue #14857: fix regression in references to PEP 3135 implicit __class__
   closure variable (Reopens issue #12370)

- Issue #14295: Add unittest.mock


-Rob





__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [tox/pep8] different result between running tox and pep8

2015-07-10 Thread Andrey Kurilin

Running pep8 ., a lot of pep8 mistkes come out.

1) `pep8 .` checks all files in directory(including build files, envs and
etc).
2) `pep8 .` doesn't use tox.ini file, which can contain list of ignored
rules. For example, it can looks like:
[flake8]
ignore = H703

3) Most of OpenStack projects use flake8 instead of pep8




On Fri, Jul 10, 2015 at 6:15 AM, Gareth academicgar...@gmail.com wrote:

 Hi guys

 My problem looks simple:

 Running tox -e pep8, the result says ok, not pep8 mistakes.
 Running pep8 ., a lot of pep8 mistkes come out.

 But in your project's tox.ini, there isn't such a long ignore list. So
 what makes this difference happen?

 Kun

 __
 OpenStack Development Mailing List (not for usage questions)
 Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev




-- 
Best regards,
Andrey Kurilin.
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all][tests] Fix it friday! [mock failure in CI]

On 2015-07-10 13:21:56 +0200 (+0200), Victor Stinner wrote:
 I see mock===1.0.1 in upper-constraints.txt, but the python27
 check job of Swift was broken by the release of mock 1.1.
 
 Can someone please explain me why Swift check job failed?

As far as I'm aware, only DevStack is making use of the constraints
file currently. Unit tests are still relying on pip to install
working versions of packages from the requirements.txt and
test-requirements.txt within each repo.
-- 
Jeremy Stanley

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [Murano] Versioning of Murano packages and MuranoPL classes

2015-07-10 Thread Alexander Tivelkov

Hi folks,

Ability to manage multiple versions of application packages and their
dependencies was always an important item in Murano roadmap, however we
still don't have a clear spec for this feature.
Yesterday we hosted a small design session to come up with a plan on what
can be done in Liberty timeframe to have proper versioning for MuranoPL
classes and packages. Stan Lagun, Kirill Zaitsev and myself participated
offline, some other muranoers joined remotely. Thanks to everybody who
joined us.

TL;DR: it turns out that now we have a clear plan which will allow us to
achieve proper versioning of the packages and classes, and we'll try to
implement the most important parts of it in Liberty.

Here is the detailed outcome of the session:


   1. We'll use the standard Semantic Versioning format
   ('Major.Minor.Patch[-dev-build.label[+metadata.label]]') to version our
   packages: changes which break backwards-compatibility should increment the
   major segment, non-breaking new features increment the minor segment and
   all non-breaking bugfixes increment the patch segment. The developers
   should be carefull with the new features part: if you add a new method to
   a class, it may be considered a breaking change if the existing subclasses
   of this class have the same method already declared. We still assume that
   such changes should lead to increment of 'minor' segment, however it is up
   to best judgement of developers in particular case: if the risk of such
   method override is really high it may worth to increment the 'major'
   segment. Proper guideline on the versioning rules will be published closer
   to L release.

   2. A new field 'Version' is introduced into package manifest file which
   should define package version in complete semver format. The field itself
   is optional (so existing apps are not broken), if it is not specified the
   package is assumed to have version '0.0.0'

   3. The existing 'Require' block of Application manifest will be used to
   specify the package dependencies. Currently it is a yaml-based dictionary,
   with the keys set to fully-qualified names of the dependency packages and
   the values set to the version of those dependencies. Currently this block
   is used only for integration with apps stored at apps.openstack.org. It
   is suggested to use this block in the deployment process as well, and
   extend its semantics.
   The version of the dependency specified there should also follow the
   semver notation, however it may be specified in the shortened format, i.e.
   without specifying the 'patch' or 'patch' and 'minior' components. In this
   case the dependency will be specified as a range of allowed versions. For
   example, a dependency version 1.2 will mean a (1.2.0 = version  1.3)
   range.
   If the version of a dependency is not specified (like in this existing
   app - [1]) then we assume the version 0 - i.e. the last available
   pre-release version of a package.

   4. Murano core library is also a package which has its own version. The
   current one is assumed to have a version 0.1.0, the one which is going to
   be released in L will be probably called 0.2.0. The lib is still quickly
   evolving, so we are not releasing a 1.0.0 until we are sure that we are not
   going to have any breaking changes anytime soon.
   As with any other package it will be possible to have several versions
   of the Core Library installed in Murano at the same moment of time.

   5. There is no mandatory need to add the the dependency on the core
   library to the Requires block of each application, as it is added there
   implicitly. However, this implicit dependency will have a version 0 -
   i.e. will reference the latest pre-release version of the Core Library
   available. So it is still better to pin the core library requirement to a
   particular version to make sure that your app does not break if we
   introduce any breaking change into the core lib.

   6. All classes defined in a package are assumed to have a version
   identical to the version of the package.

   7. Murano Extension Plugins (i.e python packages which declare
   setuptools-entrypoints in 'io.murano.extensions' namespace) also will have
   similar versioning semantics: they will have a fully qualified name
   (defined by the setuptools' package name) and a version (also defined by
   setuptools), an will get an ability to specify their own dependencies if
   needed. From the class-loader perspective the MuranoPL classes defined in
   the plugins are no difference from the classes defined in a regular package.

   8. We are going to store murano packages as Glance V3 Artifacts
   Repository, naturally mapping package's FQN and version to artifact's name
   and version.
   The package dependencies will be stored in Glance as cross-artifact
   dynamic dependencies (i.e. dependencies not on a particular artifact but on
   the last artifact matching the given name and the version range

Re: [openstack-dev] [nova] Switching the bug status of ~200 bugs at once; Problem?

2015-07-10 Thread Gary Kotton

+1

On 7/10/15, 1:32 PM, Sean Dague s...@dague.net wrote:

On 07/10/2015 04:09 AM, Markus Zoeller wrote:
 I'd like to switch the status of ~200 bugs at once to adjust the
 bookkeeping in LaunchPad. This will concern Confirmed and Triaged
 bugs which have an assignee [1]. AFAIK they should be In Progress.
 
 I'm concerned if this would affect some reports I'm not aware of in
 a bad way. So, if you think this is a bad thing, please speak up.
 
 FWIW, I'll use an enhanced version of [2] which I will provide in the
 next days. A few days later I'll have a version ready which can
 update stale In Progress bugs by querying Gerrit.
 
 [1] 
https://urldefense.proofpoint.com/v2/url?u=http-3A__bit.ly_1GbhJWud=BQIC
Agc=Sqcl0Ez6M0X8aeM67LKIiDJAXVeAw-YihVMNtXt-uEsr=VlZxHpZBmzzkWT5jqz9JYB
k8YTeq9N3-diTlNj4GyNcm=l81nvT6_KcJY11CmckP4tRUoT25Y8Hn7zMMGBuD2vC8s=0Jw
7eQetRMf4t5LhJi52-n_tjW5h71s56yiu-UO6eh4e=
 [2] https://review.openstack.org/#/c/197060/

I would not do that. Bugs with assignee tend to get ignored by other
people looking at the problem, and in *most* cases the assignee is not
actually working on the bug. I would instead go the other direction and
pop any In Progress bug that has seen no activity in the last 60 days
back to Confirmed.

   -Sean

-- 
Sean Dague
https://urldefense.proofpoint.com/v2/url?u=http-3A__dague.netd=BQICAgc=S
qcl0Ez6M0X8aeM67LKIiDJAXVeAw-YihVMNtXt-uEsr=VlZxHpZBmzzkWT5jqz9JYBk8YTeq9
N3-diTlNj4GyNcm=l81nvT6_KcJY11CmckP4tRUoT25Y8Hn7zMMGBuD2vC8s=8XJOGkHSUvo
Uv8dQrHymUt_XXKtMf0hpWsKYuUzoWnse=

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [magnum] Tom Cammann for core

2015-07-10 Thread Steven Dake (stdake)

+1 welcome aboard Tom!

Regards
-steve


On 7/9/15, 7:20 PM, Adrian Otto adrian.o...@rackspace.com wrote:

Team,

Tom Cammann (tcammann) has become a valued Magnum contributor, and
consistent reviewer helping us to shape the direction and quality of our
new contributions. I nominate Tom to join the magnum-core team as our
newest core reviewer. Please respond with a +1 vote if you agree.
Alternatively, vote -1 to disagree, and include your rationale for
consideration.

Thanks,

Adrian
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [magnum] Tom Cammann for core

2015-07-10 Thread Hongbin Lu

+1 Welcome Tom!

-Original Message-
From: Adrian Otto [mailto:adrian.o...@rackspace.com] 
Sent: July-09-15 10:21 PM
To: OpenStack Development Mailing List
Subject: [openstack-dev] [magnum] Tom Cammann for core

Team,

Tom Cammann (tcammann) has become a valued Magnum contributor, and consistent 
reviewer helping us to shape the direction and quality of our new 
contributions. I nominate Tom to join the magnum-core team as our newest core 
reviewer. Please respond with a +1 vote if you agree. Alternatively, vote -1 to 
disagree, and include your rationale for consideration.

Thanks,

Adrian
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [testing] moving testrepository outside the tox venv

2015-07-10 Thread Ian Cordasco

On 7/10/15, 18:34, Monty Taylor mord...@inaugust.com wrote:

On 07/10/2015 07:19 PM, Robert Collins wrote:
 On 10 July 2015 at 01:59, Morgan Fainberg morgan.fainb...@gmail.com
wrote:
 Or a database per python major version (or at least gracefully handle
the incompatibility).
 
 So that would partition the data, and the whole point of test
 *repository* is that it builds a database across all your tests to
 answer useful questions.

Totally understand that.

On the other hand ...

a) is that setup/behavior common enough to be more important than:

b) the dbm file format mess is both annoying and confusing

I get the theory of why - but it's also a common thing that provides rage.

Just as an example, I went to fix a pypy error and was receiving an
exception about not being able to import _bsddb. The fix? Remove
.testrepository. I understand testrepository is meant to be a metatool,
but it provides very bizarre problems in some cases. I understand why we
use it, but the situation really need to improve the situation.


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [testing] moving testrepository outside the tox venv

On 9 July 2015 at 18:34, Flavio Percoco fla...@redhat.com wrote:
 On 08/07/15 22:52 +, Jeremy Stanley wrote:

 On 2015-07-09 10:37:17 +1200 (+1200), Robert Collins wrote:

 So - I'm looking to:

 A) have a discussion and identify any issues with moving testr out of
 the venvs. (Note: this doesn't mean stop using it, just removing it
 from test-requirements.txt, in the same way that tox isn't in
 test-requirements.txt).
 B) Capture that in a spec if its non-trivial.
 C) find volunteers to make it happen.


 D) keep reminding developers to install it on their systems when
 they ask why they can't run tests

 E) keep reminding developers to upgrade to a newer version when they
 start running into bugs which aren't exhibited in our CI

 I think the original decision to install it inside the tox
 virtualenv was because:

 1. it made migrating from nose easier because we didn't have to add
 new steps in the basic workflow

 2. it's one less thing developers need to know to install on their
 systems

 3. it makes sure a new enough version is being used (matching the
 version used in our CI)

 I'm not arguing against the change, but think it's worth
 acknowledging the (perhaps marginal) ongoing costs of the solution
 and asking whether they're outweighed by the ongoing cost of working
 around the problem.


 I'm not against the proposal in this thread but I'd like to throw another
 option out there:

 We could also make testr create the `.testrepository` dir in the venv
 when running under tox. If this is currently not possible, it'd be a
 matter of adding a CLI param to it that we can pass to make it create
 the dir where we want.

 `tox` has a `toxworkdir` option that you can set to specify the...
 workdir :)

I'd be ok with that in principle. It will requiring some separation of
self.ui.here which currently roots all lookups - both repo and config.

But, testr is going to learn about test profiles to deal with the need
to run the same tests in many environments - I'm going to tie it into
tox in some fashion.

The design of testr is to be a metatool, fundamentally different to
e.g. testtools which is very location-and-context specific.

-Rob

-- 
Robert Collins rbtcoll...@hp.com
Distinguished Technologist
HP Converged Cloud

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [testing] moving testrepository outside the tox venv

2015-07-10 Thread Monty Taylor

On 07/10/2015 07:19 PM, Robert Collins wrote:
 On 10 July 2015 at 01:59, Morgan Fainberg morgan.fainb...@gmail.com wrote:
 Or a database per python major version (or at least gracefully handle the 
 incompatibility).
 
 So that would partition the data, and the whole point of test
 *repository* is that it builds a database across all your tests to
 answer useful questions.

Totally understand that.

On the other hand ...

a) is that setup/behavior common enough to be more important than:

b) the dbm file format mess is both annoying and confusing

I get the theory of why - but it's also a common thing that provides rage.

 --Morgan

 Sent via mobile

 On Jul 9, 2015, at 06:43, Doug Hellmann d...@doughellmann.com wrote:

 Excerpts from Robert Collins's message of 2015-07-09 10:37:17 +1200:
 I don't remember if this has previously been discussed, but as our
 Python3 readiness increases folk are going to feel pain from testr due
 to a silly Python 2/3 incompatibility around dbm files.

 https://bugs.launchpad.net/testrepository/+bug/1212909

 OpenStack (alone in the universe as far as I can tell) installs
 testrepository *inside* a tox venv. This then causes the timing
 database to be Python version specific. It also causes the test venv
 to suck in any dependencies testr might add.

 But testrepository is like tox- its meant to be installed systemwide
 (or at least in its own venv) and then just used to *run* test runners
 within whatever context.

 So - I'm looking to:

 A) have a discussion and identify any issues with moving testr out of
 the venvs. (Note: this doesn't mean stop using it, just removing it
 from test-requirements.txt, in the same way that tox isn't in
 test-requirements.txt).
 B) Capture that in a spec if its non-trivial.
 C) find volunteers to make it happen.

 How much work would it be to make testrepository use a database format
 that would be the same under all versions of python?
 
 I'd love a patch. Relevant tests:
 Mac OS X's default Python
 Ubuntu's default Python 2.x + 3.x, for newest LTS + current dev.
 Ditto Fedora
 And Python 2.7 and 3.4 and 3.5 for Windows (upstream distribution)
 
 -Rob
 
 


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [testing] moving testrepository outside the tox venv

2015-07-10 Thread Matthew Treinish

On Sat, Jul 11, 2015 at 11:35:16AM +1200, Robert Collins wrote:
 On 9 July 2015 at 10:52, Jeremy Stanley fu...@yuggoth.org wrote:
  On 2015-07-09 10:37:17 +1200 (+1200), Robert Collins wrote:
  So - I'm looking to:
 
  A) have a discussion and identify any issues with moving testr out of
  the venvs. (Note: this doesn't mean stop using it, just removing it
  from test-requirements.txt, in the same way that tox isn't in
  test-requirements.txt).
  B) Capture that in a spec if its non-trivial.
  C) find volunteers to make it happen.
 
  D) keep reminding developers to install it on their systems when
  they ask why they can't run tests
 
 We can make the os-testr which *is* installed in the venv look for
 testr in the PATH and error with a good explanation when its not
 there.

I guess we could do this, it would be very simple to implement and I'm not
necessarily opposed to it, if that's the direction everyone wants to go. Right
now ostestr only interacts with testr through subprocess, which makes the
implementation of something like this quite simple.

But, I have 2 concerns right now: First was that my medium/long term my plan was
to use testr's ui layer to implement ostestr's cli instead of the dirty
subprocess hack I did to get things done faster. Doing this would require either
breaking out of the venv to get the testr imports (which I wouldn't really want
to do), require all the tox venv use system site-packages (which I don't think
is really a good idea, and wouldn't solve the python version issue) or making
sure testr was installed in the venv too.

The other is ostestr isn't universally used right now, only a few projects are
using the ostestr CLI. os-testr is only really being installed in the venv
everywhere because projects use subunit-trace, which is also part of os-testr,
via a pretty_tox.sh script. So to make this a solution we would have to migrate
everything over to ostestr first. Which wouldn't really be an issue and it's
something I want to do eventually.

TBH, I tend to agree with everyone else's opinion elsewhere on this thread.
While it might be viewed as an anti-pattern for how testr was originally
intended to be used it doesn't change that this is pretty much the model used
and expected by everything and everyone using testr (or any test runner) in
OpenStack. I would view doing something like this as more of a workaround than
just fixing the storage engine to be compatible with multiple python versions.

The whole argument for making testr live outside of the venv and being an
implicit dependency like tox is based around tracking the results between the
tox venvs right? If we decoupled the database and other result storage from
testr a bit and made that the external platform independent piece wouldn't we
maintain everything this thread is trying to address without having the issues
we're seeing now or requiring that everyone change their usage model and
expectations?

-Matt Treinish

 
  E) keep reminding developers to upgrade to a newer version when they
  start running into bugs which aren't exhibited in our CI
 
 We already have to do that (because we don't hard-require -U) - we
 could easily enough cross-check the version of testr from os-testr too
 and issue appropriate errors.
 
  I think the original decision to install it inside the tox
  virtualenv was because:
 
  1. it made migrating from nose easier because we didn't have to add
  new steps in the basic workflow
 
  2. it's one less thing developers need to know to install on their
  systems
 
  3. it makes sure a new enough version is being used (matching the
  version used in our CI)
 
  I'm not arguing against the change, but think it's worth
  acknowledging the (perhaps marginal) ongoing costs of the solution
  and asking whether they're outweighed by the ongoing cost of working
  around the problem.
 
 Certainly there are tradeoffs.
 
 -Rob
 
 -- 
 Robert Collins rbtcoll...@hp.com
 Distinguished Technologist
 HP Converged Cloud


signature.asc
Description: PGP signature
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Neutron] adding new vendor driver to upstream

2015-07-10 Thread Vadivel Poonathan

Hi Armando,

Thanks for the pointer. I will go thru the same and get back to you/group,
in case of any clarification.

Here is the outdated wiki pages that i came across... these may have to be
updated/cleaned-up to reflect the recent changes about
development/contribution of new vendor plugins/drivers...
https://wiki.openstack.org/wiki/Neutron_Plugins_and_Drivers
https://wiki.openstack.org/wiki/NeutronDevelopment
https://wiki.openstack.org/wiki/Neutron

Thanks,
Vad
--


On Fri, Jul 10, 2015 at 4:54 PM, Armando M. arma...@gmail.com wrote:


 On 10 July 2015 at 16:01, Vadivel Poonathan vadivel.openst...@gmail.com
 wrote:

 Hi,

 I have a Neutron plugin (actually a mechanism_driver under ML2) developed
 for Alcatel-Lucent Omniswitches and is currently being used. But it is not
 part of Neutron upstream, nor listed in the docs/wiki section. I tried to
 make it part of Kilo release, but that was when the decomposition proposal
 was going on. Infact i reviewed the blueprint spec and provided some
 comments too. Since i had to decompose my driver as per the Kilo's
 decomposition spec, i deferred it to make it as part of Liberty release.

 Now going thru some wiki pages, blueprint specs etc, i 'm not clear on
 the procedures/criteria to make my driver integrated with upstream
 Neutron.  Its all scattered and some says all vendor code is moved
 out-of-tree in Liberty, some says only vendor library is moved, some says
 third-party CI system is no longer required, some says it is one of the key
 requirement.


 This is the most up-to-date place to look for to get you started:


 https://github.com/openstack/neutron/blob/master/doc/source/devref/contribute.rst

 There is no wiki afaik, and only one spec that targeted the Kilo release,
 so I I'd be curious to know what you mean by 'it's all scattered', as I'd
 love to clean this up if possible.



 *So i appreciate if someone could get me the exact step-by-step
  procedure (the most recent, applicable to Liberty release) to have my
 driver released as part of Liberty and its documentation. *


 If you still have questions, feel free to reach us out on
 #openstack-neutron.

 A.




 Here is my objective...
 1) make my mechanism driver pkg available in the Neutron repository,
 accessible by public
 2) update my mechanism driver in the list of supported vendor
 plugin/driver page, along with some documentation

 Pls. advise.

 Thanks and Regards,
 Vad
 --

 __
 OpenStack Development Mailing List (not for usage questions)
 Unsubscribe:
 openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



 __
 OpenStack Development Mailing List (not for usage questions)
 Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [testing] moving testrepository outside the tox venv

On 11 July 2015 at 11:34, Monty Taylor mord...@inaugust.com wrote:
 On 07/10/2015 07:19 PM, Robert Collins wrote:
 On 10 July 2015 at 01:59, Morgan Fainberg morgan.fainb...@gmail.com wrote:
 Or a database per python major version (or at least gracefully handle the 
 incompatibility).

 So that would partition the data, and the whole point of test
 *repository* is that it builds a database across all your tests to
 answer useful questions.

 Totally understand that.

 On the other hand ...

 a) is that setup/behavior common enough to be more important than:

 b) the dbm file format mess is both annoying and confusing

 I get the theory of why - but it's also a common thing that provides rage.

It happens and creates rage *because* its installed in the venv. Fix
that unsupported and against-design issue and b) stops happening.

-Rob

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Neutron] adding new vendor driver to upstream

2015-07-10 Thread Armando M.

On 10 July 2015 at 16:01, Vadivel Poonathan vadivel.openst...@gmail.com
wrote:

 Hi,

 I have a Neutron plugin (actually a mechanism_driver under ML2) developed
 for Alcatel-Lucent Omniswitches and is currently being used. But it is not
 part of Neutron upstream, nor listed in the docs/wiki section. I tried to
 make it part of Kilo release, but that was when the decomposition proposal
 was going on. Infact i reviewed the blueprint spec and provided some
 comments too. Since i had to decompose my driver as per the Kilo's
 decomposition spec, i deferred it to make it as part of Liberty release.

 Now going thru some wiki pages, blueprint specs etc, i 'm not clear on the
 procedures/criteria to make my driver integrated with upstream Neutron.
 Its all scattered and some says all vendor code is moved out-of-tree in
 Liberty, some says only vendor library is moved, some says third-party CI
 system is no longer required, some says it is one of the key requirement.


This is the most up-to-date place to look for to get you started:

https://github.com/openstack/neutron/blob/master/doc/source/devref/contribute.rst

There is no wiki afaik, and only one spec that targeted the Kilo release,
so I I'd be curious to know what you mean by 'it's all scattered', as I'd
love to clean this up if possible.



 *So i appreciate if someone could get me the exact step-by-step  procedure
 (the most recent, applicable to Liberty release) to have my driver released
 as part of Liberty and its documentation. *


If you still have questions, feel free to reach us out on
#openstack-neutron.

A.




 Here is my objective...
 1) make my mechanism driver pkg available in the Neutron repository,
 accessible by public
 2) update my mechanism driver in the list of supported vendor
 plugin/driver page, along with some documentation

 Pls. advise.

 Thanks and Regards,
 Vad
 --

 __
 OpenStack Development Mailing List (not for usage questions)
 Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [testing] moving testrepository outside the tox venv

On 9 July 2015 at 10:52, Jeremy Stanley fu...@yuggoth.org wrote:
 On 2015-07-09 10:37:17 +1200 (+1200), Robert Collins wrote:
 So - I'm looking to:

 A) have a discussion and identify any issues with moving testr out of
 the venvs. (Note: this doesn't mean stop using it, just removing it
 from test-requirements.txt, in the same way that tox isn't in
 test-requirements.txt).
 B) Capture that in a spec if its non-trivial.
 C) find volunteers to make it happen.

 D) keep reminding developers to install it on their systems when
 they ask why they can't run tests

We can make the os-testr which *is* installed in the venv look for
testr in the PATH and error with a good explanation when its not
there.

 E) keep reminding developers to upgrade to a newer version when they
 start running into bugs which aren't exhibited in our CI

We already have to do that (because we don't hard-require -U) - we
could easily enough cross-check the version of testr from os-testr too
and issue appropriate errors.

 I think the original decision to install it inside the tox
 virtualenv was because:

 1. it made migrating from nose easier because we didn't have to add
 new steps in the basic workflow

 2. it's one less thing developers need to know to install on their
 systems

 3. it makes sure a new enough version is being used (matching the
 version used in our CI)

 I'm not arguing against the change, but think it's worth
 acknowledging the (perhaps marginal) ongoing costs of the solution
 and asking whether they're outweighed by the ongoing cost of working
 around the problem.

Certainly there are tradeoffs.

-Rob

-- 
Robert Collins rbtcoll...@hp.com
Distinguished Technologist
HP Converged Cloud

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [testing] moving testrepository outside the tox venv

2015-07-10 Thread Matthew Treinish

On Thu, Jul 09, 2015 at 06:59:44AM -0700, Morgan Fainberg wrote:
 Or a database per python major version (or at least gracefully handle the 
 incompatibility). 
 
 --Morgan

 Sent via mobile

  On Jul 9, 2015, at 06:43, Doug Hellmann d...@doughellmann.com wrote:
  
  Excerpts from Robert Collins's message of 2015-07-09 10:37:17 +1200:
  I don't remember if this has previously been discussed, but as our
  Python3 readiness increases folk are going to feel pain from testr due
  to a silly Python 2/3 incompatibility around dbm files.
  
  https://bugs.launchpad.net/testrepository/+bug/1212909
  
  OpenStack (alone in the universe as far as I can tell) installs
  testrepository *inside* a tox venv. This then causes the timing
  database to be Python version specific. It also causes the test venv
  to suck in any dependencies testr might add.
  
  But testrepository is like tox- its meant to be installed systemwide
  (or at least in its own venv) and then just used to *run* test runners
  within whatever context.
  
  So - I'm looking to:
  
  A) have a discussion and identify any issues with moving testr out of
  the venvs. (Note: this doesn't mean stop using it, just removing it
  from test-requirements.txt, in the same way that tox isn't in
  test-requirements.txt).
  B) Capture that in a spec if its non-trivial.
  C) find volunteers to make it happen.
  
  How much work would it be to make testrepository use a database format
  that would be the same under all versions of python?
  
  Doug

My long term plan (for almost a year now :( ) to do this was to add a new
repository type for testrepository that leverages subunit2sql. We could then
just use a local sqlite database to track the results which would avoid the
dbm compat issues and also expose a much more feature rich interface to use
the stored data.

The current blocker for this right now is sqlite support in subunit2sql. [1]
Some of the alembic migrations do not work on sqlite, and I haven't really had
a chance to work on fixing this. If someone was willing to help me on this that
would make this move a lot faster.

-Matt Treinish


[1] We could start work on adding the new repository type now, but to use it
would would require a mysql or postgres db which is too heavyweight to be really
useful for most use cases


signature.asc
Description: PGP signature
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [glance] Why does Glance keep the deleted membership of image ?

2015-07-10 Thread Nikhil Komawar

Response inline.

 I can't understand how the impact on performance, image-members still
 have an idx. Is there any other concern on the patch ?  How to get
 result from rally gate job ?

 Can you give me suggestion on how to move forward ?  Thanks .


Your change isn't. But including deleted is increasing the possibility
of using that method somewhere else with a wider criterion. Image member
is idxed but the criterion that can be used to find members is not
necessarily going to be.

I think adding a NOTE to that method mentioning what cases it's safe to
use it should help. This looks like a okay tradeoff given the call is
mostly safe. We can discuss further on the review as ML shouldn't be
used for such.



 Best regards,
 LongQuan


 Inactive hide details for Nikhil Komawar ---2015/07/10
 22:34:16---Please find the response inline. Nikhil Komawar
 ---2015/07/10 22:34:16---Please find the response inline. 

 From: Nikhil Komawar nik.koma...@gmail.com
 To: OpenStack Development Mailing List (not for usage questions)
 openstack-dev@lists.openstack.org
 Cc: Long Quan Sha/China/IBM@IBMCN
 Date: 2015/07/10 22:34
 Subject: Re: [openstack-dev] [glance] Why does Glance keep the deleted
 membership of image ?

 



 Please find the response inline.


 Hi Glance experts,

 I'd like to send this mail again, hope I can get help and suggest
 from glance experts. The question is from a bug
 _https://bugs.launchpad.net/glance/+bug/1462315_,

 If an image-member is deleted, then create it again with the same
 parameters, glance searches db to see if there is already an
 existing one, but the result doesn't include the record which was
 marked as deleted,
 glance will try to create a new one with the same parameters, it
 works well on mysql. But it is failed on DB2 with SQL0803N error.

 The root cause is that DB2 constraint is more restricted than
 mysql. For db2, the columns under unique constrains should be NOT
 NULL, currently the column deleted_at which is one of unique
 constrain of image_members
 is nullable. A possible solution is to alter it to not null in
 migration. that means we have to insert a default timestamp value
 for the new created image-member, an active member with a no-blank
 timestamp for deleted_at seems very confusing.  



 Agree that this is confusing. And changing the behavior this way is
 NOT a good idea. A record that's never been deleted should not have
 deleted(_at) value. It will affect notifications and conflict with API
 guidelines.


 Another fix is: we may check all existing image-member records
 including the deleted image-member before create image-member,
 then update it if it exists, otherwise create a new one, that is
 proposed in _https://review.openstack.org/#/c/190895/_


 I'm wondering why can't we use only one record to maintain the
 member-ship between a pair of image and tenant. Maybe there is
 some other consideration, can you help give me some suggestion ?
 I'd like to know more. Thanks.



 Concept wise this sounds like a good idea but it could have
 performance degradation impact. Nonetheless, image-members have an idx
 that should be a relief for that query image_member_find that you
 added in your proposal. My hope is that the rally gate job will tell
 us more if there is a performance problem.


 Best regards,
 LongQuan


 __
 OpenStack Development Mailing List (not for usage questions)
 Unsubscribe:
 _openstack-dev-requ...@lists.openstack.org?subject:unsubscribe_
 mailto:openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
 _http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev_


 -- 

 Thanks,
 Nikhil

-- 

Thanks,
Nikhil


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Murano] Versioning of Murano packages and MuranoPL classes

2015-07-10 Thread Georgy Okrokvertskhov

Hi Alex,

Thank you for the great summary.

I have a concern about item #8. Can we add an option to Murano to use
previous storage engine rather then Glance v3? We need to make sure that v3
API in Glance is set by default before we do a hard dependency on it in
Murano.

Thanks
Gosha

On Fri, Jul 10, 2015 at 4:53 AM, Alexander Tivelkov ativel...@mirantis.com
wrote:

 Hi folks,

 Ability to manage multiple versions of application packages and their
 dependencies was always an important item in Murano roadmap, however we
 still don't have a clear spec for this feature.
 Yesterday we hosted a small design session to come up with a plan on what
 can be done in Liberty timeframe to have proper versioning for MuranoPL
 classes and packages. Stan Lagun, Kirill Zaitsev and myself participated
 offline, some other muranoers joined remotely. Thanks to everybody who
 joined us.

 TL;DR: it turns out that now we have a clear plan which will allow us to
 achieve proper versioning of the packages and classes, and we'll try to
 implement the most important parts of it in Liberty.

 Here is the detailed outcome of the session:


1. We'll use the standard Semantic Versioning format
('Major.Minor.Patch[-dev-build.label[+metadata.label]]') to version our
packages: changes which break backwards-compatibility should increment the
major segment, non-breaking new features increment the minor segment and
all non-breaking bugfixes increment the patch segment. The developers
should be carefull with the new features part: if you add a new method to
a class, it may be considered a breaking change if the existing subclasses
of this class have the same method already declared. We still assume that
such changes should lead to increment of 'minor' segment, however it is up
to best judgement of developers in particular case: if the risk of such
method override is really high it may worth to increment the 'major'
segment. Proper guideline on the versioning rules will be published closer
to L release.

2. A new field 'Version' is introduced into package manifest file
which should define package version in complete semver format. The field
itself is optional (so existing apps are not broken), if it is not
specified the package is assumed to have version '0.0.0'

3. The existing 'Require' block of Application manifest will be used
to specify the package dependencies. Currently it is a yaml-based
dictionary, with the keys set to fully-qualified names of the dependency
packages and the values set to the version of those dependencies. Currently
this block is used only for integration with apps stored at
apps.openstack.org. It is suggested to use this block in the
deployment process as well, and extend its semantics.
The version of the dependency specified there should also follow the
semver notation, however it may be specified in the shortened format, i.e.
without specifying the 'patch' or 'patch' and 'minior' components. In this
case the dependency will be specified as a range of allowed versions. For
example, a dependency version 1.2 will mean a (1.2.0 = version  1.3)
range.
If the version of a dependency is not specified (like in this existing
app - [1]) then we assume the version 0 - i.e. the last available
pre-release version of a package.

4. Murano core library is also a package which has its own version.
The current one is assumed to have a version 0.1.0, the one which is going
to be released in L will be probably called 0.2.0. The lib is still quickly
evolving, so we are not releasing a 1.0.0 until we are sure that we are not
going to have any breaking changes anytime soon.
As with any other package it will be possible to have several versions
of the Core Library installed in Murano at the same moment of time.

5. There is no mandatory need to add the the dependency on the core
library to the Requires block of each application, as it is added there
implicitly. However, this implicit dependency will have a version 0 -
i.e. will reference the latest pre-release version of the Core Library
available. So it is still better to pin the core library requirement to a
particular version to make sure that your app does not break if we
introduce any breaking change into the core lib.

6. All classes defined in a package are assumed to have a version
identical to the version of the package.

7. Murano Extension Plugins (i.e python packages which declare
setuptools-entrypoints in 'io.murano.extensions' namespace) also will have
similar versioning semantics: they will have a fully qualified name
(defined by the setuptools' package name) and a version (also defined by
setuptools), an will get an ability to specify their own dependencies if
needed. From the class-loader perspective the MuranoPL classes defined in
the plugins are no

Re: [openstack-dev] [manila][devstack] xtrace in devstack plugin

2015-07-10 Thread Valeriy Ponomaryov

Hello Emmanuel,

There is no strong reason to keep it disabled. Thanks for pointing  this
out.
I agree that it is better to have enabled.
So, I already did a change for it - https://review.openstack.org/#/c/200585/

Regards,
Valeriy Ponomaryov

On Wed, Jul 8, 2015 at 12:10 PM, Emmanuel Cazenave cont...@emcaz.fr wrote:

 Oups, forgot the [openstack-dev] in the subject.

 __
 OpenStack Development Mailing List (not for usage questions)
 Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev




-- 
Kind Regards
Valeriy Ponomaryov
www.mirantis.com
vponomar...@mirantis.com
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Fuel] wrong network for keystone endpoint in 6.1 ?

2015-07-10 Thread Vladimir Kuklin

Dani

You are always welcome - I am adding fuel documentation team into the
thread.

On Fri, Jul 10, 2015 at 5:45 PM, Daniel Comnea comnea.d...@gmail.com
wrote:

 Okay Vladimir, thanks for confirmation!

 So then you happy to stick my sketch proposal (of course needs re-wording)
 into documentation?

 Dani

 On Fri, Jul 10, 2015 at 11:31 AM, Vladimir Kuklin vkuk...@mirantis.com
 wrote:

 Daniel

 Yes, if you want to do some administrative stuff you need to have access
 to management network to be able to work with internal and admin endpoints.

 On Fri, Jul 10, 2015 at 9:58 AM, Daniel Comnea comnea.d...@gmail.com
 wrote:

 I know about the flow but what i'm questioning is:

 admin endpoint is mapped to br-mgmt subnet (you do have the HAproxy as
 below defined in 6.1. In 6.0 and before you had no HAproxy)

 listen keystone-2
   bind 192.168.20.3:35357
   option  httpchk
   option  httplog
   option  httpclose
   server node-17 192.168.20.20:35357   check inter 10s fastinter 2s
 downinter 3s rise 3 fall 3
   server node-18 192.168.20.21:35357   check inter 10s fastinter 2s
 downinter 3s rise 3 fall 3
   server node-23 192.168.20.26:35357   check inter 10s fastinter 2s
 downinter 3s rise 3 fall 3

 public endpoint is mapped to br-ex

 So with this behavior you are saying the bt-mgmt subnet (which i thought
 is only for controller  compute traffic, isolated network) should be
 routable in the same way br-ex is?

 Dani


 On Thu, Jul 9, 2015 at 11:30 PM, Stanislaw Bogatkin 
 sbogat...@mirantis.com wrote:

 Hi Daniel,

 answer is no - actually there is no strong dependency between public
 and internal/admin endpoints. In your case keystone client ask keystone on
 address 10.52.71.39 (which, I think, was provided by system
 variable OS_AUTH_URL), auth on it and then keystone give endpoints list to
 client. Client selected admin endpoint from this list (192.168.20.3
 address) and tried to get information you asked. It's a normal behavior.

 So, in Fuel by default we have 3 different endpoints for keystone -
 public on public VIP, port 5000; internal on management VIP, port 5000,
 admin on management VIP, port 35357.

 On Thu, Jul 9, 2015 at 4:59 PM, Daniel Comnea comnea.d...@gmail.com
 wrote:

 Hi,

 I'm running Fuel 6.1 and i've seen an interesting behavior which i
 think match bug [1]

 Basically the adminUrl  publicUrl part of keystone endpoint are
 different

 And the result of that is that you can't run keystone cli - i.e
 create/list tenants etc

 keystone --debug tenant-list
 /usr/local/lib/python2.7/site-packages/keystoneclient/shell.py:65:
 DeprecationWarning: The keystone CLI is deprecated in favor of python-
 openstackclient. For a Python library, continue using python-keys
 toneclient.
   'python-keystoneclient.', DeprecationWarning)
 DEBUG:keystoneclient.auth.identity.v2:Making authentication request
 to http://10.20.71.39:5000/v2.0/tokens
 INFO:requests.packages.urllib3.connectionpool:Starting new HTTP
 connection (1): 10.52.71.39
 DEBUG:requests.packages.urllib3.connectionpool:POST /v2.0/tokens
 HTTP/1.1 200 3709
 DEBUG:keystoneclient.session:REQ: curl -g -i -X GET
 http://192.168.20.3:35357/v2.0/tenants -H User-Agent: python-
 keystoneclient -H Accept: application/json -H X-Auth-Token:
 {SHA1}cc918b89c2dca563edda43e01964b1f1979c552b

 shouldn't adminURL = publicURL = br-ex for keystone?


 Dani


 [1] https://bugs.launchpad.net/fuel/+bug/1441855


 __
 OpenStack Development Mailing List (not for usage questions)
 Unsubscribe:
 openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev




 __
 OpenStack Development Mailing List (not for usage questions)
 Unsubscribe:
 openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev




 __
 OpenStack Development Mailing List (not for usage questions)
 Unsubscribe:
 openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev




 --
 Yours Faithfully,
 Vladimir Kuklin,
 Fuel Library Tech Lead,
 Mirantis, Inc.
 +7 (495) 640-49-04
 +7 (926) 702-39-68
 Skype kuklinvv
 35bk3, Vorontsovskaya Str.
 Moscow, Russia,
 www.mirantis.com http://www.mirantis.ru/
 www.mirantis.ru
 vkuk...@mirantis.com

 __
 OpenStack Development Mailing List (not for usage questions)
 Unsubscribe:
 openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



 __
 OpenStack Development Mailing List (not for usage questions)
 Unsubscribe:

Re: [openstack-dev] [Fuel] Separate repo for Fuel Agent

2015-07-10 Thread Vladimir Kozhukalov

Guys, we are next to moving fuel_agent directory into a separate
repository. Action flow is going to be as follows:

1) Create verify jobs on our CI https://review.fuel-infra.org/#/c/9186
(DONE)
2) Freeze fuel_agent directory in https://github.com/stackforge/fuel-web
(will announce in a separate mail thread). That means we stop merging
patches into master which change fuel_agent directory. Unfortunately, all
review requests need to be re-sent, but it is not going to be very
difficult.
3) Create temporary upstream repository with fuel_agent/* as a content. I'm
not planning to move 5.x and 6.x branches. Only master. So, all fixes for
5.x and 6.x will be living in fuel-web.
4) This upstream repository is going to be sucked in by openstack-infra.
Patch is here https://review.openstack.org/#/c/199178/ (review is welcome)
I don't know how long it is going to take. Will try to poke infra people to
do this today.
5) Then we need to accept two patches into new fuel-agent repository:
 - rpm spec (extraction from fuel-web/specs/nailgun.spec) (ready, but there
is no review request)
 - run_tests.sh (to run tests) (ready, but there is no review request)

!!! By this moment there won't be any impact on ISO build process !!!

6) Then we need to change two things at the same time (review is welcome)
  - fuel-web/specs/nailgun.spec in order to prevent fuel-agent package
building  https://review.openstack.org/#/c/200595
  - fuel-main so as to introduce new fuel-agent repository into the build
process https://review.openstack.org/#/c/200025

And good luck to me -)


Vladimir Kozhukalov

On Wed, Jul 8, 2015 at 12:53 PM, Vladimir Kozhukalov 
vkozhuka...@mirantis.com wrote:

 There were some questions from Alexandra Fedorova about independent
 release cycle.

 according to the configuration [1] Infra team won't be able to do
 branching or any kind of release management for new repository.

 Could you please clarify, do we plan to version new repository the
 same way as we do for main fuel repositories or there going to be
 separate releases as in python-fuelclient [2]? Who should drive the
 release process for this repo and how this change will affect Fuel ISO
 release?

 [1]
 https://review.openstack.org/#/c/199178/1/gerrit/acls/stackforge/fuel-agent.config,cm
 [2]
 http://lists.openstack.org/pipermail/openstack-dev/2015-July/068837.html

 IMO all Fuel components should be as much independent as possible with
 highly defined APIs used for their interaction, with their own teams, with
 their own independent release cycles. But we cannot switch to this model
 immediately. For the start, we can just move those components into separate
 repositories, leaving the same access rights and core team as we have for
 fuel-web.

 When Fuel Agent is a separate repository we discuss team. It looks like a
 team leader is the best person to manage releases for a particular
 component. This thread is totally about separation stuff and how to do this
 not breaking anything.



 Vladimir Kozhukalov

 On Wed, Jul 8, 2015 at 12:24 PM, Vladimir Kozhukalov 
 vkozhuka...@mirantis.com wrote:

 Dear colleagues,

 I am going to move Fuel Agent into a separate git repository. The thing
 is that we have quite a few review requests to fuel-web with changes for
 Fuel Agent. The new repository is going to look like this
 https://github.com/kozhukalov/fuel-agent i.e. there is no additional
 sub-directory fuel_agent. In fact, I don't think it is a big deal to update
 all fuel agent related review requests.

 Work items:
 0) request to openstack-infra https://review.openstack.org/#/c/199178/1
 0.1) upstream for this request with commit history
 https://github.com/kozhukalov/fuel-agent
 1) fuel-agent/specs/fuel-agent.spec is an extraction from
 fuel-web/specs/nailgun.spec (separate commit, in progress)
 2) modify fuel-main to build fuel-agent package (in progress)
 3) create jenkins-jobs/servers/fuel-ci/verify-fuel-agent.yaml (in
 progress)

 For the start Fuel Agent core team will be the same as in fuel-web.

 If there is anything I forgot, please remind me about that.

 Vladimir Kozhukalov



__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Cinder] python-cinderclient functional tests

2015-07-10 Thread Ivan Kolodyazhny

Hi Mike,

Unfortunately, we don't get a lot of stats [1] because we don't run it
often. I've added 'check experimental' comment to latest
python-cinderclient review request to get more stats.

Review request to make this voting: https://review.openstack.org/#/c/200522/

[1]
http://graphite.openstack.org/render/?width=877height=548_salt=1436533755.887from=00%3A00_20150524until=23%3A59_20150710target=stats.zuul.pipeline.experimental.job.check-cinderclient-dsvm-functional.FAILUREtarget=stats.zuul.pipeline.experimental.job.check-cinderclient-dsvm-functional.SUCCESStarget=stats.zuul.pipeline.experimental.openstack.python-cinderclient.total_changestitle=check-cinderclient-dsvm-functional

Regards,
Ivan Kolodyazhny,
Web Developer,
http://blog.e0ne.info/,
http://notacash.com/,
http://kharkivpy.org.ua/

On Thu, Jul 9, 2015 at 10:14 PM, Mike Perez thin...@gmail.com wrote:

 On 21:48 Jul 06, Ivan Kolodyazhny wrote:

 snip

  Aslo, what do you think about moving cinderclient functional tests from
  experimental to non-voting queue to make it more public and run it with
  every patch to python-cinderclient?

 I'm fine with this as long as the job has been stable since it was
 introduced
 as experimental. You mind posting the stats?

 --
 Mike Perez

 __
 OpenStack Development Mailing List (not for usage questions)
 Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [nova][nova-docker] docker environment variables and error handling

2015-07-10 Thread Federico Michele Facca

hi,
trying to add a little of context to the request of daniel :)

we are playing with nova-docker, and we realized that

1) it's unclear (or not supported) passing enviroment variable through
userdata. probably the current version of IBM Cloud Manager with OpenStack
V4.3 indeed does implement a mechanism [1], but is not commited on
stackforge

2) a large number of containers will fail during the start because of
missing required enviroment parameters. the user in the interface will be
notified a No valid host found, which is quite puzzling and not very user
friendly :)

so we are tyring to find out:

about 1) is there already an official solution? not much work also to
implement the same solution proposed in [1]. most important is agreeing on
how to pass such parameters (also without a metadata server and cloud-init,
e.g. using user_data).

about 2) unfortunately the docker remote client regardless any error during
the start of a containers returns you a simple 204 code. what can be done,
is checking - after the start command - if the container as an error code
using the container inspector. still this has 2 inconvinients: a) if the
inspections occurs before the container exiting with error, you will not be
able to detect the issue; b) we tried to retrieve the error message but
apparently is not available through the docker apis, while on command line
using docker run... you would get a nice message such as: please set this
variable.
any nice idea on how to solve this?

cheers,
federico

[1]
http://www-01.ibm.com/support/knowledgecenter/#!/SST55W_4.3.0/liaca/liaca_container_instance.html

On Fri, Jul 10, 2015 at 11:03 AM, Daniel Depaoli
daniel.depa...@create-net.org wrote:

Hi all.
I have a brief question about nova and nova-docker.
In nova-docker I add a piece of code when I raise an exception, in
particular in 'novadocker/virt/docker/driver.py', function
'_start_container':

*
exitcode=self.docker.inspect_container(container_id)['State']['ExitCode']*
*if exitcode 0:*
*raise exception.NoValidHost(reason=Error in docker exit
code)*

In nova cli or horizon I expect to see my error and not the general error
raised after my code.
Can you help me?

Daniel

--
--
Future Internet is closer than you think!
http://www.fiware.org

Official Mirantis partner for OpenStack Training
https://www.create-net.org/community/openstack-training

--
Dr. Federico M. Facca

CREATE-NET
Via alla Cascata 56/D
38123 Povo Trento (Italy)

P +39 0461 312471
M +39 334 6049758
E federico.fa...@create-net.org
T @chicco785
W www.create-net.org
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [murano] [congress] Congress needs to fetch environments from all tenants.

2015-07-10 Thread Dolph Mathews

How about using domain-based role assignments in keystone and requiring
domain-level authorization in policy, and then only returning data about
the collection of tenants that belong to the authorized domain? That way
you don't have an API that violates multi-tenant isolation, consumable only
by cloud operators.

On Wed, Jul 8, 2015 at 6:27 AM, Filip Blaha filip.bl...@hp.com wrote:

 Hi all,

 I started implement bp [1]. Problem is that congress needs data about
 environments from all tenants but murano API lists only environments of
 user's current tenant. We decided to ipmplement it similarly like listing
 servers in nova where is query parameter all_tenants=true for that (user
 must be admin) I have 2 questions about that:

 1) Are there any security concerns about this approach?
 2) Has someone better idea how to implement this?

 [1]
 https://blueprints.launchpad.net/murano/+spec/murano-api-all-tenants-search

 Regards
 Filip



 __
 OpenStack Development Mailing List (not for usage questions)
 Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [glance] Why does Glance keep the deleted membership of image ?

2015-07-10 Thread Long Quan Sha



I can't understand how the impact on performance, image-members still have
an idx. Is there any other concern on the patch ?  How to get result from
rally gate job ?

Can you give me suggestion on how to move forward ?  Thanks .



Best regards,
LongQuan




From:   Nikhil Komawar nik.koma...@gmail.com
To: OpenStack Development Mailing List (not for usage questions)
openstack-dev@lists.openstack.org
Cc: Long Quan Sha/China/IBM@IBMCN
Date:   2015/07/10 22:34
Subject:Re: [openstack-dev] [glance] Why does Glance keep the deleted
membership of image ?



Please find the response inline.


  Hi Glance experts,

  I'd like to send this mail again, hope I can get help and suggest
  from glance experts. The question is from a bug
  https://bugs.launchpad.net/glance/+bug/1462315,

  If an image-member is deleted, then create it again with the same
  parameters, glance searches db to see if there is already an existing
  one, but the result doesn't include the record which was marked as
  deleted,
  glance will try to create a new one with the same parameters, it
  works well on mysql. But it is failed on DB2 with SQL0803N error.

  The root cause is that DB2 constraint is more restricted than mysql.
  For db2, the columns under unique constrains should be NOT NULL,
  currently the column deleted_at which is one of unique constrain of
  image_members
  is nullable. A possible solution is to alter it to not null in
  migration. that means we have to insert a default timestamp value for
  the new created image-member, an active member with a no-blank
  timestamp for deleted_at seems very confusing.




Agree that this is confusing. And changing the behavior this way is NOT a
good idea. A record that's never been deleted should not have deleted(_at)
value. It will affect notifications and conflict with API guidelines.



  Another fix is: we may check all existing image-member records
  including the deleted image-member before create image-member, then
  update it if it exists, otherwise create a new one, that is proposed
  in https://review.openstack.org/#/c/190895/


  I'm wondering why can't we use only one record to maintain the
  member-ship between a pair of image and tenant. Maybe there is some
  other consideration, can you help give me some suggestion ? I'd like
  to know more. Thanks.





Concept wise this sounds like a good idea but it could have performance
degradation impact. Nonetheless, image-members have an idx that should be a
relief for that query image_member_find that you added in your proposal. My
hope is that the rally gate job will tell us more if there is a performance
problem.



  Best regards,
  LongQuan





  __

  OpenStack Development Mailing List (not for usage questions)
  Unsubscribe:
  openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
  http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


--

Thanks,
Nikhil__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Neutron] New Neutron subproject for a Neutron backed Docker remote networking driver

2015-07-10 Thread Gurucharan Shetty

On Fri, Jul 10, 2015 at 8:16 AM, Gurucharan Shetty shet...@nicira.com wrote:
 From a 2 min look, this seems to wrap docker commands instead of using
 the new plugin
 The plugin is here:
 https://github.com/shettyg/ovn-docker/blob/master/ovn-docker-overlay-driver



 I would very much welcome to join efforts so that we define a vendor neutral,
 Neutron based generic container that provides the container Network plumbing
 using the plugin framework.
 Please continue with what you were doing. If you would like, you can
 use the above plugin driver as a starting point.
I am not familiar with OpenStack sub-project development.  I am happy
to contribute OVN specific code to the generic plugin system that you
have in mind.

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Sahara] Questions about how Sahara use trust ?

2015-07-10 Thread Andrew Lazarev

Hi Chen,

As I remember, proxy users were added for security reasons. When one user
creates cluster in Sahara he should not get access to data of other users.

Thanks,
Andrew.

On Thu, Jul 9, 2015 at 11:12 PM, Li, Chen chen...@intel.com wrote:

  Hi Sahara guys,





 When sahara create a transient cluster, it create a trust with sahara
 admin user.


 https://github.com/openstack/sahara/blob/master/sahara/service/ops.py#L239-L240


 https://github.com/openstack/sahara/blob/master/sahara/service/trusts.py#L79



 When sahara deal with swift, it create a trust too, but :

 sahara admin user = create a proxy domain =  set in sahara.conf

 ð  sahara create proxy user in the domain.

 ð  create a trust with the proxy user.

 https://github.com/openstack/sahara/blob/master/sahara/utils/proxy.py#L110

 https://github.com/openstack/sahara/blob/master/sahara/utils/proxy.py#L265





 My questions are :

 Why not user proxy user for transient cluster ?

 Or, why a proxy user is needed for swift but not use sahara admin user
 directly ?



 Looking forward to your reply.





 Thanks.

 -chen

 __
 OpenStack Development Mailing List (not for usage questions)
 Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all][tests] Fix it friday! [mock failure in CI]

2015-07-10 Thread Victor Stinner


On 10/07/2015 10:42, Robert Collins wrote:

Releasing this on a Friday sounds like bad timing, especially without
advance notice that we'd have to rush to fix the dozens of new issues it
would expose.


There would never be a good time to release such things. The whole
point of the constraints system we're rolling out is to make us not
sensitive to the release schedule of upstreams: we can't dictate the
release schedule of the world to be convenient to our cycles.


I see mock===1.0.1 in upper-constraints.txt, but the python27 check 
job of Swift was broken by the release of mock 1.1.


Can someone please explain me why Swift check job failed?

Patch were I noticed the regression of mock 1.1, on the patch set 6:
https://review.openstack.org/#/c/199034/

(test_wsgi started to fail because the attribute assert_called doesn't 
exist.)


My fix for mock 1.1 in Swift:
https://review.openstack.org/#/c/200474/

Victor

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] [tests] Considering mock alternatives?

2015-07-10 Thread Dmitry Tantsur


On 07/10/2015 01:13 PM, Robert Collins wrote:

On 10 July 2015 at 23:04, Dmitry Tantsur dtant...@redhat.com wrote:

Hi all,

Recent breakage makes me finally raise the question that bothered me for
some time: are there possible alternatives to mock library we could use?


There are. Personally, mock is probably about equal evil-wise to any
mocking library - I find mocking leads to a very heavy dependence on
large functional test suites. But thats a different discussion.


A couple reasons for that:

1. Devs don't seem to care about semver, backward compatibility and all this
boring stuff. Releasing a minor version that breaks all or vast majority of
users is not nice at all.


Fortunately thats not what happened. Openstack had a lot of tests that
were broken, mock 1.1.0 held up a mirror and let us see that. Its the
same mock thats in the standard library - unittest.mock - backported;
any use of unittest.mock from Python 3.5 would have shown the same
thing. There are no breaks to the actual API as far as I can tell[1].


I see that a lot of code started breaking due to

 side_effect = Exception()

no longer working. Which was a declared way for some time, at least to 
my best knowledge. And I do realize it's just a back port, but as a user 
of a _library_ I'd like such breakages to be treated with care.


Maybe it happened and I was just not looking at a proper place.




2. side_effect syntax is no longer sane. Previously it was awesome:

  side_effect = Exception()
  side_effect = [value, Exception()]

etc. Now it's a big typing disaster:

  side_effect = iter([Exception()])

ok, I can live with [], I understand that it may be required for some corner
cases. I can't understand why mock can't call iter() internally.
And seriously, it's a breaking change, and should have been
communicated/issued a warning for some time.


It doesn't show up in the test suite - and I put specific code in
place to deal with the potentially related case where Python 2.7's
next() is different to the 3.x next() [2.6 didn't have next()]. So -
if you've found something that works in unittest.most on Python 3.6
[or 3.5beta3] then its a genuine backport bug and I can fix that up
asap. I haven't seen a bug filed from you about it, so perhaps thats
the first step.


People say it's stopped working in Python 3.4.2 and it did work in 
Python 3.4.0. I didn't verify this sentence.





If someone has contacts within mock team, are there any chances that will
provide a convenient alternative to side_effect (though any new attribute
would be a breaking change for Mock class)?


Any new features should be discussed in either the Python bug tracker
(bugs.python.org) or the python-ideas mailing list. But first lets
make sure this isn't just a bug in the backports.


Any ideas?
Dmitry.


[1]: There is one under investigation from Neutron, but it certainly
wasn't a known issue, and the fairly extensive test suite doesn't show
it.

-Rob





__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all][tests] Fix it friday! [mock failure in CI]

On 2015-07-10 19:45:10 +1200 (+1200), Robert Collins wrote:
[...]
 Python 2.6 support was dropped in 1.1.0, so we need to use markers
 to select 1.0.1 for the remaining 2.6 gate jobs. (We should kill
 those of asap).
[...]

Unless we convince ourselves it's worthwhile to EOL Juno early, we
can't drop 2.6 jobs until October.
-- 
Jeremy Stanley

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [nova] Switching the bug status of ~200 bugs at once; Problem?

On 07/10/2015 09:34 AM, Markus Zoeller wrote:
 Sean Dague s...@dague.net wrote on 07/10/2015 12:32:00 PM:

 From: Sean Dague s...@dague.net
 To: openstack-dev@lists.openstack.org
 Date: 07/10/2015 12:32 PM
 Subject: Re: [openstack-dev] [nova] Switching the bug status of ~200 
 bugs at once; Problem?

 On 07/10/2015 04:09 AM, Markus Zoeller wrote:
 I'd like to switch the status of ~200 bugs at once to adjust the
 bookkeeping in LaunchPad. This will concern Confirmed and Triaged
 bugs which have an assignee [1]. AFAIK they should be In Progress.

 I'm concerned if this would affect some reports I'm not aware of in
 a bad way. So, if you think this is a bad thing, please speak up.

 FWIW, I'll use an enhanced version of [2] which I will provide in the
 next days. A few days later I'll have a version ready which can
 update stale In Progress bugs by querying Gerrit.

 [1] http://bit.ly/1GbhJWu
 [2] https://review.openstack.org/#/c/197060/

 I would not do that. Bugs with assignee tend to get ignored by other
 people looking at the problem, and in *most* cases the assignee is not
 actually working on the bug. I would instead go the other direction and
 pop any In Progress bug that has seen no activity in the last 60 days
 back to Confirmed.

-Sean

 The bugs I'm referring to are *not* set to In Progress but have an
 assignee. I think this is an inconsistency which I try to resolve. 
 I can treat them like In Progress bugs and remove the assignee
 if there is no activity in the last 60 days. Is this what you say?

Right, that's what I'm saying. Removing the assignee is typically the
resolution to these when I've processed them manually, because they are
old, and no one is working on it.

-Sean

-- 
Sean Dague
http://dague.net

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [glance] Why does Glance keep the deleted membership of image ?

2015-07-10 Thread Nikhil Komawar

Please find the response inline.

 Hi Glance experts,

 I'd like to send this mail again, hope I can get help and suggest from
 glance experts. The question is from a bug
 https://bugs.launchpad.net/glance/+bug/1462315,

 If an image-member is deleted, then create it again with the same
 parameters, glance searches db to see if there is already an existing
 one, but the result doesn't include the record which was marked as
 deleted,
 glance will try to create a new one with the same parameters, it works
 well on mysql. But it is failed on DB2 with SQL0803N error.

 The root cause is that DB2 constraint is more restricted than mysql.
 For db2, the columns under unique constrains should be NOT NULL,
 currently the column deleted_at which is one of unique constrain of
 image_members
 is nullable. A possible solution is to alter it to not null in
 migration. that means we have to insert a default timestamp value for
 the new created image-member, an active member with a no-blank
 timestamp for deleted_at seems very confusing.  


Agree that this is confusing. And changing the behavior this way is NOT
a good idea. A record that's never been deleted should not have
deleted(_at) value. It will affect notifications and conflict with API
guidelines.

 Another fix is: we may check all existing image-member records
 including the deleted image-member before create image-member, then
 update it if it exists, otherwise create a new one, that is proposed
 in https://review.openstack.org/#/c/190895/


 I'm wondering why can't we use only one record to maintain the
 member-ship between a pair of image and tenant. Maybe there is some
 other consideration, can you help give me some suggestion ? I'd like
 to know more. Thanks.



Concept wise this sounds like a good idea but it could have performance
degradation impact. Nonetheless, image-members have an idx that should
be a relief for that query image_member_find that you added in your
proposal. My hope is that the rally gate job will tell us more if there
is a performance problem.

 Best regards,
 LongQuan



 __
 OpenStack Development Mailing List (not for usage questions)
 Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

-- 

Thanks,
Nikhil

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Murano] Documentation on how to Start Contributing

2015-07-10 Thread Georgy Okrokvertskhov

Hi,

If I understand correctly, you want to be able to install modified Murano
from your own repository. There is a devstack integration script in Murano
repository which does this. Here are lines where you can point to specific
repository for Murano installation in devstack:
https://github.com/openstack/murano/blob/master/devstack/plugin.sh#L17-L18

Installation procedure in the README.rst file in the folder devstack of
murano repository.

Thanks
Gosha

On Thu, Jul 9, 2015 at 9:54 PM, Nikolay Starodubtsev 
nstarodubt...@mirantis.com wrote:

 Hi,
 Can you describe what problems do you have with bring code changes into
 a live Devstack environment, and test them.
 If you want a real-time QA experience you can ask your questions at
 #murano on freenode.



 Nikolay Starodubtsev

 Software Engineer

 Mirantis Inc.


 Skype: dark_harlequine1

 2015-07-10 2:32 GMT+03:00 Vahid S Hashemian vahidhashem...@us.ibm.com:

 Hello,

 I am wondering if there is any documentation for new contributors that
 explains how to get up-to-speed with Murano development; something that
 covers how to bring code changes into a live Devstack environment, and test
 them.
 I've looked at the Murano documentation online and have not been able to
 find it.

 Any pointer is very much appreciated.

 Thanks.

 -
   *Vahid
 Hashemian, Ph.D.*
 Advisory Software Engineer, IBM Cloud Labs


 __
 OpenStack Development Mailing List (not for usage questions)
 Unsubscribe:
 openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



 __
 OpenStack Development Mailing List (not for usage questions)
 Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev




-- 
Georgy Okrokvertskhov
Architect,
OpenStack Platform Products,
Mirantis
http://www.mirantis.com
Tel. +1 650 963 9828
Mob. +1 650 996 3284
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Cinder] python-cinderclient functional tests

2015-07-10 Thread Ivan Kolodyazhny

On Fri, Jul 10, 2015 at 4:28 PM, Ivan Kolodyazhny e...@e0ne.info wrote:

 Review request to make this voting


I'm sorry for typo. Of course, it's a review request to make this job
non-voting in check queue.

Regards,
Ivan Kolodyazhny
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Fuel][Fuel-library] Using librarian-puppet to manage upstream fuel-library modules

2015-07-10 Thread Alex Schultz

Done. Sorry about that.

-Alex

On Fri, Jul 10, 2015 at 9:22 AM, Simon Pasquier spasqu...@mirantis.com
wrote:

Alex, could you enable the comments for all on your document?
Thanks!
Simon

On Thu, Jul 9, 2015 at 11:07 AM, Bogdan Dobrelya bdobre...@mirantis.com
wrote:

Hello everyone,

I took some time this morning to write out a document[0] that outlines
one possible ways for us to manage our upstream modules in a more
consistent fashion. I know we've had a few emails bouncing around
lately around this topic of our use of upstream modules and how can we
improve this. I thought I would throw out my idea of leveraging
librarian-puppet to manage the upstream modules within our
fuel-library repository. Ideally, all upstream modules should come
from upstream sources and be removed from the fuel-library itself.
Unfortunately because of the way our repository sits today, this is a
very large undertaking and we do not currently have a way to manage
the inclusion of the modules in an automated way. I believe this is
where librarian-puppet can come in handy and provide a way to manage
the modules. Please take a look at my document[0] and let me know if
there are any questions.

Thanks,
-Alex

[0]
https://docs.google.com/document/d/13aK1QOujp2leuHmbGMwNeZIRDr1bFgJi88nxE642xLA/edit?usp=sharing

The document is great, Alex!
I'm fully support the idea to start adapting fuel-library by
the suggested scheme. The monitoring feature of ibrarian looks not
intrusive and we have no blockers to start using the librarian just
immediately.

--
Best regards,
Bogdan Dobrelya,
Irc #bogdando

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe:
openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Fuel] wrong network for keystone endpoint in 6.1 ?

2015-07-10 Thread Daniel Comnea

Okay Vladimir, thanks for confirmation!

So then you happy to stick my sketch proposal (of course needs re-wording)
into documentation?

Dani

On Fri, Jul 10, 2015 at 11:31 AM, Vladimir Kuklin vkuk...@mirantis.com
wrote:

 Daniel

 Yes, if you want to do some administrative stuff you need to have access
 to management network to be able to work with internal and admin endpoints.

 On Fri, Jul 10, 2015 at 9:58 AM, Daniel Comnea comnea.d...@gmail.com
 wrote:

 I know about the flow but what i'm questioning is:

 admin endpoint is mapped to br-mgmt subnet (you do have the HAproxy as
 below defined in 6.1. In 6.0 and before you had no HAproxy)

 listen keystone-2
   bind 192.168.20.3:35357
   option  httpchk
   option  httplog
   option  httpclose
   server node-17 192.168.20.20:35357   check inter 10s fastinter 2s
 downinter 3s rise 3 fall 3
   server node-18 192.168.20.21:35357   check inter 10s fastinter 2s
 downinter 3s rise 3 fall 3
   server node-23 192.168.20.26:35357   check inter 10s fastinter 2s
 downinter 3s rise 3 fall 3

 public endpoint is mapped to br-ex

 So with this behavior you are saying the bt-mgmt subnet (which i thought
 is only for controller  compute traffic, isolated network) should be
 routable in the same way br-ex is?

 Dani


 On Thu, Jul 9, 2015 at 11:30 PM, Stanislaw Bogatkin 
 sbogat...@mirantis.com wrote:

 Hi Daniel,

 answer is no - actually there is no strong dependency between public and
 internal/admin endpoints. In your case keystone client ask keystone on
 address 10.52.71.39 (which, I think, was provided by system
 variable OS_AUTH_URL), auth on it and then keystone give endpoints list to
 client. Client selected admin endpoint from this list (192.168.20.3
 address) and tried to get information you asked. It's a normal behavior.

 So, in Fuel by default we have 3 different endpoints for keystone -
 public on public VIP, port 5000; internal on management VIP, port 5000,
 admin on management VIP, port 35357.

 On Thu, Jul 9, 2015 at 4:59 PM, Daniel Comnea comnea.d...@gmail.com
 wrote:

 Hi,

 I'm running Fuel 6.1 and i've seen an interesting behavior which i
 think match bug [1]

 Basically the adminUrl  publicUrl part of keystone endpoint are
 different

 And the result of that is that you can't run keystone cli - i.e
 create/list tenants etc

 keystone --debug tenant-list
 /usr/local/lib/python2.7/site-packages/keystoneclient/shell.py:65:
 DeprecationWarning: The keystone CLI is deprecated in favor of python-
 openstackclient. For a Python library, continue using python-keys
 toneclient.
   'python-keystoneclient.', DeprecationWarning)
 DEBUG:keystoneclient.auth.identity.v2:Making authentication request to
 http://10.20.71.39:5000/v2.0/tokens
 INFO:requests.packages.urllib3.connectionpool:Starting new HTTP
 connection (1): 10.52.71.39
 DEBUG:requests.packages.urllib3.connectionpool:POST /v2.0/tokens
 HTTP/1.1 200 3709
 DEBUG:keystoneclient.session:REQ: curl -g -i -X GET
 http://192.168.20.3:35357/v2.0/tenants -H User-Agent: python-
 keystoneclient -H Accept: application/json -H X-Auth-Token:
 {SHA1}cc918b89c2dca563edda43e01964b1f1979c552b

 shouldn't adminURL = publicURL = br-ex for keystone?


 Dani


 [1] https://bugs.launchpad.net/fuel/+bug/1441855


 __
 OpenStack Development Mailing List (not for usage questions)
 Unsubscribe:
 openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev




 __
 OpenStack Development Mailing List (not for usage questions)
 Unsubscribe:
 openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



 __
 OpenStack Development Mailing List (not for usage questions)
 Unsubscribe:
 openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev




 --
 Yours Faithfully,
 Vladimir Kuklin,
 Fuel Library Tech Lead,
 Mirantis, Inc.
 +7 (495) 640-49-04
 +7 (926) 702-39-68
 Skype kuklinvv
 35bk3, Vorontsovskaya Str.
 Moscow, Russia,
 www.mirantis.com http://www.mirantis.ru/
 www.mirantis.ru
 vkuk...@mirantis.com

 __
 OpenStack Development Mailing List (not for usage questions)
 Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [nova] Switching the bug status of ~200 bugs at once; Problem?

Markus Zoeller wrote:
 The bugs I'm referring to are *not* set to In Progress but have an
 assignee. I think this is an inconsistency which I try to resolve. 
 I can treat them like In Progress bugs and remove the assignee
 if there is no activity in the last 60 days. Is this what you say?

So you have a bunch of bugs that are Confirmed (or Triaged) + an
assignee set. I would argue that you need to separate two cases:

the bug had no activity for the last 60 days: assignee should be removed

the bug had activity in the last 60 days: status should be in progress

otherwise you further hide the fact that the bug is abandoned by setting
status in addition to the assignee.

Cheers,

-- 
Thierry Carrez (ttx)

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] Suggestion - github - Puppet-openstack repository.

2015-07-10 Thread Emilien Macchi



On 07/10/2015 09:05 AM, cool dharma06 wrote:
 Hi all,
 
 i trying to install openstack with puppet. i saw two repositories are
 active related to puppet-openstack. i need some advice to go on with
 which repository.
 
 1. https://github.com/stackforge/puppet-openstack-cloud

I personally know this one, I'm one of the main contributors :-)
If you need more context at why we did this module, please read
http://spinalstack.enovance.com

You also need to know this module only supports OpenStack Juno and is on
its end of life.

 
 2. https://github.com/puppetlabs/puppetlabs-openstack
 
 i just confused which one to go.?
 
 give some suggestion regarding this. i am newbie for this things.
 
 regards,
 cooldharma06
 
 
 __
 OpenStack Development Mailing List (not for usage questions)
 Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
 

-- 
Emilien Macchi



signature.asc
Description: OpenPGP digital signature
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [nova] Switching the bug status of ~200 bugs at once; Problem?

2015-07-10 Thread Markus Zoeller

Sean Dague s...@dague.net wrote on 07/10/2015 12:32:00 PM:

 From: Sean Dague s...@dague.net
 To: openstack-dev@lists.openstack.org
 Date: 07/10/2015 12:32 PM
 Subject: Re: [openstack-dev] [nova] Switching the bug status of ~200 
 bugs at once; Problem?

 On 07/10/2015 04:09 AM, Markus Zoeller wrote:
  I'd like to switch the status of ~200 bugs at once to adjust the
  bookkeeping in LaunchPad. This will concern Confirmed and Triaged
  bugs which have an assignee [1]. AFAIK they should be In Progress.

  I'm concerned if this would affect some reports I'm not aware of in
  a bad way. So, if you think this is a bad thing, please speak up.

  FWIW, I'll use an enhanced version of [2] which I will provide in the
  next days. A few days later I'll have a version ready which can
  update stale In Progress bugs by querying Gerrit.

  [1] http://bit.ly/1GbhJWu
  [2] https://review.openstack.org/#/c/197060/

 I would not do that. Bugs with assignee tend to get ignored by other
 people looking at the problem, and in *most* cases the assignee is not
 actually working on the bug. I would instead go the other direction and
 pop any In Progress bug that has seen no activity in the last 60 days
 back to Confirmed.

-Sean

The bugs I'm referring to are *not* set to In Progress but have an
assignee. I think this is an inconsistency which I try to resolve. 
I can treat them like In Progress bugs and remove the assignee
if there is no activity in the last 60 days. Is this what you say?

Regards,
Markus Zoeller (markus_z)

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [Cinder]Cinder services issues after add/remove/add driver

2015-07-10 Thread Eduard Matei

Hi,

Potential bug, but still investigating:
- configured a driver on cinder (besides lvm) - all ok
[service1]
volume_driver = cinder.volume.drivers.openvstorage.OVSVolumeDriver
volume_backend_name = service1
vpool_name = service1
- removed it, restarted services - all ok (cinder service-list shows the
service as enabled and down)
- configured ANOTHER driver on cinder (similar to first) - new volumes
stuck in status creating

Tried cinder service-disable, service showed disabled and up (?) then
later disabled and down.
Tried changing in mysql (deleted = 1, then actually deleted).

Cinder-scheduler logs:
- Choosing NODE1@service2#service2
- Flow 'volume_create_scheduler' (737556d0-a096-4639-899a-4bfd1c5bf166)
transitioned into state 'SUCCESS' from state 'RUNNING'
but the volume remains in status creating, nothing is logged in c-vol on
any node

Details:
- 3 nodes setup, Devstack 2014.2.2, Ubuntu 14.04
- c-api, c-sch and c-vol running on all 3 nodes

Any help appreciated.

Thanks,

Eduard
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Fuel][Fuel-library] Using librarian-puppet to manage upstream fuel-library modules

2015-07-10 Thread Simon Pasquier

Alex, could you enable the comments for all on your document?
Thanks!
Simon

On Thu, Jul 9, 2015 at 11:07 AM, Bogdan Dobrelya bdobre...@mirantis.com
wrote:

  Hello everyone,
 
  I took some time this morning to write out a document[0] that outlines
  one possible ways for us to manage our upstream modules in a more
  consistent fashion. I know we've had a few emails bouncing around
  lately around this topic of our use of upstream modules and how can we
  improve this. I thought I would throw out my idea of leveraging
  librarian-puppet to manage the upstream modules within our
  fuel-library repository. Ideally, all upstream modules should come
  from upstream sources and be removed from the fuel-library itself.
  Unfortunately because of the way our repository sits today, this is a
  very large undertaking and we do not currently have a way to manage
  the inclusion of the modules in an automated way. I believe this is
  where librarian-puppet can come in handy and provide a way to manage
  the modules. Please take a look at my document[0] and let me know if
  there are any questions.
 
  Thanks,
  -Alex
 
  [0]
 https://docs.google.com/document/d/13aK1QOujp2leuHmbGMwNeZIRDr1bFgJi88nxE642xLA/edit?usp=sharing

 The document is great, Alex!
 I'm fully support the idea to start adapting fuel-library by
 the suggested scheme. The monitoring feature of ibrarian looks not
 intrusive and we have no blockers to start using the librarian just
 immediately.

 --
 Best regards,
 Bogdan Dobrelya,
 Irc #bogdando

 __
 OpenStack Development Mailing List (not for usage questions)
 Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Nova] Friday July 10 is Nova's non-priority feature review bash day

2015-07-10 Thread Matt Riedemann




On 7/7/2015 12:35 PM, John Garbutt wrote:

Hi,

Friday is: non-priority feature review bash day
https://wiki.openstack.org/wiki/Nova/Liberty_Release_Schedule#Dates_overview

The idea is the whole nova community is invited to concentrate on
reviewing some some of the low priority blueprints that have been
hanging around for a while.

*Note*
the whole nova community are invited, its not just core reviewers, as
with all review efforts

The suggested starting point are low priority blueprints marked as
NeedsCodeReview here:
https://blueprints.launchpad.net/nova/liberty

To help focus our efforts I am trying to curate a list of blueprints
that have been waiting a long time for a review, see the Low
priority, and longest waiting Blueprint patches section of the usual
etherpad as a starting point:
https://etherpad.openstack.org/p/liberty-nova-priorities-tracking

Please lets group together and really get some of these long standing
blueprints merged before next weeks non-priority feature proposal
freeze, and the upcoming non-priority feature freeze.

As usual, any questions or ideas, please do get in touch.

Thanks,
johnthetubaguy

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



This is a good idea and thanks for pulling together the list in the 
etherpad.


However, Friday is probably not the best for this (as I'm writing this 
at 1pm on the day of the bash).  We are having gate issues today 
draining some resources, plus quite a few people aren't really around, 
and I had a big lunch and it's quiet and I feel like sleeping. :)


So maybe we should reschedule this for like, Tuesday 7/14 when people 
are actually around?


--

Thanks,

Matt Riedemann


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [neutron][lbaas] Horizon support for neutron-lbaas v2

2015-07-10 Thread Eichberger, German

Hi Vivek,

Hope things are well. With the Midccyle next week I am wondering if you made 
any progress and/or how we can best help with the panels.

Thanks,
German

From: Jain, Vivek vivekj...@ebay.commailto:vivekj...@ebay.com
Reply-To: OpenStack Development Mailing List (not for usage questions) 
openstack-dev@lists.openstack.orgmailto:openstack-dev@lists.openstack.org
Date: Wednesday, April 8, 2015 at 3:32 PM
To: OpenStack Development Mailing List (not for usage questions) 
openstack-dev@lists.openstack.orgmailto:openstack-dev@lists.openstack.org
Cc: Balle Balle, Susanne susanne.ba...@hp.commailto:susanne.ba...@hp.com, 
Tonse, Milan mto...@ebay.commailto:mto...@ebay.com
Subject: Re: [openstack-dev] [neutron][lbaas] Horizon support for neutron-lbaas 
v2

Thanks German for the etherpad link. If you have any documentation for flows, 
please share those too.

I will work with my team at ebay to publish wireframes for design we are 
working on. It will be mostly along the lines I demo’ed in Paris.

Thanks,
Vivek

From: Eichberger, German 
german.eichber...@hp.commailto:german.eichber...@hp.com
Reply-To: OpenStack Development Mailing List (not for usage questions) 
openstack-dev@lists.openstack.orgmailto:openstack-dev@lists.openstack.org
Date: Wednesday, April 8, 2015 at 11:24 AM
To: OpenStack Development Mailing List (not for usage questions) 
openstack-dev@lists.openstack.orgmailto:openstack-dev@lists.openstack.org
Cc: Balle, Susanne susanne.ba...@hp.commailto:susanne.ba...@hp.com
Subject: Re: [openstack-dev] [neutron][lbaas] Horizon support for neutron-lbaas 
v2

Hi,

We briefly talked about it a few Neutron meetings back (LBaaS is now on demand) 
and created an etherpad to track things: 
https://etherpad.openstack.org/p/LBaaS_Horizon_Use_Cases

Susanne and I met with HP’s UX designer to work on the design for some flows 
for the Horizon panel (cc’d) but I am glad that Vivek is taking the lead. 
Please check that etherpad for more information and feel free to update as 
things happen…

Thanks,
German


From: Jain, Vivek [mailto:vivekj...@ebay.com]
Sent: Tuesday, April 07, 2015 9:01 AM
To: OpenStack Development Mailing List (not for usage questions)
Subject: Re: [openstack-dev] [neutron][lbaas] Horizon support for neutron-lbaas 
v2

Hi Evgeny,
We have just started working on Horizon lbaasv2 support. I have to sync up with 
my team on the time-line but it is not targeted for Kilo release.
Since it is a major effort, we will need more hands. Let me know if anyone is 
interested to contribute.

On a related note, Do we have a sample code which uses neutronclient lbaas v2 
methods? That will greatly speedup our horizon integration.

Thanks,
Vivek

From: Phillip Toohill 
phillip.tooh...@rackspace.commailto:phillip.tooh...@rackspace.com
Reply-To: OpenStack Development Mailing List (not for usage questions) 
openstack-dev@lists.openstack.orgmailto:openstack-dev@lists.openstack.org
Date: Tuesday, April 7, 2015 at 7:50 AM
To: OpenStack Development Mailing List (not for usage questions) 
openstack-dev@lists.openstack.orgmailto:openstack-dev@lists.openstack.org
Subject: Re: [openstack-dev] [neutron][lbaas] Horizon support for neutron-lbaas 
v2


Hey Evgeny,



I believe Vivek is working on Horizon lbaasv2 support. We werent given a 
timeline or anything but sounds like its being actively worked on. I would 
reach out to him to get better timelines if concerned.


Phillip V. Toohill III
Software Developer
[http://600a2794aa4ab5bae6bd-8d3014ab8e4d12d3346853d589a26319.r53.cf1.rackcdn.com/signatures/images/rackspace_logo.png]
phone: 210-312-4366
mobile: 210-440-8374


From: Evgeny Fedoruk evge...@radware.commailto:evge...@radware.com
Sent: Tuesday, April 7, 2015 5:50 AM
To: OpenStack Development Mailing List (not for usage questions)
Subject: [openstack-dev] [neutron][lbaas] Horizon support for neutron-lbaas v2

Hi guys,

LBaaS v2 has no horizon support as for now and I want to know if this work is 
planned to be done and , if yes, in what time frame.
Is there a plan to develop it for Kilo or for L release?

Thanks,
Evg
__ 
OpenStack Development Mailing List (not for usage questions) Unsubscribe: 
openstack-dev-requ...@lists.openstack.orgmailto:openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [neutron][qos] Priorities Status for QoS

2015-07-10 Thread Miguel Angel Ajo



I've been working on assembling a QoS[1] POC since last day of the 
coding sprint in Israel [2],

Ihar has reported to the list our plan to get into master [3].

I've been able to validate and integrate lots of the patches, and find 
the gaps, while still
finishing the top-down assembly may require a few more hours, or an 
extra day, I believe
we should prioritize the work to make such POC available for easy 
consumption in

feature/qos.

For that to happen, we should prioritize review and merge of the 
following patches into the branch:
(I'd be very thankful to any review cycles anyone could spend on this, 
specially cores, of course)


QoS service plugin:

* https://review.openstack.org/#/c/197564/
* https://review.openstack.org/#/c/197898/

Agent side:

* https://review.openstack.org/#/c/195440/ (I just found a bug in the 
logic, working to submit a new PS)
* https://review.openstack.org/#/c/197557/ (waiting for merge, dependent 
on other patch)


DB Models  Versioned Objects (unit tests+ fixes):

* https://review.openstack.org/#/c/25/
* https://review.openstack.org/#/c/200036/
* https://review.openstack.org/#/c/200418/
--
ready for merge, waiting on rebase from master, failing on the mock-hell:
* https://review.openstack.org/#/c/198163/
* https://review.openstack.org/#/c/197898/
* https://review.openstack.org/#/c/199627/
* https://review.openstack.org/#/c/199634/


Extra stones:
a)

@armax, please check where I make sense here, I believe I do, but of 
course I want your opinion:

We also need to introduce new BEFORE_UPDATE callbacks for ports and networks
before any call to plugin. So we'll be able to retrieve qos_profile_id, 
and check it, and

associate/dissociate network/port to profile.

AFTER_UPDATE is working ok here, with a few workarounds, since, for example,
it's called from ML2, and ML2, will only push information of the port 
when the port

has changed anything relevant to ML2, and won't even provide the port_id ...
I'm not sure where this is used/what's the use case.


b) rule list handling in the policy versioned object (Ihar is handling 
that here, WIP): https://review.openstack.org/#/c/200608/
c) serializing and deserializing the policy - rules dictionary over the 
wire (thanks to versioned objects).



Afterwards, we must follow with the plan Ihar explained in [3], we don't 
have much time,
so, if you have an interest on QoS, please join the effort and help us 
with anything you can

if you want it as an experimental feature available by L.

Quality Regards ;)
Miguel Ángel

[1] 
http://specs.openstack.org/openstack/neutron-specs/specs/liberty/qos-api-extension.html
[2] TL;DR report, with nice pictures : 
http://www.ajo.es/post/123458887419/neutron-quality-of-service-coding-sprint

[3] http://lists.openstack.org/pipermail/openstack-dev/2015-July/069188.html


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Fuel][Fuel-Packaging][Fuel-CI][Fuel-Infra] How to Deal with CI Failures Induced by Libraries Requirements Drift

2015-07-10 Thread Dmitry Borodaenko

Vladimir,

You have identified the root cause of the problem:

 The key issue here is that our CI tests are using some code not from
packages of particular releases, but from other sources, e.g. pypi for
python.

Using frozen pypi, rubygems, nodejs and other language specific package
mirrors is a workaround that doesn't address the root cause. Making that
workaround even more complex by implementing a system to manage, freeze,
and unfreeze these mirrors is a waste of effort since it still does nothing
to address the root cause.

Addressing the root cause means modifying the tests to use the same package
repositories that are used in production, and for which we already have all
the necessary tools.




On Fri, Jul 10, 2015 at 9:44 AM Vladimir Kuklin vkuk...@mirantis.com
wrote:

 Folks

 I am writing this email to discuss very important topic which appeared to
 be rather hot with several bugs associated with it.

 For example these ones:
 https://bugs.launchpad.net/fuel/+bug/1472018
 https://bugs.launchpad.net/fuel/+bug/1468053

 There were several fixes applied. One of them was merged (unfortunately)
 and one was stopped by me and other reviewers of stable branches
 (fortunately),

 I want to  describe why these bugs happened, why such fixes should not be
 applied and which solution I envision the best for such cases.

 The key issue here is that our CI tests are using some code not from
 packages of particular releases, but from other sources, e.g. pypi for
 python.
 Libraries being installed are determined by using particular
 requirements.txt/test-requirements.txt files. These files usually do not
 contain upper bound for libraries versions, thus the newest versions are
 getting installed. This leads to the issues which are especially no good
 for already released code:

 1) if there is a new conflicting version, you need to set this
 upper-bound, thus you need to modify bits which get released
 2) you are actually testing your code by linking it with libraries which
 are different from those that users are really using when running your code
 3) even if you specify an upper bound (or even fix the version) for this
 particular library, you may still fetch its newer dependency implicitly (by
 traversing indirect dependencies) with which you will be linking your
 libraries and which will actually be different from the code that you (and
 your users) use
 4) you may also break production installation if you fix some library
 version as it may not exist in the code bundle which gets delivered to your
 users as a set of packages

 All this means that you will get false-negative (you are lucky in this
 case) -1's from CI or false-positive (oops, it can ruin the production
 installation) +1's from CI.

 So far I suggest that we do the following (which, strictly speaking, seems
 to be the only reasonable one): in order to do good testing of components,
 we need to do complete freeze of all other libraries and components code we
 are linking/running against. Especially, this should happen for already
 released code which gets into state of being maintained after that.

 This means that we need to not only freeze our repositories, but use a
 frozen copy of rubygems/pypi/nodejs repositories for all types of tests we
 run. This will require certain amount of effort from our Infra teams to
 build up a mechanism of freezing/unfreezing the mirrors, determining how we
 should add new pieces of code in such mirrors, may be, add some additional
 code-review policies, create real mirroring scripts and so on.

 But I think we should start doing this now in order to finish it at least
 before 8.0 timeframe.

 As a temporary solution, I suggest that we apply a series of changes into
 these requirements.txt files before running actual tests instead of
 modifying product code.

 Please provide your comments and thoughts.


 --
 Yours Faithfully,
 Vladimir Kuklin,
 Fuel Library Tech Lead,
 Mirantis, Inc.
 +7 (495) 640-49-04
 +7 (926) 702-39-68
 Skype kuklinvv
 35bk3, Vorontsovskaya Str.
 Moscow, Russia,
 www.mirantis.com http://www.mirantis.ru/
 www.mirantis.ru
 vkuk...@mirantis.com
  __
 OpenStack Development Mailing List (not for usage questions)
 Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [nova] Switching the bug status of ~200 bugs at once; Problem?

2015-07-10 Thread Markus Zoeller

Thierry Carrez thie...@openstack.org wrote on 07/10/2015 03:43:06 PM:

 From: Thierry Carrez thie...@openstack.org
 To: openstack-dev@lists.openstack.org
 Date: 07/10/2015 03:43 PM
 Subject: Re: [openstack-dev] [nova] Switching the bug status of ~200 
 bugs at once; Problem?

 Markus Zoeller wrote:
  The bugs I'm referring to are *not* set to In Progress but have an
  assignee. I think this is an inconsistency which I try to resolve. 
  I can treat them like In Progress bugs and remove the assignee
  if there is no activity in the last 60 days. Is this what you say?

 So you have a bunch of bugs that are Confirmed (or Triaged) + an
 assignee set. I would argue that you need to separate two cases:

 the bug had no activity for the last 60 days: assignee should be removed

 the bug had activity in the last 60 days: status should be in progress

 otherwise you further hide the fact that the bug is abandoned by setting
 status in addition to the assignee.

 Cheers,

 -- 
 Thierry Carrez (ttx)

Right. My first intention was to switch all to in progress and then
switch a subset of that back to the previous status, depending on their
activity. I see now that this would cause confusion. 
I'll process them seperately like Thierry suggested to avoid that
confusion. I'll announce the expected changes in the next days before
I actually execute them.

Regards,
Markus Zoeller (markus_z)

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all][tests] Fix it friday! [mock failure in CI]

2015-07-10 Thread Victor Stinner


Le 10/07/2015 13:39, Jeremy Stanley a écrit :

On 2015-07-10 13:21:56 +0200 (+0200), Victor Stinner wrote:

I see mock===1.0.1 in upper-constraints.txt, but the python27
check job of Swift was broken by the release of mock 1.1.

Can someone please explain me why Swift check job failed?


As far as I'm aware, only DevStack is making use of the constraints
file currently. Unit tests are still relying on pip to install
working versions of packages from the requirements.txt and
test-requirements.txt within each repo.


Is there a plan to use pinned versions on other gates to avoid similar 
issues in the future? (Decide when we upgrade a dependency)


Victor

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Fuel] Fuel library CI changes due to transition to Kilo

2015-07-10 Thread Aleksandra Fedorova

Hi everyone,

latest Fuel Library CI update:

* Currently we have two tests based on Kilo ISO enabled in voting mode:

 - master.fuel-library.pkgs.ubuntu.ha_neutron_vlan
 - fuellib_review_pkgs_master_node

Both run Ubuntu deployment and pass on current master (see [1] and [2]).

Note that you need to rebase all your patches on top of the fix [3]

* The nova-based Ubuntu test is blocked by bug [4]

* CentOS-based test is disabled



@Tatyana,

let's update the bug

  https://bugs.launchpad.net/fuel/+bug/1423511

bvt_2 takes about twice more time then tests we run at the moment,
thus maybe we can find a bit lighter test case with Ceph?



[1] 
https://ci.fuel-infra.org/job/master.fuel-library.pkgs.ubuntu.ha_neutron_vlan/1978/
[2] https://ci.fuel-infra.org/job/fuellib_review_pkgs_master_node/1891/
[3] https://review.openstack.org/200143
[4] https://bugs.launchpad.net/fuel/+bug/1472529

On Wed, Jul 8, 2015 at 5:47 PM, Aleksandra Fedorova
afedor...@mirantis.com wrote:
 Hi, everyone, let's discuss a bit CI changes required for fuel-library
 repository as we merge Kilo support and discard support for CentOS
 6.5.

 Current situation
 

 We run 6 deployment tests:

 Three jobs running against Juno-based ISO:

   master.fuel-library.pkgs.centos.ha_nova_vlan
   master.fuel-library.pkgs.ubuntu.ha_neutron_vlan
   master.fuellib_review_pkgs_master_node

 Three jobs in non-voting mode running against Kilo ISO:

   kilo.fuel-library.pkgs.centos.ha_nova_vlan
   kilo.fuel-library.pkgs.ubuntu.ha_neutron_vlan
   kilo.fuellib_review_pkgs_master_node

 Today we merged kilo support to master branch, so Juno deployment and
 CentOS deployment won't pass anymore.

 Proposal
 -

 Let's set three voting jobs:

  - master.fuel-library.pkgs.UBUNTU.ha_nova_vlan
  - master.fuel-library.pkgs.ubuntu.ha_neutron_vlan
  - master.fuellib_review_pkgs_master_node

 and one non-voting:

  master.fuel-library.pkgs.centos.ha_NEUTRON_vlan

 All of them based on latest ISO with Kilo support.

 Any ideas, objections?

 --
 Aleksandra Fedorova
 Fuel CI Engineer
 bookwar



-- 
Aleksandra Fedorova
Fuel CI Engineer
bookwar

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [neutron][lbaas] Radware unit test issues

2015-07-10 Thread Brandon Logan

python's mock library had an update yesterday that exposed some issues with 
unit tests that are using the mock assert calls.  The radware tests were using 
a method called assert_called_once, which actually is not a real assert method 
off a mock.  assert_called_once_with is, though.  However, before the update 
mock would allow this method to run as it allowed any method calls to be run.  
Now with the update, only actual assert methods can be used, so that broke 
these tests.  Changing the method to something like self.assertEquals(1, 
mocked_object.call_count) failed as well because the call_count was not 
actually 1.  As such, I've pushed up a review to skip these tests.  It would be 
great if radware folks could fix these tests and unskip them as I didn't have 
the time to look into actually figuring out if the call count discrepancy is a 
real issue or not.


https://review.openstack.org/#/c/200616/?


Thanks,
Brandon
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [murano] [congress] Congress needs to fetch environments from all tenants.

2015-07-10 Thread Tim Hinrichs

We sometimes want the ability to write policy across tenants, e.g. VMs from
Coke and Pepsi must always be deployed on different hosts.

I didn't think there were any roles that could see everything without
all_tenants=true.  If there are such roles, I'd be happy to remove the
all_tenants=true from the datasource drivers.

Tim


On Fri, Jul 10, 2015 at 8:00 AM Dolph Mathews dolph.math...@gmail.com
wrote:

 How about using domain-based role assignments in keystone and requiring
 domain-level authorization in policy, and then only returning data about
 the collection of tenants that belong to the authorized domain? That way
 you don't have an API that violates multi-tenant isolation, consumable only
 by cloud operators.

 On Wed, Jul 8, 2015 at 6:27 AM, Filip Blaha filip.bl...@hp.com wrote:

 Hi all,

 I started implement bp [1]. Problem is that congress needs data about
 environments from all tenants but murano API lists only environments of
 user's current tenant. We decided to ipmplement it similarly like listing
 servers in nova where is query parameter all_tenants=true for that (user
 must be admin) I have 2 questions about that:

 1) Are there any security concerns about this approach?
 2) Has someone better idea how to implement this?

 [1]
 https://blueprints.launchpad.net/murano/+spec/murano-api-all-tenants-search

 Regards
 Filip



 __
 OpenStack Development Mailing List (not for usage questions)
 Unsubscribe:
 openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


 __
 OpenStack Development Mailing List (not for usage questions)
 Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [Fuel][Fuel-Packaging][Fuel-CI][Fuel-Infra] How to Deal with CI Failures Induced by Libraries Requirements Drift

2015-07-10 Thread Vladimir Kuklin

Folks

I am writing this email to discuss very important topic which appeared to
be rather hot with several bugs associated with it.

For example these ones:
https://bugs.launchpad.net/fuel/+bug/1472018
https://bugs.launchpad.net/fuel/+bug/1468053

There were several fixes applied. One of them was merged (unfortunately)
and one was stopped by me and other reviewers of stable branches
(fortunately),

I want to  describe why these bugs happened, why such fixes should not be
applied and which solution I envision the best for such cases.

The key issue here is that our CI tests are using some code not from
packages of particular releases, but from other sources, e.g. pypi for
python.
Libraries being installed are determined by using particular
requirements.txt/test-requirements.txt files. These files usually do not
contain upper bound for libraries versions, thus the newest versions are
getting installed. This leads to the issues which are especially no good
for already released code:

1) if there is a new conflicting version, you need to set this upper-bound,
thus you need to modify bits which get released
2) you are actually testing your code by linking it with libraries which
are different from those that users are really using when running your code
3) even if you specify an upper bound (or even fix the version) for this
particular library, you may still fetch its newer dependency implicitly (by
traversing indirect dependencies) with which you will be linking your
libraries and which will actually be different from the code that you (and
your users) use
4) you may also break production installation if you fix some library
version as it may not exist in the code bundle which gets delivered to your
users as a set of packages

All this means that you will get false-negative (you are lucky in this
case) -1's from CI or false-positive (oops, it can ruin the production
installation) +1's from CI.

So far I suggest that we do the following (which, strictly speaking, seems
to be the only reasonable one): in order to do good testing of components,
we need to do complete freeze of all other libraries and components code we
are linking/running against. Especially, this should happen for already
released code which gets into state of being maintained after that.

This means that we need to not only freeze our repositories, but use a
frozen copy of rubygems/pypi/nodejs repositories for all types of tests we
run. This will require certain amount of effort from our Infra teams to
build up a mechanism of freezing/unfreezing the mirrors, determining how we
should add new pieces of code in such mirrors, may be, add some additional
code-review policies, create real mirroring scripts and so on.

But I think we should start doing this now in order to finish it at least
before 8.0 timeframe.

As a temporary solution, I suggest that we apply a series of changes into
these requirements.txt files before running actual tests instead of
modifying product code.

Please provide your comments and thoughts.


-- 
Yours Faithfully,
Vladimir Kuklin,
Fuel Library Tech Lead,
Mirantis, Inc.
+7 (495) 640-49-04
+7 (926) 702-39-68
Skype kuklinvv
35bk3, Vorontsovskaya Str.
Moscow, Russia,
www.mirantis.com http://www.mirantis.ru/
www.mirantis.ru
vkuk...@mirantis.com
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Fuel] Separate repo for Fuel Agent

2015-07-10 Thread Vladimir Kozhukalov

Ok, guys.

Looks like there are no any objections. At the moment I need to create
actual version of upstream repository which is going to be sucked in by
OpenStack Infra. Please, be informed that all patches changing
fuel-web/fuel_agent that will be merged after this moment will need to be
ported into the new fuel-agent repository.


Vladimir Kozhukalov

On Fri, Jul 10, 2015 at 6:38 PM, Vladimir Kozhukalov 
vkozhuka...@mirantis.com wrote:

 Guys, we are next to moving fuel_agent directory into a separate
 repository. Action flow is going to be as follows:

 1) Create verify jobs on our CI https://review.fuel-infra.org/#/c/9186
 (DONE)
 2) Freeze fuel_agent directory in https://github.com/stackforge/fuel-web
 (will announce in a separate mail thread). That means we stop merging
 patches into master which change fuel_agent directory. Unfortunately, all
 review requests need to be re-sent, but it is not going to be very
 difficult.
 3) Create temporary upstream repository with fuel_agent/* as a content.
 I'm not planning to move 5.x and 6.x branches. Only master. So, all fixes
 for 5.x and 6.x will be living in fuel-web.
 4) This upstream repository is going to be sucked in by openstack-infra.
 Patch is here https://review.openstack.org/#/c/199178/ (review is
 welcome) I don't know how long it is going to take. Will try to poke infra
 people to do this today.
 5) Then we need to accept two patches into new fuel-agent repository:
  - rpm spec (extraction from fuel-web/specs/nailgun.spec) (ready, but
 there is no review request)
  - run_tests.sh (to run tests) (ready, but there is no review request)

 !!! By this moment there won't be any impact on ISO build process !!!

 6) Then we need to change two things at the same time (review is welcome)
   - fuel-web/specs/nailgun.spec in order to prevent fuel-agent package
 building  https://review.openstack.org/#/c/200595
   - fuel-main so as to introduce new fuel-agent repository into the build
 process https://review.openstack.org/#/c/200025

 And good luck to me -)


 Vladimir Kozhukalov

 On Wed, Jul 8, 2015 at 12:53 PM, Vladimir Kozhukalov 
 vkozhuka...@mirantis.com wrote:

 There were some questions from Alexandra Fedorova about independent
 release cycle.

 according to the configuration [1] Infra team won't be able to do
 branching or any kind of release management for new repository.

 Could you please clarify, do we plan to version new repository the
 same way as we do for main fuel repositories or there going to be
 separate releases as in python-fuelclient [2]? Who should drive the
 release process for this repo and how this change will affect Fuel ISO
 release?

 [1]
 https://review.openstack.org/#/c/199178/1/gerrit/acls/stackforge/fuel-agent.config,cm
 [2]
 http://lists.openstack.org/pipermail/openstack-dev/2015-July/068837.html

 IMO all Fuel components should be as much independent as possible with
 highly defined APIs used for their interaction, with their own teams, with
 their own independent release cycles. But we cannot switch to this model
 immediately. For the start, we can just move those components into separate
 repositories, leaving the same access rights and core team as we have for
 fuel-web.

 When Fuel Agent is a separate repository we discuss team. It looks like a
 team leader is the best person to manage releases for a particular
 component. This thread is totally about separation stuff and how to do this
 not breaking anything.



 Vladimir Kozhukalov

 On Wed, Jul 8, 2015 at 12:24 PM, Vladimir Kozhukalov 
 vkozhuka...@mirantis.com wrote:

 Dear colleagues,

 I am going to move Fuel Agent into a separate git repository. The thing
 is that we have quite a few review requests to fuel-web with changes for
 Fuel Agent. The new repository is going to look like this
 https://github.com/kozhukalov/fuel-agent i.e. there is no additional
 sub-directory fuel_agent. In fact, I don't think it is a big deal to update
 all fuel agent related review requests.

 Work items:
 0) request to openstack-infra https://review.openstack.org/#/c/199178/1
 0.1) upstream for this request with commit history
 https://github.com/kozhukalov/fuel-agent
 1) fuel-agent/specs/fuel-agent.spec is an extraction from
 fuel-web/specs/nailgun.spec (separate commit, in progress)
 2) modify fuel-main to build fuel-agent package (in progress)
 3) create jenkins-jobs/servers/fuel-ci/verify-fuel-agent.yaml (in
 progress)

 For the start Fuel Agent core team will be the same as in fuel-web.

 If there is anything I forgot, please remind me about that.

 Vladimir Kozhukalov




__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all][tests] Fix it friday! [mock failure in CI]