Re: [openstack-dev] [nova][cinder][libvirt] Serious problem of block migration of a hybrid instance (instance based on local disk and cinder volume attached)

2015-05-20 Thread Matthew Gilliard
The change Dan refers to merged in Kilo. I assume that if you actually
found this problem then you're using an earlier OpenStack, so it's
worth noting that there's an outstanding backport:
https://review.openstack.org/#/c/176768/

Matthew


On Wed, May 20, 2015 at 3:21 AM, Daniel P. Berrange berra...@redhat.com wrote:
 On Wed, May 20, 2015 at 06:11:26PM +0800, Luo Gangyi wrote:
 Hi devs,


 I find a serious problem when I try to live migrate a hybrid instance
 (instance based on local disk and cinder volume attached) .


 In current nova, such operation is allowed.


 But I don't think libvirt can deal with such condition correctly.


 When block migration of a hybird intance is triggered, nova will initiate
 a new connection to the volume in destination compute node and then
 call libvirt to do the block migration.

 However, libvirt doesn't distinguish local disk and a network volume,
 it may copy both local disk and cinder volume from source to destination.

 It is dangerous  to write the same volume simutaneously on source and
 destination!! and apparently there is no need to do this!

 I believe we should do something to correct this, maybe patch libvirt?
 or just forbid user do such operation?

 Both. Nova now forbids this

 commit d667b6a63e80b2f8d6311c2cf224ba32628eed84
 Author: Chris St. Pierre stpie...@metacloud.com
 Date:   Wed Dec 3 16:16:34 2014 -0600

 libvirt: Fail when live block migrating instance with volumes

 This raises an exception when attempting to live block migrate (nova
 live-migration --block-migrate) an instance with attached volumes.
 libvirt copies these volumes from themselves to themselves. At a
 minimum, this is horribly slow and de-sparses a sparse volume; at
 worst, this could cause massive data corruption.

 Closes-Bug: 1398999
 Change-Id: Ibcd423976bb9fea46e3e1cb23cc8e5cd944d8fc2

 And work is being done to allow libvirt to be told to skip cinder
 volumes when migrating:

   https://www.redhat.com/archives/libvir-list/2015-May/msg00345.html

 Regards,
 Daniel
 --
 |: http://berrange.com  -o-http://www.flickr.com/photos/dberrange/ :|
 |: http://libvirt.org  -o- http://virt-manager.org :|
 |: http://autobuild.org   -o- http://search.cpan.org/~danberr/ :|
 |: http://entangle-photo.org   -o-   http://live.gnome.org/gtk-vnc :|

 __
 OpenStack Development Mailing List (not for usage questions)
 Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Nova][Neutron] Cross-project coordination

2015-04-24 Thread Matthew Gilliard
John,

  If there are no objections, I would like to volunteer myself as
API-WG liaison for Nova.

Matthew

On Thu, Apr 23, 2015 at 2:29 PM, John Garbutt j...@johngarbutt.com wrote:
 On 22 April 2015 at 23:41, Jay Pipes jaypi...@gmail.com wrote:
 On 04/22/2015 04:33 PM, Kyle Mestery wrote:

 On Wed, Apr 22, 2015 at 3:28 PM, Sean M. Collins s...@coreitpro.com
 mailto:s...@coreitpro.com wrote:

 Hi,

 Kyle Mestery has asked me to serve as a cross-project liaison between
 Neutron and Nova. So, I'd like to say hello, and
 direct everyone towards an etherpad that I have created.

 https://etherpad.openstack.org/p/nova-neutron

 The etherpad can serve as a way to collect items that should be
 discussed between the projects.

 I will be attending the Nova main meetings to field Neutron
 questions, or at least provide who on the Neutron side would know the
 answer to a question.

 This is a big job, and I'd like to see if there is a volunteer on the
 Nova side who would be interested in also helping this effort, who
 could
 do the reverse (Sit in on Neutron meetings, field Nova questions).

 Thank you, and I look forward to working with everyone!

 Sean, thank you so much for volunteering to take on this incredibly
 important job. I'm hoping we can get someone from the nova side to work
 hand-in-hand with you and ensure we continue to drive issues related to
 both nova and neutron to conclusion in a way which benefits are mutual
 users!

 +1

 Agreed. I'm hoping that someone in the Nova community -- note, this does not
 need to be a Nova core contributor -- can step up to the plate and serve in
 this critical role.

 +1

 I hope we can also find someone in Nova to attend the Glance meetings
 in a similar way, given the proposed changes in that area.

 More generally, I am hoping we can fill in some of the gaps in this
 list (and ideally replace my name where it appears):
 https://wiki.openstack.org/wiki/CrossProjectLiaisons

 Thanks,
 johnthetubaguy

 __
 OpenStack Development Mailing List (not for usage questions)
 Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Nova] Liberty specs are now open

2015-04-13 Thread Matthew Gilliard
 Dumb question from me, is there an easy way to get a view that filters out 
 specs that haven't been re-submitted against the liberty directory?

You can use a regex in the files: filter, so I think you mean:

https://review.openstack.org/#/q/status:open+project:openstack/nova-specs+file:%255E.*liberty.*,n,z

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [nova][libvirt] Block migrations and Cinder volumes

2015-04-02 Thread Matthew Gilliard
 Thanks for the clarification, is there a  bug tracking this in libvirt
 already?

 Actually I don't think there is one, so feel free to file one

I took the liberty of doing so:
https://bugzilla.redhat.com/show_bug.cgi?id=1208588

On Wed, Mar 18, 2015 at 6:11 PM, Daniel P. Berrange berra...@redhat.com wrote:
 On Wed, Mar 18, 2015 at 10:59:19AM -0700, Joe Gordon wrote:
 On Wed, Mar 18, 2015 at 3:09 AM, Daniel P. Berrange berra...@redhat.com
 wrote:

  On Wed, Mar 18, 2015 at 08:33:26AM +0100, Thomas Herve wrote:
Interesting bug.  I think I agree with you that there isn't a good
  solution
currently for instances that have a mix of shared and not-shared
  storage.
   
I'm curious what Daniel meant by saying that marking the disk
  shareable is
not
as reliable as we would want.
  
   I think this is the bug I reported here:
  https://bugs.launchpad.net/nova/+bug/1376615
  
   My initial approach was indeed to mark the disks are shareable: the
  patch (https://review.openstack.org/#/c/125616/) has comments around the
  issues, mainly around I/Ocache and SELinux isolation being disabled.
 
  Yep, those are both show stopper issues. The only solution is to fix the
  libvirt API for this first.
 

 Thanks for the clarification, is there a  bug tracking this in libvirt
 already?

 Actually I don't think there is one, so feel free to file one


 Regards,
 Daniel
 --
 |: http://berrange.com  -o-http://www.flickr.com/photos/dberrange/ :|
 |: http://libvirt.org  -o- http://virt-manager.org :|
 |: http://autobuild.org   -o- http://search.cpan.org/~danberr/ :|
 |: http://entangle-photo.org   -o-   http://live.gnome.org/gtk-vnc :|

 __
 OpenStack Development Mailing List (not for usage questions)
 Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [nova] is it possible to microversion a static class method?

2015-03-18 Thread Matthew Gilliard
I think that both ways of doing this should be supported.

Decorated private methods make sense if the different microversions
have nicely interchangeable bits of functionality but not if one of
the private methods would have to be a no-op. A method which just
passes is noise. Additionally there has been talk (but no code I'm
aware of yet) about having a check that version ranges of decorated
methods are both continuous and non-overlapping.

Explicit inline checks do lose some visual impact, but there will be
cases where it makes sense to do an extra-small something (add a
single key to a response-body dict or similar, for example) and a
couple-line code change is a much simpler code change in that case.

So, I've commented on all the patches in what I think is the correct
sequence to leave both suggestions in the devref.

  MG

On Mon, Mar 16, 2015 at 11:32 AM, Alex Xu sou...@gmail.com wrote:


 2015-03-16 12:26 GMT+08:00 Christopher Yeoh cbky...@gmail.com:

 So ultimately I think this is a style issue rather than a technical one. I
 think there
 are situations where one way looks clearer than another the other way
 does. Sorry I can't get around to putting up a couple of examples,
 ATM but to be clear there is no difference in the end result (no different
 side effects etc)


 Emmone more point for multiple version toplevel method: we should
 declare the version explicitly. multiple version internal method and manual
 version comparison are just hide version declaration into the code.





 On Mon, Mar 16, 2015 at 12:21 PM, Alex Xu sou...@gmail.com wrote:



 2015-03-16 9:48 GMT+08:00 Alex Xu sou...@gmail.com:



 2015-03-13 19:10 GMT+08:00 Sean Dague s...@dague.net:

 On 03/13/2015 02:55 AM, Chris Friesen wrote:
  On 03/12/2015 12:13 PM, Sean Dague wrote:
  On 03/12/2015 02:03 PM, Chris Friesen wrote:
  Hi,
 
  I'm having an issue with microversions.
 
  The api_version() code has a comment saying This decorator MUST
  appear
  first (the outermost decorator) on an API method for it to work
  correctly
 
  I tried making a microversioned static class method like this:
 
   @wsgi.Controller.api_version(2.4)  # noqa
   @staticmethod
   def _my_func(req, foo):
 
  and pycharm highlighted the api_version decorator and complained
  that
  This decorator will not receive a callable it may expect; the
  built-in
  decorator returns a special object.
 
  Is this a spurious warning from pycharm?  The pep8 checks don't
  complain.
 
  If I don't make it static, then pycharm suggests that the method
  could
  be static.
 
  *API method*
 
  This is not intended for use by methods below the top controller
  level.
  If you want conditionals lower down in your call stack pull the
  request
  version out yourself and use that.
 
  Both the original spec and doc/source/devref/api_microversions.rst
  contain text talking about decorating a private method.  The latter
  gives this example:
 
  @api_version(2.1, 2.4)
  def _version_specific_func(self, req, arg1):
  pass
 
  @api_version(min_version=2.5) #noqa
  def _version_specific_func(self, req, arg1):
  pass
 
  def show(self, req, id):
   common stuff 
  self._version_specific_func(req, foo)
   common stuff 
 
  It's entirely possible that such a private method might not need to
  reference self, and could therefore be static, so I think it's a
  valid
  question.

 That's a doc bug, we should change it.



 Actually it is not a bug. It's controversial point in the spec, but
 finally that was keep in the spec.

 http://specs.openstack.org/openstack/nova-specs/specs/kilo/approved/api-microversions.html

 The discussion at line 268
 https://review.openstack.org/#/c/127127/7/specs/kilo/approved/api-microversions.rst


 Submit a patch for devref https://review.openstack.org/164555  Let see
 whether we can get agreement





 -Sean

 --
 Sean Dague
 http://dague.net


 __
 OpenStack Development Mailing List (not for usage questions)
 Unsubscribe:
 openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev





 __
 OpenStack Development Mailing List (not for usage questions)
 Unsubscribe:
 openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



 __
 OpenStack Development Mailing List (not for usage questions)
 Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



 __
 OpenStack Development Mailing List (not for usage questions)
 Unsubscribe: 

Re: [openstack-dev] [nova] Temporary freeze on API changes

2015-02-23 Thread Matthew Gilliard
The devref has merged:
http://docs.openstack.org/developer/nova/devref/api_microversions.html

There are a few corner cases being worked on now, but the information
in there should be enough to add a new API change  version.

MG

On Fri, Feb 20, 2015 at 12:33 AM, Christopher Yeoh cbky...@gmail.com wrote:
 Hi,

 The Nova API subgroup is planning on releasing V2.1 on Monday (changing its
 status from experimental to CURRENT) and merging the first patch which uses
 microversions on top of 2.1 on the Wednesday.

 In the meantime we'd appreciate it if reviewers did not +A any patchset
 which changes the REST API (as it potentially means we'd have to sync
 changes to 2.1 as well. Any future api changes should only be applied to the
 v2.1/v3 tree and use the microversions mechanism.

 When https://review.openstack.org/#/c/140313/ is merged, that will signal
 that microversions
 is enabled and we are open for patches to v2.1 microversions. At this point
 no changes should be accepted to the old v2 api code and anything to the
 v2.1 code base should be done using the microversions mechanism.

 There is devref docs on how to use microversions either merged or working
 its way through gerrit.

 Regards,

 Chris

 __
 OpenStack Development Mailing List (not for usage questions)
 Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [nova] Questions on pep8 F811 hacking check for microversion

2015-01-28 Thread Matthew Gilliard
F811 is not part of our hacking lib - it's in flake8.  As far as I know,
it's not possible to selectively disable that for particular files or
methods.  And as mentioned earlier in the list and when I asked in
#openstack-dev the feeling was that we don't want to disable F811 globally
because it's a useful check. So I think we have to choose between:

* Continuing to use #noqa
* Disabling F811 globally
* Modifying flake8
* Something else I haven't thought of

  Matthew

On Wed, Jan 28, 2015 at 7:43 AM, Chen CH Ji jiche...@cn.ibm.com wrote:

 Is there a way to overwrite the rule in our hacking (not familiar with it
 ...)?
 if so ,maybe we can do as suggested to avoid 811 for the class which has
 Microversion definition? Thanks

 Best Regards!

 Kevin (Chen) Ji 纪 晨

 Engineer, zVM Development, CSTL
 Notes: Chen CH Ji/China/IBM@IBMCN   Internet: jiche...@cn.ibm.com
 Phone: +86-10-82454158
 Address: 3/F Ring Building, ZhongGuanCun Software Park, Haidian District,
 Beijing 100193, PRC

 [image: Inactive hide details for Christopher Yeoh ---01/28/2015 09:37:00
 AM---On Tue, 06 Jan 2015 07:31:19 -0500 Jay Pipes jaypipes@g]Christopher
 Yeoh ---01/28/2015 09:37:00 AM---On Tue, 06 Jan 2015 07:31:19 -0500 Jay
 Pipes jaypi...@gmail.com wrote:

 From: Christopher Yeoh cbky...@gmail.com
 To: openstack-dev@lists.openstack.org
 Date: 01/28/2015 09:37 AM
 Subject: Re: [openstack-dev] [nova] Questions on pep8 F811 hacking check
 for microversion
 --



 On Tue, 06 Jan 2015 07:31:19 -0500
 Jay Pipes jaypi...@gmail.com wrote:

  On 01/06/2015 06:25 AM, Chen CH Ji wrote:
   Based on nova-specs api-microversions.rst
   we support following function definition format, but it violate the
   hacking rule pep8 F811 because duplicate function definition
   we should use #noqa for them , but considering microversion may
   live for long time ,
   keep adding #noqa may be a little bit ugly, can anyone suggest a
   good solution for it ? thanks
  
   @api_version(min_version='2.1')
   def _version_specific_func(self, req, arg1):
  pass

   @api_version(min_version='2.5')
   def _version_specific_func(self, req, arg1):
  pass
 
  Hey Kevin,
 
  This was actually one of my reservations about the proposed
  microversioning implementation -- i.e. having functions that are
  named exactly the same, only decorated with the microversioning
  notation. It kinda reminds me of the hell of debugging C++ code that
  uses STL: how does one easily know which method one is in when inside
  a debugger?
 
  That said, the only other technique we could try to use would be to
  not use a decorator and instead have a top-level dispatch function
  that would inspect the API microversion (only when the API version
  makes a difference to the output or input of that function) and then
  dispatch the call to a helper method that had the version in its name.
 
  So, for instance, let's say you are calling the controller's GET
  /$tenant/os-hosts method, which happens to get routed to the
  nova.api.openstack.compute.contrib.hosts.HostController.index()
  method. If you wanted to modify the result of that method and the API
  microversion is at 2.5, you might do something like:
 
def index(self, req):
req_api_ver = utils.get_max_requested_api_version(req)
if req_api_ver == (2, 5):
return self.index_2_5(req)
return self.index_2_1(req)
 
def index_2_5(self, req):
results = self.index_2_1(req)
# Replaces 'host' with 'host_name'
for result in results:
result['host_name'] = result['host']
del result['host']
return results
 
def index_2_1(self, req):
# Would be a rename of the existing index() method on
# the controller
 

 So having to manually add switching code everything we have an API
 patch I think is not only longer and more complicated but more error
 prone when updating. If we change something at the core in the future it
 means changing all the microversioned code rather than just the
 switching architecture at the core of wsgi.


  Another option would be to use something like JSON-patch to determine
  the difference between two output schemas and automatically translate
  one to another... but that would be a huge effort.
 
  That's the only other way I can think of besides disabling F811,
  which I really would not recommend, since it's a valuable safeguard
  against duplicate function names (especially duplicated test methods).

 So I don't think we need to disable F811 in general - why not just
 disable it for any method with the api_version decorator? On those ones
 we can do checks on what is passed to api_version which will help
 verify that there hasn't been a typo to an api_version decorator.

 Chris

 __
 OpenStack Development Mailing List (not for usage questions)
 Unsubscribe: 

Re: [openstack-dev] [nova] novaclient support for V2.1 micro versions

2015-01-27 Thread Matthew Gilliard
As I understand it, novaclient would by default not pass the microversion
HTTP header (X-OpenStack-Compute-API-Version) so it would get the server's
MIN_VERSION, ie 2.1 for the foreseeable future.  It would be
straightforward to add something like a --api-version=xx command line
argument, or it might be useful to read a value from a local config file.
I don't think either of those has been done yet, though.

gilliard

On Fri, Jan 23, 2015 at 1:05 PM, Chen CH Ji jiche...@cn.ibm.com wrote:

 No, AFAICT it's not supported because the v2.1 microversion and related bp
 are still under implementation ,there is no change on novaclient now ...

 Best Regards!

 Kevin (Chen) Ji 纪 晨

 Engineer, zVM Development, CSTL
 Notes: Chen CH Ji/China/IBM@IBMCN   Internet: jiche...@cn.ibm.com
 Phone: +86-10-82454158
 Address: 3/F Ring Building, ZhongGuanCun Software Park, Haidian District,
 Beijing 100193, PRC

 [image: Inactive hide details for Day, Phil ---01/23/2015 05:56:47
 PM---Hi Folks, Is there any support yet in novaclient for requesti]Day,
 Phil ---01/23/2015 05:56:47 PM---Hi Folks, Is there any support yet in
 novaclient for requesting a specific microversion ?   (looking

 From: Day, Phil philip@hp.com
 To: OpenStack Development Mailing List (openstack-dev@lists.openstack.org)
 openstack-dev@lists.openstack.org
 Date: 01/23/2015 05:56 PM
 Subject: [openstack-dev] [nova] novaclient support for V2.1 micro versions
 --



 Hi Folks,

 Is there any support yet in novaclient for requesting a specific
 microversion ?   (looking at the final leg of extending clean-shutdown to
 the API, and wondering how to test this in devstack via the novaclient)

 Phil


 __
 OpenStack Development Mailing List (not for usage questions)
 Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


 __
 OpenStack Development Mailing List (not for usage questions)
 Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


[openstack-dev] [libvirt][vpnaas] IRC Meeting Clash

2014-12-19 Thread Matthew Gilliard
Hello,

  At the moment, both Libvirt [1] and VPNaaS [2] are down as having
meetings in #openstack-meeting-3 at 1500UTC on Tuesdays. Of course,
there can be only one - and it looks as if the VPN meeting is the one
that actually takes place there.

  What's the status of the libvirt meetings? Have they moved, or are
they not happening any more?

  Thanks,

Matthew

[1] https://wiki.openstack.org/wiki/Meetings#Libvirt_Meeting
[2] 
https://wiki.openstack.org/wiki/Meetings#VPN_as_a_Service_.28VPNaaS.29_team_meeting

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [nova] Complexity check and v2 API

2014-12-17 Thread Matthew Gilliard
Hello Pasquale

  The problem is that you are trying to add a new if/else branch into
a method which is already ~250 lines long, and has the highest
complexity of any function in the nova codebase. I assume that you
didn't contribute much to that complexity, but we've recently added a
limit to stop it getting any worse. So, regarding your 4 suggestions:

1/ As I understand it, v2.1 should be the same as v2 at the
moment, so they need to be kept the same
2/ You can't ignore it - it will fail CI
3/ No thank you. This limit should only ever be lowered :-)
4/ This is 'the right way'. Your suggestion for the refactor does
sound good.

I suggest a single patch that refactors and lowers the limit in
tox.ini.  Once you've done that then you can add the new parameter in
a following patch. Please feel free to add me to any patches you
create.

Matthew



On Wed, Dec 17, 2014 at 4:18 PM, Pasquale Porreca
pasquale.porr...@dektech.com.au wrote:
 Hello

 I am working on an API extension that adds a parameter on create server
 call; to implement the v2 API I added few lines of code to
 nova/api/openstack/compute/servers.py

 In particular just adding something like

 new_param = None
 if self.ext_mgr.is_loaded('os-new-param'):
 new_param = server_dict.get('new_param')

 leads to a pep8 fail with message 'Controller.create' is too complex (47)
 (Note that in tox.ini the max complexity is fixed to 47 and there is a note
 specifying 46 is the max complexity present at the moment).

 It is quite easy to make this test pass creating a new method just to
 execute these lines of code, anyway all other extensions are handled in that
 way and one of most important stylish rule states to be consistent with
 surrounding code, so I don't think a separate function is the way to go
 (unless it implies a change in how all other extensions are handled too).

 My thoughts on this situation:

 1) New extensions should not consider v2 but only v2.1, so that file should
 not be touched
 2) Ignore this error and go on: if and when the extension will be merged the
 complexity in tox.ini will be changed too
 3) The complexity in tox.ini should be raised to allow new v2 extensions
 4) The code of that module should be refactored to lower the complexity
 (i.e. move the load of each extension in a separate function)

 I would like to know if any of my point is close to the correct solution.

 --
 Pasquale Porreca

 DEK Technologies
 Via dei Castelli Romani, 22
 00040 Pomezia (Roma)

 Mobile +39 3394823805
 Skype paskporr


 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] People of OpenStack (and their IRC nicks)

2014-12-10 Thread Matthew Gilliard
So, are we agreed that http://www.openstack.org/community/members/ is
the authoritative place for IRC lookups? In which case, I'll take the
old content out of https://wiki.openstack.org/wiki/People and leave a
message directing people where to look.

I don't have the imagination to use anything other than my real name
on IRC but for people who do, should we try to encourage putting the
IRC nick in the gerrit name?

On Tue, Dec 9, 2014 at 11:56 PM, Clint Byrum cl...@fewbar.com wrote:
 Excerpts from Angus Salkeld's message of 2014-12-09 15:25:59 -0800:
 On Wed, Dec 10, 2014 at 5:11 AM, Stefano Maffulli stef...@openstack.org
 wrote:

  On 12/09/2014 06:04 AM, Jeremy Stanley wrote:
   We already have a solution for tracking the contributor-IRC
   mapping--add it to your Foundation Member Profile. For example, mine
   is in there already:
  
   http://www.openstack.org/community/members/profile/5479
 
  I recommend updating the openstack.org member profile and add IRC
  nickname there (and while you're there, update your affiliation history).
 
  There is also a search engine on:
 
  http://www.openstack.org/community/members/
 
 
 Except that info doesn't appear nicely in review. Some people put their
 nick in their Full Name in
 gerrit. Hopefully Clint doesn't mind:

 https://review.openstack.org/#/q/owner:%22Clint+%27SpamapS%27+Byrum%22+status:open,n,z


 Indeed, I really didn't like that I'd be reviewing somebody's change,
 and talking to them on IRC, and not know if they knew who I was.

 It also has the odd side effect that gerritbot triggers my IRC filters
 when I 'git review'.

 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] People of OpenStack (and their IRC nicks)

2014-12-10 Thread Matthew Gilliard
 I'll take the
 old content out of https://wiki.openstack.org/wiki/People and leave a
 message directing people where to look.
 Yes, please, let me know if you need help.

Done.

 to link
 directly gerrit IDs to openstack.org profile URL

This may be possible with a little javascript hackery in gerrit - I'll
see what I can do there.

 which IRC nick goes to which person. Does anyone know how to do
 that with the Foundation directory?
 I don't think there's a lookup for that (might be worth logging a
 feature request)

Done: https://bugs.launchpad.net/openstack-org/+bug/1401264

Thanks for your time everyone.


  Matthew

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


[openstack-dev] People of OpenStack (and their IRC nicks)

2014-12-09 Thread Matthew Gilliard
Sometimes, I want to ask the author of a patch about it on IRC.
However, there doesn't seem to be a reliable way to find out someone's
IRC handle.  The potential for useful conversation is sometimes
missed.  Unless there's a better alternative which I didn't find,
https://wiki.openstack.org/wiki/People seems to fulfill that purpose,
but is neither complete nor accurate.

  What do people think about this? Should we put more effort into
keeping the People wiki up-to-date?  That's a(nother) manual process
though - can we autogenerate it somehow?

  Matthew

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [nova] global or per-project specific ssl config options, or both?

2014-12-05 Thread Matthew Gilliard
Hi Matt, Nova,

  I'll look into this.

Gilliard

On Thu, Dec 4, 2014 at 9:51 PM, Matt Riedemann
mrie...@linux.vnet.ibm.com wrote:


 On 12/4/2014 6:02 AM, Davanum Srinivas wrote:

 +1 to @markmc's default is global value and override for project
 specific key suggestion.

 -- dims



 On Wed, Dec 3, 2014 at 11:57 PM, Matt Riedemann
 mrie...@linux.vnet.ibm.com wrote:

 I've posted this to the 12/4 nova meeting agenda but figured I'd
 socialize
 it here also.

 SSL options - do we make them per-project or global, or both? Neutron and
 Cinder have config-group specific SSL options in nova, Glance is using
 oslo
 sslutils global options since Juno which was contentious for a time in a
 separate review in Icehouse [1].

 Now [2] wants to break that out for Glance, but we also have a patch [3]
 for
 Keystone to use the global oslo SSL options, we should be consistent, but
 does that require a blueprint now?

 In the Icehouse patch, markmc suggested using a DictOpt where the default
 value is the global value, which could be coming from the oslo [ssl]
 group
 and then you could override that with a project-specific key, e.g.
 cinder,
 neutron, glance, keystone.

 [1] https://review.openstack.org/#/c/84522/
 [2] https://review.openstack.org/#/c/131066/
 [3] https://review.openstack.org/#/c/124296/

 --

 Thanks,

 Matt Riedemann


 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev





 The consensus in the nova meeting today, I think, was that we generally like
 the idea of the DictOpt with global oslo ssl as the default and then be able
 to configure that per-service if needed.

 Does anyone want to put up a POC on how that would work to see how ugly
 and/or usable that would be?  I haven't dug into the DictOpt stuff yet and
 am kind of time-constrained at the moment.


 --

 Thanks,

 Matt Riedemann


 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [nova] global or per-project specific ssl config options, or both?

2014-12-05 Thread Matthew Gilliard
I just put up a quick pre-weekend POC at
https://review.openstack.org/#/c/139672/ - comments welcome on that
patch.

Thanks :)

On Fri, Dec 5, 2014 at 10:07 AM, Matthew Gilliard
matthew.gilli...@gmail.com wrote:
 Hi Matt, Nova,

   I'll look into this.

 Gilliard

 On Thu, Dec 4, 2014 at 9:51 PM, Matt Riedemann
 mrie...@linux.vnet.ibm.com wrote:


 On 12/4/2014 6:02 AM, Davanum Srinivas wrote:

 +1 to @markmc's default is global value and override for project
 specific key suggestion.

 -- dims



 On Wed, Dec 3, 2014 at 11:57 PM, Matt Riedemann
 mrie...@linux.vnet.ibm.com wrote:

 I've posted this to the 12/4 nova meeting agenda but figured I'd
 socialize
 it here also.

 SSL options - do we make them per-project or global, or both? Neutron and
 Cinder have config-group specific SSL options in nova, Glance is using
 oslo
 sslutils global options since Juno which was contentious for a time in a
 separate review in Icehouse [1].

 Now [2] wants to break that out for Glance, but we also have a patch [3]
 for
 Keystone to use the global oslo SSL options, we should be consistent, but
 does that require a blueprint now?

 In the Icehouse patch, markmc suggested using a DictOpt where the default
 value is the global value, which could be coming from the oslo [ssl]
 group
 and then you could override that with a project-specific key, e.g.
 cinder,
 neutron, glance, keystone.

 [1] https://review.openstack.org/#/c/84522/
 [2] https://review.openstack.org/#/c/131066/
 [3] https://review.openstack.org/#/c/124296/

 --

 Thanks,

 Matt Riedemann


 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev





 The consensus in the nova meeting today, I think, was that we generally like
 the idea of the DictOpt with global oslo ssl as the default and then be able
 to configure that per-service if needed.

 Does anyone want to put up a POC on how that would work to see how ugly
 and/or usable that would be?  I haven't dug into the DictOpt stuff yet and
 am kind of time-constrained at the moment.


 --

 Thanks,

 Matt Riedemann


 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [nova] Proposal new hacking rules

2014-11-24 Thread Matthew Gilliard
1/ assertFalse() vs assertEqual(x, False) - these are semantically
different because of python's notion of truthiness, so I don't think
we ought to make this a rule.

2/ expected/actual - incorrect failure messages have cost me more time
than I should admit to. I don't see any reason not to try to improve
in this area, even if it's difficult to automate.

3/ warn{ing} - 
https://github.com/openstack/nova/blob/master/nova/hacking/checks.py#L322

On the overarching point: There is no way to get started with
OpenStack, other than starting small.  My first ever patch (a tidy-up)
was rejected for being trivial, and that was confusing and
disheartening. Nova has a lot on its plate, sure, and plenty of
pending code reviews.  But there is also a lot of inconsistency and
unloved code which *is* worth fixing, because a tidy codebase is a joy
to work with, *and* these changes are ideal to bring new reviewers and
developers into the project.

Linus' post on this from the LKML is almost a decade old (!) but worth reading.
https://lkml.org/lkml/2004/12/20/255

  MG

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [all] add cyclomatic complexity check to pep8 target

2014-10-17 Thread Matthew Gilliard
I like measuring code metrics, and I definitely support Joe's change
here. I think of McCabe complexity as a proxy for testability and
readability of code, both of which are IMO actual real problems in the
nova codebase. If you are an experienced openstack dev you might find
the code easy to move around, but large and complex functions are
difficult for beginners to grok.

As an exercise, I took the method in libvirt/config.py and removed
everything except the flow-control keywords (ie the things that affect
the McCabe complexity): http://paste.openstack.org/show/121589/ - I
would find it difficult to hold all that in my head at once. It's
possible to argue that this is a false-positive, but my experience is
that this tool finds code which need improvement.

That said, these should be descriptive metrics rather than
prescriptive targets. There are products which chart a codebase's
evolution over time, such as www.sonarsource.com, which are really
great for provoking thought and conversation about code quality. Now
I'm interested, I'll have a look into it.

  Matthew

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


[openstack-dev] [tripleo] [ironic] Multiple VM sizes for devtest

2014-07-14 Thread Matthew Gilliard
  As the stacks we are trying to stand up get more and more complicated,
the demands on developers' hardware is increasing.  Currently if you need N
VMs for your devtest run, you will create N which are identically sized (
https://github.com/openstack/tripleo-incubator/blob/master/scripts/devtest_testenv.sh#L245)
So, you are forced to make every node the size of the largest one you
need.  Of course this size isn't actually necessary for most nodes, and a
way of having differently-sized nodes would save hardware resources (so we
can create even more VMs!)

  Devananda and I worked up a patch for Ironic (
https://review.openstack.org/#/c/105802/) which will allow arbitrary
capabilities (ie k/v pairs) to be added to ironic nodes.  These nodes can
then be targeted by Nova using flavor-keys.  Within devtest, we could think
of these as roles for nodes.  I think that is what lifeless is hinting at
here:
https://github.com/openstack/tripleo-incubator/blob/master/scripts/setup-baremetal#L101

  In my head, a rough plan is: allow a user to provide a file detailing the
specs of the VMs they want.  (If it's not supplied we can fallback to the
current behaviour).  Nodes are created and given roles according to the
contents of that file, and flavors are created to match.  When the UC and
OC VMs are booted, the flavor appropriate to the role is used.  As you
can imagine, most parts of devtest would need some changes - I guess around
a half-dozen individual patchsets might do it, maybe more.

  So, I put this email out to see what people's reactions are.  If they are
generally positive I think it would be worth a tripleo-spec to put a bit
more detail together.  What do you think?

  Matthew
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


[openstack-dev] [neutron] Port leak when instance fails to boot normally

2014-06-03 Thread Matthew Gilliard
Bug report: https://bugs.launchpad.net/nova/+bug/1324934

TL;DR version of the bug report:  A dropped connection during nova-neutron
call to create a port will result in nova deciding that the instance can't
be booted, and therefore terminating it.  Nova calls get_ports() from
neutron during this termination, but an empty list is returned so no ports
are deleted.  However, the port actually is created, and associated to the
now-deleted instance.

I suspect there is a race condition between create_port and get_ports in
neutron, which could be fixed by using
@lockutils.synchronized(nstance_uuid) on those 2 methods.

I'm not very familiar with the neutron codebase though, and would
appreciate some feedback or discussion on how likely people think my
diagnosis and suggested fix are, and whether similar problems have been
found in the past.

  Thanks,

Matthew Gilliard
(HP Cloud)
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [oslo][nova] how to best deal with default periodic task spacing behavior

2014-05-22 Thread Matthew Gilliard
We (HP Helion) are completely fine with a change that reduces server load
and makes things more predictable.  It would have been very hard to rely
heavily on the old behaviour.  Thanks for adding me to the patch, too, Matt

  Matthew


On Wed, May 21, 2014 at 4:40 PM, Doug Hellmann
doug.hellm...@dreamhost.comwrote:

 On Tue, May 20, 2014 at 10:15 AM, Matt Riedemann
 mrie...@linux.vnet.ibm.com wrote:
  Between patch set 1 and patch set 3 here [1] we have different solutions
 to
  the same issue, which is if you don't specify a spacing value for
 periodic
  tasks then they run whenever the periodic task processor runs, which is
  non-deterministic and can be staggered if some tasks don't complete in a
  reasonable amount of time.
 
  I'm bringing this to the mailing list to see if there are more opinions
 out
  there, especially from operators, since patch set 1 changes the default

 You may get more feedback from operators on the main openstack list.
 I'm still catching up on backlog after the summit, so apologies if
 you've already posted there.

 Doug

  behavior to have the spacing value be the DEFAULT_INTERVAL (hard-coded 60
  seconds) versus patch set 3 which makes that behavior configurable so the
  admin can set global default spacing for tasks, but defaults to the
 current
  behavior of running every time if not specified.
 
  I don't like a new config option, but I'm also not crazy about changing
  existing behavior without consensus.
 
  [1] https://review.openstack.org/#/c/93767/
 
  --
 
  Thanks,
 
  Matt Riedemann
 
 
  ___
  OpenStack-dev mailing list
  OpenStack-dev@lists.openstack.org
  http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [nova] Making periodic tasks config consistent.

2014-02-07 Thread Matthew Gilliard
I think I mangled my explanation a bit, sorry.  I'm suggesting 2 things:

1 - make Nova's use of 0-length polling period consistent with the Oslo
decorator.  Nova-only.

2 - have Oslo's decorator use 0-length to mean call regularly at the
default period,
rather than call whenever any other task is run.

These can be done separately, but both are breaking changes to the current
behaviours so the approach of logging deprecation warnings and changing
behaviour in Juno seems appropriate in both cases.  I will make a start
soon on the Nova side, and see how it is received by the reviewers.

  Matthew





On Thu, Feb 6, 2014 at 11:44 AM, Matthew Gilliard 
matthew.gilli...@gmail.com wrote:

 If there is agreement that it's a change worth making, then I expect
 something like:

 1/ Add a warning for users who use period of 0 or use the default.  Both
 in the literal sense of log.warning() and in the documentation.
 2/ wait for a full release-cycle
 3/ make the actual change in Juno.

 Does that make sense?


 On Thu, Feb 6, 2014 at 9:46 AM, Michael Still mi...@stillhq.com wrote:

 On Thu, Feb 6, 2014 at 8:16 PM, Matthew Gilliard
 matthew.gilli...@gmail.com wrote:
  Hello everyone.
 
wrt these bugs: https://bugs.launchpad.net/nova/+bug/1276203
  https://bugs.launchpad.net/nova/+bug/1272830 - I'd just like to make
 sure
  that the approach I'm planning makes sense.
 
To summarise: Currently there are a number of methods in
  compute/manager.py that use the @periodic_task decorator.  Some of them
 also
  do their own checks about how often they are called, and use a
 convention of
  polling period = 0 to disable the method by returning early (although
 this
  is sometimes implemented as =0 [1] and sometimes as ==0 [2]).  In the
  decorator itself though, a polling period of 0 is used to mean call
 this
  method any time any other period task is run [3].  It's difficult to
  predict how often this might be, and it may not be at regular intervals.
 
I'd like to make this more consistent and predictable.  My plan is to
 use
  the following:
 
- Any positive integer: the method is called every this many
 seconds,
  best effort is made not to call it more or less often.
- 0: the method will be called regularly at the default period.
  Currently
  hard-coded to 60s [4] this could be made into a config option
- Any negative integer: the method will not be called
 
All this logic would be contained in the decorator so that the methods
  themselves can just get on with whatever business they have.  So far, I
 hope
  this isn't too contentious - just clean code.  Is there any case that
 I've
  missed?  The fix will necessarily be a breaking change.  So how do you
  suggest I approach that aspect?  As it's common code, should I actually
 be
  looking to make these changes in Oslo first then porting them in?

 The decorator comes from oslo, so you're talking about changing the
 default flag behaviour for pretty much every openstack project here.
 How do we do this in a way which doesn't have unexpected side effects
 for deployments?

 Michael

 --
 Rackspace Australia

 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


[openstack-dev] [nova] Making periodic tasks config consistent.

2014-02-06 Thread Matthew Gilliard
Hello everyone.

  wrt these bugs: https://bugs.launchpad.net/nova/+bug/1276203
https://bugs.launchpad.net/nova/+bug/1272830 - I'd just like to make sure
that the approach I'm planning makes sense.

  To summarise: Currently there are a number of methods in
compute/manager.py that use the @periodic_task decorator.  Some of them
also do their own checks about how often they are called, and use a
convention of polling period = 0 to disable the method by returning early
(although this is sometimes implemented as =0 [1] and sometimes as ==0
[2]).  In the decorator itself though, a polling period of 0 is used to
mean call this method any time any other period task is run [3].  It's
difficult to predict how often this might be, and it may not be at regular
intervals.

  I'd like to make this more consistent and predictable.  My plan is to use
the following:

  - Any positive integer: the method is called every this many seconds,
best effort is made not to call it more or less often.
  - 0: the method will be called regularly at the default period.
Currently hard-coded to 60s [4] this could be made into a config option
  - Any negative integer: the method will not be called

  All this logic would be contained in the decorator so that the methods
themselves can just get on with whatever business they have.  So far, I
hope this isn't too contentious - just clean code.  Is there any case that
I've missed?  The fix will necessarily be a breaking change.  So how do you
suggest I approach that aspect?  As it's common code, should I actually be
looking to make these changes in Oslo first then porting them in?

  Thanks,

Matthew Gilliard

[1]
https://github.com/openstack/nova/blob/master/nova/compute/manager.py#L4702
[2]
https://github.com/openstack/nova/blob/master/nova/compute/manager.py#L4702
[3]
https://github.com/openstack/nova/blob/master/nova/openstack/common/periodic_task.py#L144
[4]
https://github.com/openstack/nova/blob/master/nova/openstack/common/periodic_task.py#L39
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [nova] Making periodic tasks config consistent.

2014-02-06 Thread Matthew Gilliard
If there is agreement that it's a change worth making, then I expect
something like:

1/ Add a warning for users who use period of 0 or use the default.  Both in
the literal sense of log.warning() and in the documentation.
2/ wait for a full release-cycle
3/ make the actual change in Juno.

Does that make sense?


On Thu, Feb 6, 2014 at 9:46 AM, Michael Still mi...@stillhq.com wrote:

 On Thu, Feb 6, 2014 at 8:16 PM, Matthew Gilliard
 matthew.gilli...@gmail.com wrote:
  Hello everyone.
 
wrt these bugs: https://bugs.launchpad.net/nova/+bug/1276203
  https://bugs.launchpad.net/nova/+bug/1272830 - I'd just like to make
 sure
  that the approach I'm planning makes sense.
 
To summarise: Currently there are a number of methods in
  compute/manager.py that use the @periodic_task decorator.  Some of them
 also
  do their own checks about how often they are called, and use a
 convention of
  polling period = 0 to disable the method by returning early (although
 this
  is sometimes implemented as =0 [1] and sometimes as ==0 [2]).  In the
  decorator itself though, a polling period of 0 is used to mean call this
  method any time any other period task is run [3].  It's difficult to
  predict how often this might be, and it may not be at regular intervals.
 
I'd like to make this more consistent and predictable.  My plan is to
 use
  the following:
 
- Any positive integer: the method is called every this many seconds,
  best effort is made not to call it more or less often.
- 0: the method will be called regularly at the default period.
  Currently
  hard-coded to 60s [4] this could be made into a config option
- Any negative integer: the method will not be called
 
All this logic would be contained in the decorator so that the methods
  themselves can just get on with whatever business they have.  So far, I
 hope
  this isn't too contentious - just clean code.  Is there any case that
 I've
  missed?  The fix will necessarily be a breaking change.  So how do you
  suggest I approach that aspect?  As it's common code, should I actually
 be
  looking to make these changes in Oslo first then porting them in?

 The decorator comes from oslo, so you're talking about changing the
 default flag behaviour for pretty much every openstack project here.
 How do we do this in a way which doesn't have unexpected side effects
 for deployments?

 Michael

 --
 Rackspace Australia

 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev