Re: [openstack-dev] [Nova][FFE] Feature freeze exception for virt-driver-numa-placement

2014-09-05 Thread Nikola Đipanov
On 09/04/2014 07:42 PM, Murray, Paul (HP Cloud) wrote:
  
 
 
 
 Anyway, not enough to -1 it, but enough to at least say something.
 
 
 
 
 
 
 
 
 
 
 
 .. but I do not want to get into the discussion about software testing
 
 
 
 here, not the place really.
 
 
 
 
 
 
 
 However, I do think it is very harmful to respond to FFE request with
 
 
 
 such blanket statements and generalizations, if only for the message it
 
 
 
 sends to the contributors (that we really care more about upholding our
 
 
 
 own myths as a community than users and features).
 
 
 
 
 
 
 
 
 
 
 
 I believe you brought this up as one of your justifications for the FFE.
 
 When I read your statement it does sound as though you want to put
 
 experimental code in at the final release. I am sure that is not what
 
 you had in mind, but I am also sure you can also understand Sean's point
 
 of view. His point is clear and pertinent to your request.
 
 
 
 
 
 
 
 As the person responsible for Nova in HP I will be interested to see how
 
 it operates in practice. I can assure you we will do extensive testing
 
 on it before it goes into the wild and we will not put it into practice
 
 if we are not happy.
 
 
 
  
 
 That is awesome and we as a project are lucky to have that! I would not
 
 want things put into practice that users can't use or see huge flaws with.
 
  
 
 I can't help but read this as you being OK with the feature going ahead,
 
 though :).
 
  
 
  
 
 Actually, let’s say I have no particular objection. Just thought Sean’s
 point is worth noting.
 
  
 
 Now, if this had been done as an extensible resource I could easily
 decouple deploying it from all the bug fixes that come through with the
 release. But that’s another matter…
 
  

Quick response as not to hijack the thread:

I think we all agree on the benefits of having resources you can turn
off and on at will.

The current implementation of it, however, has some glaring drawbacks
that made it impossible for me to base my work on it, that have been
discussed in detail on other threads and IRC heavily, hence we need to
rethink how to get there.

 
 Paul
 
 
 
 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
 


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Nova][FFE] Feature freeze exception for virt-driver-numa-placement

2014-09-05 Thread Nikola Đipanov
Since this did not get an 'Approved' as of yet, I want to make sure that
this is not because the number of sponsors. 2 core members have already
sponsored it, and as per [1] cores can sponsor their own FFEs so that's 3.

N.

[1]
http://lists.openstack.org/pipermail/openstack-dev/2014-September/044669.html

On 09/04/2014 01:58 PM, Nikola Đipanov wrote:
 Hi team,
 
 I am requesting the exception for the feature from the subject (find
 specs at [1] and outstanding changes at [2]).
 
 Some reasons why we may want to grant it:
 
 First of all all patches have been approved in time and just lost the
 gate race.
 
 Rejecting it makes little sense really, as it has been commented on by a
 good chunk of the core team, most of the invasive stuff (db migrations
 for example) has already merged, and the few parts that may seem
 contentious have either been discussed and agreed upon [3], or can
 easily be addressed in subsequent bug fixes.
 
 It would be very beneficial to merge it so that we actually get real
 testing on the feature ASAP (scheduling features are not tested in the
 gate so we need to rely on downstream/3rd party/user testing for those).
 
 Thanks,
 
 Nikola
 
 [1]
 http://git.openstack.org/cgit/openstack/nova-specs/tree/specs/juno/virt-driver-numa-placement.rst
 [2]
 https://review.openstack.org/#/q/status:open+project:openstack/nova+branch:master+topic:bp/virt-driver-numa-placement,n,z
 [3] https://review.openstack.org/#/c/111782/
 
 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
 


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Nova][FFE] Feature freeze exception for virt-driver-numa-placement

2014-09-05 Thread Jay Pipes

On 09/05/2014 04:48 AM, Nikola Đipanov wrote:

Quick response as not to hijack the thread:

I think we all agree on the benefits of having resources you can turn
off and on at will.


I don't agree at all. There's no cost whatsoever in turning on a 
resource. It doesn't need to be extensible. Resources just need to 
properly modelled so that the things they represent can be properly 
compared in a quantitative manner. For example, a NUMA cell resource 
needs to be modelled in a Python object that can be compared to a 
request for a certain NUMA topology.


There was and continues to be no need to make resources extensible. They 
just needed to be properly modelled into Python objects, and those 
objects used in scheduling decisions and tracking.



The current implementation of it, however, has some glaring drawbacks
that made it impossible for me to base my work on it, that have been
discussed in detail on other threads and IRC heavily, hence we need to
rethink how to get there.


++

-jay

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Nova][FFE] Feature freeze exception for virt-driver-numa-placement

2014-09-05 Thread Daniel P. Berrange
On Fri, Sep 05, 2014 at 04:20:14PM +0100, John Garbutt wrote:
 On 5 September 2014 13:59, Nikola Đipanov ndipa...@redhat.com wrote:
  Since this did not get an 'Approved' as of yet, I want to make sure that
  this is not because the number of sponsors. 2 core members have already
  sponsored it, and as per [1] cores can sponsor their own FFEs so that's 3.
 
 While I am no fan of that idea, this was already in the gate, so 2
 cores should be more than enough.
 
 Mikal has said I could approve FFEs in his absence, but given the
 conflict in the thread, I want to leave this approval to him :)
 
 I know its 10 patches, but I think we can try resolve the discussion
 on the thread, and look to approve this on Monday morning, and still
 make it before the deadline. Do shout up if that seems impossible
 and/or stupid.

If you want to delegate to Mikal for casting vote I think that
is not the end of the world. After all, few of us are going to
be seriously working over the weekend, so approving FFE tonight
vs monday morning austrailia time isn't a significant difference.
I assume you'll ping him to ensure he sees it on monday as a high
priority item.

As a general point though, I'm wary of letting the FFE proposal
be a place where we re-open the design discussions on any of the
FFE requests. These patches have been under active review and were
all approved to merge with no -2-severity objections. If it had
not been for the huge backlog this would all be in tree already
and we'd be focusing on testing it and ironing out any bugs we
identify, which is ultimately where our focus needs to be now.

Regards,
Daniel
-- 
|: http://berrange.com  -o-http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org  -o- http://virt-manager.org :|
|: http://autobuild.org   -o- http://search.cpan.org/~danberr/ :|
|: http://entangle-photo.org   -o-   http://live.gnome.org/gtk-vnc :|

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Nova][FFE] Feature freeze exception for virt-driver-numa-placement

2014-09-05 Thread Jay Pipes

On 09/05/2014 11:20 AM, John Garbutt wrote:

On 5 September 2014 13:59, Nikola Đipanov ndipa...@redhat.com wrote:

Since this did not get an 'Approved' as of yet, I want to make sure that
this is not because the number of sponsors. 2 core members have already
sponsored it, and as per [1] cores can sponsor their own FFEs so that's 3.


While I am no fan of that idea, this was already in the gate, so 2
cores should be more than enough.


I am currently reviewing the final patch series in this and am willing 
to be the third sponsor. I've reviewed most of the NUMA-related patches 
from Dan and Nikola in the past few months so I'm pretty familiar with 
the work, as well as the difficulties Nikola has run into in the 
scheduler code regarding adding this functionality.


-jay


Mikal has said I could approve FFEs in his absence, but given the
conflict in the thread, I want to leave this approval to him :)

I know its 10 patches, but I think we can try resolve the discussion
on the thread, and look to approve this on Monday morning, and still
make it before the deadline. Do shout up if that seems impossible
and/or stupid.

Thanks,
John


[1]
http://lists.openstack.org/pipermail/openstack-dev/2014-September/044669.html

On 09/04/2014 01:58 PM, Nikola Đipanov wrote:

Hi team,

I am requesting the exception for the feature from the subject (find
specs at [1] and outstanding changes at [2]).

Some reasons why we may want to grant it:

First of all all patches have been approved in time and just lost the
gate race.

Rejecting it makes little sense really, as it has been commented on by a
good chunk of the core team, most of the invasive stuff (db migrations
for example) has already merged, and the few parts that may seem
contentious have either been discussed and agreed upon [3], or can
easily be addressed in subsequent bug fixes.

It would be very beneficial to merge it so that we actually get real
testing on the feature ASAP (scheduling features are not tested in the
gate so we need to rely on downstream/3rd party/user testing for those).

Thanks,

Nikola

[1]
http://git.openstack.org/cgit/openstack/nova-specs/tree/specs/juno/virt-driver-numa-placement.rst
[2]
https://review.openstack.org/#/q/status:open+project:openstack/nova+branch:master+topic:bp/virt-driver-numa-placement,n,z
[3] https://review.openstack.org/#/c/111782/

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev




___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Nova][FFE] Feature freeze exception for virt-driver-numa-placement

2014-09-05 Thread Michael Still
For better or for worse we have already merged about half of the
patches for this series, so I think stopping now because of concerns
about CI is pretty arbitrary. I do think Sean's point about scheduler
tests outside of tempest is valid though and I'd like to see it
reflected in the review comments on the relevant patch(es).

On the CI front it is true that we have features not covered by CI in
the gate now (live migration for example), but it is also true that
they are some of our least reliable features. I see this as an
important aspect of the need to pay down tech debt, and I'd like to
see us have a more serious go at doing that in Kilo than we managed in
Juno.

This has three sponsors, so I am therefore approving it.

Michael

On Fri, Sep 5, 2014 at 3:23 PM, Jay Pipes jaypi...@gmail.com wrote:
 On 09/05/2014 11:20 AM, John Garbutt wrote:

 On 5 September 2014 13:59, Nikola Đipanov ndipa...@redhat.com wrote:

 Since this did not get an 'Approved' as of yet, I want to make sure that
 this is not because the number of sponsors. 2 core members have already
 sponsored it, and as per [1] cores can sponsor their own FFEs so that's
 3.


 While I am no fan of that idea, this was already in the gate, so 2
 cores should be more than enough.


 I am currently reviewing the final patch series in this and am willing to be
 the third sponsor. I've reviewed most of the NUMA-related patches from Dan
 and Nikola in the past few months so I'm pretty familiar with the work, as
 well as the difficulties Nikola has run into in the scheduler code regarding
 adding this functionality.

 -jay


 Mikal has said I could approve FFEs in his absence, but given the
 conflict in the thread, I want to leave this approval to him :)

 I know its 10 patches, but I think we can try resolve the discussion
 on the thread, and look to approve this on Monday morning, and still
 make it before the deadline. Do shout up if that seems impossible
 and/or stupid.

 Thanks,
 John

 [1]

 http://lists.openstack.org/pipermail/openstack-dev/2014-September/044669.html

 On 09/04/2014 01:58 PM, Nikola Đipanov wrote:

 Hi team,

 I am requesting the exception for the feature from the subject (find
 specs at [1] and outstanding changes at [2]).

 Some reasons why we may want to grant it:

 First of all all patches have been approved in time and just lost the
 gate race.

 Rejecting it makes little sense really, as it has been commented on by a
 good chunk of the core team, most of the invasive stuff (db migrations
 for example) has already merged, and the few parts that may seem
 contentious have either been discussed and agreed upon [3], or can
 easily be addressed in subsequent bug fixes.

 It would be very beneficial to merge it so that we actually get real
 testing on the feature ASAP (scheduling features are not tested in the
 gate so we need to rely on downstream/3rd party/user testing for those).

 Thanks,

 Nikola

 [1]

 http://git.openstack.org/cgit/openstack/nova-specs/tree/specs/juno/virt-driver-numa-placement.rst
 [2]

 https://review.openstack.org/#/q/status:open+project:openstack/nova+branch:master+topic:bp/virt-driver-numa-placement,n,z
 [3] https://review.openstack.org/#/c/111782/

 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



-- 
Rackspace Australia

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Nova][FFE] Feature freeze exception for virt-driver-numa-placement

2014-09-04 Thread Daniel P. Berrange
On Thu, Sep 04, 2014 at 01:58:58PM +0200, Nikola Đipanov wrote:
 Hi team,
 
 I am requesting the exception for the feature from the subject (find
 specs at [1] and outstanding changes at [2]).
 
 Some reasons why we may want to grant it:
 
 First of all all patches have been approved in time and just lost the
 gate race.
 
 Rejecting it makes little sense really, as it has been commented on by a
 good chunk of the core team, most of the invasive stuff (db migrations
 for example) has already merged, and the few parts that may seem
 contentious have either been discussed and agreed upon [3], or can
 easily be addressed in subsequent bug fixes.
 
 It would be very beneficial to merge it so that we actually get real
 testing on the feature ASAP (scheduling features are not tested in the
 gate so we need to rely on downstream/3rd party/user testing for those).

I think this NUMA work is a very important step forwards for Nova in
general, whch will benefit our entire userbase of KVM deployments,
and be especially useful to the NFV user group's needs.

As such, I'll be one sponsor for the FFE

Regards,
Daniel
-- 
|: http://berrange.com  -o-http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org  -o- http://virt-manager.org :|
|: http://autobuild.org   -o- http://search.cpan.org/~danberr/ :|
|: http://entangle-photo.org   -o-   http://live.gnome.org/gtk-vnc :|

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Nova][FFE] Feature freeze exception for virt-driver-numa-placement

2014-09-04 Thread Sean Dague
On 09/04/2014 07:58 AM, Nikola Đipanov wrote:
 Hi team,
 
 I am requesting the exception for the feature from the subject (find
 specs at [1] and outstanding changes at [2]).
 
 Some reasons why we may want to grant it:
 
 First of all all patches have been approved in time and just lost the
 gate race.
 
 Rejecting it makes little sense really, as it has been commented on by a
 good chunk of the core team, most of the invasive stuff (db migrations
 for example) has already merged, and the few parts that may seem
 contentious have either been discussed and agreed upon [3], or can
 easily be addressed in subsequent bug fixes.
 
 It would be very beneficial to merge it so that we actually get real
 testing on the feature ASAP (scheduling features are not tested in the
 gate so we need to rely on downstream/3rd party/user testing for those).

This statement bugs me. It seems kind of backwards to say we should
merge a thing that we don't have a good upstream test plan on and put it
in a release so that the testing will happen only in the downstream case.

Anyway, not enough to -1 it, but enough to at least say something.

-Sean

-- 
Sean Dague
http://dague.net

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Nova][FFE] Feature freeze exception for virt-driver-numa-placement

2014-09-04 Thread Pádraig Brady
On 09/04/2014 12:58 PM, Nikola Đipanov wrote:
 Hi team,
 
 I am requesting the exception for the feature from the subject (find
 specs at [1] and outstanding changes at [2]).
 
 Some reasons why we may want to grant it:
 
 First of all all patches have been approved in time and just lost the
 gate race.
 
 Rejecting it makes little sense really, as it has been commented on by a
 good chunk of the core team, most of the invasive stuff (db migrations
 for example) has already merged, and the few parts that may seem
 contentious have either been discussed and agreed upon [3], or can
 easily be addressed in subsequent bug fixes.
 
 It would be very beneficial to merge it so that we actually get real
 testing on the feature ASAP (scheduling features are not tested in the
 gate so we need to rely on downstream/3rd party/user testing for those).
 
 Thanks,
 
 Nikola
 
 [1]
 http://git.openstack.org/cgit/openstack/nova-specs/tree/specs/juno/virt-driver-numa-placement.rst
 [2]
 https://review.openstack.org/#/q/status:open+project:openstack/nova+branch:master+topic:bp/virt-driver-numa-placement,n,z
 [3] https://review.openstack.org/#/c/111782/

I'll sponsor this too, and I've already reviewed this set a few times


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Nova][FFE] Feature freeze exception for virt-driver-numa-placement

2014-09-04 Thread Nikola Đipanov
On 09/04/2014 02:31 PM, Sean Dague wrote:
 On 09/04/2014 07:58 AM, Nikola Đipanov wrote:
 Hi team,

 I am requesting the exception for the feature from the subject (find
 specs at [1] and outstanding changes at [2]).

 Some reasons why we may want to grant it:

 First of all all patches have been approved in time and just lost the
 gate race.

 Rejecting it makes little sense really, as it has been commented on by a
 good chunk of the core team, most of the invasive stuff (db migrations
 for example) has already merged, and the few parts that may seem
 contentious have either been discussed and agreed upon [3], or can
 easily be addressed in subsequent bug fixes.

 It would be very beneficial to merge it so that we actually get real
 testing on the feature ASAP (scheduling features are not tested in the
 gate so we need to rely on downstream/3rd party/user testing for those).
 
 This statement bugs me. It seems kind of backwards to say we should
 merge a thing that we don't have a good upstream test plan on and put it
 in a release so that the testing will happen only in the downstream case.
 

The objective reality is that many other things have not had upstream
testing for a long time (anything that requires more than 1 compute node
in Nova for example, and any scheduling feature - as I mention clearly
above), so not sure how that is backwards from any reasonable point.

Thanks to folks using them, it is still kept working and bugs get fixed.
Getting features into the hands of users is extremely important...

 Anyway, not enough to -1 it, but enough to at least say something.
 

.. but I do not want to get into the discussion about software testing
here, not the place really.

However, I do think it is very harmful to respond to FFE request with
such blanket statements and generalizations, if only for the message it
sends to the contributors (that we really care more about upholding our
own myths as a community than users and features).

N.

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Nova][FFE] Feature freeze exception for virt-driver-numa-placement

2014-09-04 Thread Daniel P. Berrange
On Thu, Sep 04, 2014 at 03:07:24PM +0200, Nikola Đipanov wrote:
 On 09/04/2014 02:31 PM, Sean Dague wrote:
  On 09/04/2014 07:58 AM, Nikola Đipanov wrote:
  Hi team,
 
  I am requesting the exception for the feature from the subject (find
  specs at [1] and outstanding changes at [2]).
 
  Some reasons why we may want to grant it:
 
  First of all all patches have been approved in time and just lost the
  gate race.
 
  Rejecting it makes little sense really, as it has been commented on by a
  good chunk of the core team, most of the invasive stuff (db migrations
  for example) has already merged, and the few parts that may seem
  contentious have either been discussed and agreed upon [3], or can
  easily be addressed in subsequent bug fixes.
 
  It would be very beneficial to merge it so that we actually get real
  testing on the feature ASAP (scheduling features are not tested in the
  gate so we need to rely on downstream/3rd party/user testing for those).
  
  This statement bugs me. It seems kind of backwards to say we should
  merge a thing that we don't have a good upstream test plan on and put it
  in a release so that the testing will happen only in the downstream case.
  
 
 The objective reality is that many other things have not had upstream
 testing for a long time (anything that requires more than 1 compute node
 in Nova for example, and any scheduling feature - as I mention clearly
 above), so not sure how that is backwards from any reasonable point.

More critically with NUMA feature, AFAIK, there is no public cloud in
existance which exposes NUMA to the guest. So unless someone is willing
to pay for 100's of bare metal servers to run tempest on, I don't know
of any infrastructure on which we can test NUMA today.

Of course once we include NUMA features in Nova and release Nova, then
the Rackspace and/or HP clouds will be in a position to start considering
how  when they might expose NUMA features for instances they host. So by
including it in Nova today, we would be helping move towards a future
where we will be able to run tempest against NUMA features.

Blocking NUMA from Nova for lack of automated testing will leave us trapped
in a chicken and egg scenario, potentially forever. That's not in anyones
best interests IMHO

Regards,
Daniel
-- 
|: http://berrange.com  -o-http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org  -o- http://virt-manager.org :|
|: http://autobuild.org   -o- http://search.cpan.org/~danberr/ :|
|: http://entangle-photo.org   -o-   http://live.gnome.org/gtk-vnc :|

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Nova][FFE] Feature freeze exception for virt-driver-numa-placement

2014-09-04 Thread Sean Dague
On 09/04/2014 09:21 AM, Daniel P. Berrange wrote:
 On Thu, Sep 04, 2014 at 03:07:24PM +0200, Nikola Đipanov wrote:
 On 09/04/2014 02:31 PM, Sean Dague wrote:
 On 09/04/2014 07:58 AM, Nikola Đipanov wrote:
 Hi team,

 I am requesting the exception for the feature from the subject (find
 specs at [1] and outstanding changes at [2]).

 Some reasons why we may want to grant it:

 First of all all patches have been approved in time and just lost the
 gate race.

 Rejecting it makes little sense really, as it has been commented on by a
 good chunk of the core team, most of the invasive stuff (db migrations
 for example) has already merged, and the few parts that may seem
 contentious have either been discussed and agreed upon [3], or can
 easily be addressed in subsequent bug fixes.

 It would be very beneficial to merge it so that we actually get real
 testing on the feature ASAP (scheduling features are not tested in the
 gate so we need to rely on downstream/3rd party/user testing for those).

 This statement bugs me. It seems kind of backwards to say we should
 merge a thing that we don't have a good upstream test plan on and put it
 in a release so that the testing will happen only in the downstream case.


 The objective reality is that many other things have not had upstream
 testing for a long time (anything that requires more than 1 compute node
 in Nova for example, and any scheduling feature - as I mention clearly
 above), so not sure how that is backwards from any reasonable point.
 
 More critically with NUMA feature, AFAIK, there is no public cloud in
 existance which exposes NUMA to the guest. So unless someone is willing
 to pay for 100's of bare metal servers to run tempest on, I don't know
 of any infrastructure on which we can test NUMA today.
 
 Of course once we include NUMA features in Nova and release Nova, then
 the Rackspace and/or HP clouds will be in a position to start considering
 how  when they might expose NUMA features for instances they host. So by
 including it in Nova today, we would be helping move towards a future
 where we will be able to run tempest against NUMA features.
 
 Blocking NUMA from Nova for lack of automated testing will leave us trapped
 in a chicken and egg scenario, potentially forever. That's not in anyones
 best interests IMHO

The spec specifically calls out the scheduler piece being the part that
probably most needs to be tested, especially at large scales here. Those
pieces don't need Tempest to test them, they need more solid functional
tests around the scheduler under those circumstances.

There are interesting (and not all that difficult) ways to do this given
the resources we have, which don't seem to be being explored, which is
my concern.

-Sean

-- 
Sean Dague
http://dague.net

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Nova][FFE] Feature freeze exception for virt-driver-numa-placement

2014-09-04 Thread Gary Kotton


On 9/4/14, 4:30 PM, Sean Dague s...@dague.net wrote:

On 09/04/2014 09:21 AM, Daniel P. Berrange wrote:
 On Thu, Sep 04, 2014 at 03:07:24PM +0200, Nikola Đipanov wrote:
 On 09/04/2014 02:31 PM, Sean Dague wrote:
 On 09/04/2014 07:58 AM, Nikola Đipanov wrote:
 Hi team,

 I am requesting the exception for the feature from the subject (find
 specs at [1] and outstanding changes at [2]).

 Some reasons why we may want to grant it:

 First of all all patches have been approved in time and just lost the
 gate race.

 Rejecting it makes little sense really, as it has been commented on
by a
 good chunk of the core team, most of the invasive stuff (db
migrations
 for example) has already merged, and the few parts that may seem
 contentious have either been discussed and agreed upon [3], or can
 easily be addressed in subsequent bug fixes.

 It would be very beneficial to merge it so that we actually get real
 testing on the feature ASAP (scheduling features are not tested in
the
 gate so we need to rely on downstream/3rd party/user testing for
those).

 This statement bugs me. It seems kind of backwards to say we should
 merge a thing that we don't have a good upstream test plan on and put
it
 in a release so that the testing will happen only in the downstream
case.


 The objective reality is that many other things have not had upstream
 testing for a long time (anything that requires more than 1 compute
node
 in Nova for example, and any scheduling feature - as I mention clearly
 above), so not sure how that is backwards from any reasonable point.
 
 More critically with NUMA feature, AFAIK, there is no public cloud in
 existance which exposes NUMA to the guest. So unless someone is willing
 to pay for 100's of bare metal servers to run tempest on, I don't know
 of any infrastructure on which we can test NUMA today.
 
 Of course once we include NUMA features in Nova and release Nova, then
 the Rackspace and/or HP clouds will be in a position to start
considering
 how  when they might expose NUMA features for instances they host. So
by
 including it in Nova today, we would be helping move towards a future
 where we will be able to run tempest against NUMA features.
 
 Blocking NUMA from Nova for lack of automated testing will leave us
trapped
 in a chicken and egg scenario, potentially forever. That's not in
anyones
 best interests IMHO

The spec specifically calls out the scheduler piece being the part that
probably most needs to be tested, especially at large scales here. Those
pieces don't need Tempest to test them, they need more solid functional
tests around the scheduler under those circumstances.

There are interesting (and not all that difficult) ways to do this given
the resources we have, which don't seem to be being explored, which is
my concern.

I share your concern with this feature. I stated it on review
https://review.openstack.org/#/c/115007/ in PS 16. I think that we have
well known scheduling issues and these will be accentuated by a feature
like this. My feeling is that this feature and the PCI feature are both
going to be problematic under scale.

My reservations are when the feature is not enabled that a lot of
unnecessary data will be passed between hosts and the scheduler (this is
why we should have gone with the extensible resources (but that is opening
a can of worms)).

Having said that I think that Nova needs features like this. I am in favor
of moving ahead with this for a number of reasons:
1. The filter is not enabled by default
2. We can fix things moving forwards

So I am +1 on this. If we can document that it is experimental or use at
your own risk then I am +2. But I think that the fact that the admin needs
to configure the filter she/he knows it is at their own risk.

A luta continua



   -Sean

-- 
Sean Dague
https://urldefense.proofpoint.com/v1/url?u=http://dague.net/k=oIvRg1%2BdG
AgOoM1BIlLLqw%3D%3D%0Ar=eH0pxTUZo8NPZyF6hgoMQu%2BfDtysg45MkPhCZFxPEq8%3D%
0Am=Vr9ci4W1jJwlMVh7NJWsxGeY52I2AJ113JDTFO2CluA%3D%0As=45070dc04c1c3bb93
93b6273d23a8310ea404b716cf40c299b487e24ba5a8552

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Nova][FFE] Feature freeze exception for virt-driver-numa-placement

2014-09-04 Thread Murray, Paul (HP Cloud)

On 4 September 2014 14:07, Nikola Đipanov ndipa...@redhat.com wrote:
On 09/04/2014 02:31 PM, Sean Dague wrote:
 On 09/04/2014 07:58 AM, Nikola Đipanov wrote:
 Hi team,

 I am requesting the exception for the feature from the subject (find
 specs at [1] and outstanding changes at [2]).

 Some reasons why we may want to grant it:

 First of all all patches have been approved in time and just lost the
 gate race.

 Rejecting it makes little sense really, as it has been commented on by a
 good chunk of the core team, most of the invasive stuff (db migrations
 for example) has already merged, and the few parts that may seem
 contentious have either been discussed and agreed upon [3], or can
 easily be addressed in subsequent bug fixes.

 It would be very beneficial to merge it so that we actually get real
 testing on the feature ASAP (scheduling features are not tested in the
 gate so we need to rely on downstream/3rd party/user testing for those).

 This statement bugs me. It seems kind of backwards to say we should
 merge a thing that we don't have a good upstream test plan on and put it
 in a release so that the testing will happen only in the downstream case.


The objective reality is that many other things have not had upstream
testing for a long time (anything that requires more than 1 compute node
in Nova for example, and any scheduling feature - as I mention clearly
above), so not sure how that is backwards from any reasonable point.

Thanks to folks using them, it is still kept working and bugs get fixed.
Getting features into the hands of users is extremely important...

 Anyway, not enough to -1 it, but enough to at least say something.


.. but I do not want to get into the discussion about software testing
here, not the place really.

However, I do think it is very harmful to respond to FFE request with
such blanket statements and generalizations, if only for the message it
sends to the contributors (that we really care more about upholding our
own myths as a community than users and features).


I believe you brought this up as one of your justifications for the FFE. When I 
read your statement it does sound as though you want to put experimental code 
in at the final release. I am sure that is not what you had in mind, but I am 
also sure you can also understand Sean's point of view. His point is clear and 
pertinent to your request.

As the person responsible for Nova in HP I will be interested to see how it 
operates in practice. I can assure you we will do extensive testing on it 
before it goes into the wild and we will not put it into practice if we are not 
happy.

Paul

Paul Murray
Nova Technical Lead, HP Cloud
+44 117 312 9309

Hewlett-Packard Limited registered Office: Cain Road, Bracknell, Berks RG12 1HN 
Registered No: 690597 England. The contents of this message and any attachments 
to it are confidential and may be legally privileged. If you have received this 
message in error, you should delete it from your system immediately and advise 
the sender. To any recipient of this message within HP, unless otherwise stated 
you should consider this message and attachments as HP CONFIDENTIAL.

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Nova][FFE] Feature freeze exception for virt-driver-numa-placement

2014-09-04 Thread Nikola Đipanov
On 09/04/2014 04:51 PM, Murray, Paul (HP Cloud) wrote:
  
 
 On 4 September 2014 14:07, Nikola Đipanov ndipa...@redhat.com wrote:
 
 On 09/04/2014 02:31 PM, Sean Dague wrote:
 
 On 09/04/2014 07:58 AM, Nikola Đipanov wrote:
 
 Hi team,
 
 
 
 I am requesting the exception for the feature from the subject (find
 
 specs at [1] and outstanding changes at [2]).
 
 
 
 Some reasons why we may want to grant it:
 
 
 
 First of all all patches have been approved in time and just lost the
 
 gate race.
 
 
 
 Rejecting it makes little sense really, as it has been commented on by a
 
 good chunk of the core team, most of the invasive stuff (db migrations
 
 for example) has already merged, and the few parts that may seem
 
 contentious have either been discussed and agreed upon [3], or can
 
 easily be addressed in subsequent bug fixes.
 
 
 
 It would be very beneficial to merge it so that we actually get real
 
 testing on the feature ASAP (scheduling features are not tested in the
 
 gate so we need to rely on downstream/3rd party/user testing for those).
 
 
 
 This statement bugs me. It seems kind of backwards to say we should
 
 merge a thing that we don't have a good upstream test plan on and put it
 
 in a release so that the testing will happen only in the downstream case.
 
 
 
  
 
 The objective reality is that many other things have not had upstream
 
 testing for a long time (anything that requires more than 1 compute node
 
 in Nova for example, and any scheduling feature - as I mention clearly
 
 above), so not sure how that is backwards from any reasonable point.
 
  
 
 Thanks to folks using them, it is still kept working and bugs get fixed.
 
 Getting features into the hands of users is extremely important...
 
  
 
 Anyway, not enough to -1 it, but enough to at least say something.
 
 
 
  
 
 .. but I do not want to get into the discussion about software testing
 
 here, not the place really.
 
  
 
 However, I do think it is very harmful to respond to FFE request with
 
 such blanket statements and generalizations, if only for the message it
 
 sends to the contributors (that we really care more about upholding our
 
 own myths as a community than users and features).
 
  
 
  
 
 I believe you brought this up as one of your justifications for the FFE.
 When I read your statement it does sound as though you want to put
 experimental code in at the final release. I am sure that is not what
 you had in mind, but I am also sure you can also understand Sean's point
 of view. His point is clear and pertinent to your request.
 
  
 
 As the person responsible for Nova in HP I will be interested to see how
 it operates in practice. I can assure you we will do extensive testing
 on it before it goes into the wild and we will not put it into practice
 if we are not happy.
 

That is awesome and we as a project are lucky to have that! I would not
want things put into practice that users can't use or see huge flaws with.

I can't help but read this as you being OK with the feature going ahead,
though :).

N.

  
 
 Paul
 
  
 
 Paul Murray
 
 Nova Technical Lead, HP Cloud
 
 +44 117 312 9309
 
 Hewlett-Packard Limited registered Office: Cain Road, Bracknell, Berks
 RG12 1HN Registered No: 690597 England. The contents of this message and
 any attachments to it are confidential and may be legally privileged. If
 you have received this message in error, you should delete it from your
 system immediately and advise the sender. To any recipient of this
 message within HP, unless otherwise stated you should consider this
 message and attachments as HP CONFIDENTIAL.
 
  
 
 
 
 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
 


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Nova][FFE] Feature freeze exception for virt-driver-numa-placement

2014-09-04 Thread Czesnowicz, Przemyslaw


 -Original Message-
 From: Nikola Đipanov [mailto:ndipa...@redhat.com]
 Sent: Thursday, September 04, 2014 4:22 PM
 To: openstack-dev@lists.openstack.org
 Subject: Re: [openstack-dev] [Nova][FFE] Feature freeze exception for virt-
 driver-numa-placement
 
 On 09/04/2014 04:51 PM, Murray, Paul (HP Cloud) wrote:
 
 
  On 4 September 2014 14:07, Nikola Đipanov ndipa...@redhat.com wrote:
 
  On 09/04/2014 02:31 PM, Sean Dague wrote:
 
  On 09/04/2014 07:58 AM, Nikola Đipanov wrote:
 
  Hi team,
 
 
 
  I am requesting the exception for the feature from the subject (find
 
  specs at [1] and outstanding changes at [2]).
 
 
 
  Some reasons why we may want to grant it:
 
 
 
  First of all all patches have been approved in time and just lost
  the
 
  gate race.
 
 
 
  Rejecting it makes little sense really, as it has been commented on
  by a
 
  good chunk of the core team, most of the invasive stuff (db
  migrations
 
  for example) has already merged, and the few parts that may seem
 
  contentious have either been discussed and agreed upon [3], or can
 
  easily be addressed in subsequent bug fixes.
 
 
 
  It would be very beneficial to merge it so that we actually get real
 
  testing on the feature ASAP (scheduling features are not tested in
  the
 
  gate so we need to rely on downstream/3rd party/user testing for those).
 
 
 
  This statement bugs me. It seems kind of backwards to say we should
 
  merge a thing that we don't have a good upstream test plan on and put
  it
 
  in a release so that the testing will happen only in the downstream case.
 
 
 
 
 
  The objective reality is that many other things have not had upstream
 
  testing for a long time (anything that requires more than 1 compute
  node
 
  in Nova for example, and any scheduling feature - as I mention clearly
 
  above), so not sure how that is backwards from any reasonable point.
 
 
 
  Thanks to folks using them, it is still kept working and bugs get fixed.
 
  Getting features into the hands of users is extremely important...
 
 
 
  Anyway, not enough to -1 it, but enough to at least say something.
 
 
 
 
 
  .. but I do not want to get into the discussion about software testing
 
  here, not the place really.
 
 
 
  However, I do think it is very harmful to respond to FFE request with
 
  such blanket statements and generalizations, if only for the message
  it
 
  sends to the contributors (that we really care more about upholding
  our
 
  own myths as a community than users and features).
 
 
 
 
 
  I believe you brought this up as one of your justifications for the FFE.
  When I read your statement it does sound as though you want to put
  experimental code in at the final release. I am sure that is not what
  you had in mind, but I am also sure you can also understand Sean's
  point of view. His point is clear and pertinent to your request.
 
 
 
  As the person responsible for Nova in HP I will be interested to see
  how it operates in practice. I can assure you we will do extensive
  testing on it before it goes into the wild and we will not put it into
  practice if we are not happy.
 
 
 That is awesome and we as a project are lucky to have that! I would not want
 things put into practice that users can't use or see huge flaws with.
 
 I can't help but read this as you being OK with the feature going ahead, 
 though
 :).
 
 N.

We (Intel) are as well very interested in this feature and would like to see it 
land in Juno.  
If FFE is granted we will test this feature before release and work with Nikola 
on fixing bugs if any are found.

Regards
Przemek

 
 
 
  Paul
 
 
 
  Paul Murray
 
  Nova Technical Lead, HP Cloud
 
  +44 117 312 9309
 
  Hewlett-Packard Limited registered Office: Cain Road, Bracknell, Berks
  RG12 1HN Registered No: 690597 England. The contents of this message
  and any attachments to it are confidential and may be legally
  privileged. If you have received this message in error, you should
  delete it from your system immediately and advise the sender. To any
  recipient of this message within HP, unless otherwise stated you
  should consider this message and attachments as HP CONFIDENTIAL.
 
 
 
 
 
  ___
  OpenStack-dev mailing list
  OpenStack-dev@lists.openstack.org
  http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
 
 
 
 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Nova][FFE] Feature freeze exception for virt-driver-numa-placement

2014-09-04 Thread Murray, Paul (HP Cloud)


 Anyway, not enough to -1 it, but enough to at least say something.





 .. but I do not want to get into the discussion about software testing

 here, not the place really.



 However, I do think it is very harmful to respond to FFE request with

 such blanket statements and generalizations, if only for the message it

 sends to the contributors (that we really care more about upholding our

 own myths as a community than users and features).





 I believe you brought this up as one of your justifications for the FFE.
 When I read your statement it does sound as though you want to put
 experimental code in at the final release. I am sure that is not what
 you had in mind, but I am also sure you can also understand Sean's point
 of view. His point is clear and pertinent to your request.



 As the person responsible for Nova in HP I will be interested to see how
 it operates in practice. I can assure you we will do extensive testing
 on it before it goes into the wild and we will not put it into practice
 if we are not happy.


That is awesome and we as a project are lucky to have that! I would not
want things put into practice that users can't use or see huge flaws with.

I can't help but read this as you being OK with the feature going ahead,
though :).


Actually, let's say I have no particular objection. Just thought Sean's point 
is worth noting.

Now, if this had been done as an extensible resource I could easily decouple 
deploying it from all the bug fixes that come through with the release. But 
that's another matter...

Paul
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev