Re: [openstack-dev] [nova] [ironic] Input types for scheduler filters

2016-07-11 Thread Miles Gould

On 08/07/16 15:22, Miles Gould wrote:

On 07/07/16 17:43, Miles Gould wrote:

Further evidence that this isn't the intended behaviour: if you remove
all the calls to str(), then the original tests still pass, but the
' e' (substring matching) one doesn't.


I've now proposed this as a patch:
https://review.openstack.org/#/c/339576/ Please review!


Status update on this: Ruby Loo found a place in nova where the 
thing-being-matched is cast to a string before matching:


https://github.com/openstack/nova/blob/90ec46c87fbf572805c7758377431e26c93622a4/nova/scheduler/filters/compute_capabilities_filter.py#L87

This means  will match substrings and not subsets; we talked 
about this in the nova-scheduler meeting and agreed it's a bug. I'll 
submit a patch to fix it in Nova.


Alexis Lee has submitted a patch to the oslo.utils version to enforce 
the type of the value being matched:


https://review.openstack.org/#/c/339596/

There's some discussion about whether this is the right approach, but 
the Oslo cores have made clear that without some type-enforcement the 
code won't be merged into Oslo.


If the matcher code can't be merged into Oslo, we may copy it directly 
into ironic-lib until it can be; understandably, there's some resistance 
to this idea.


https://review.openstack.org/#/c/334431/

Reviews on all the above would be much appreciated!

Miles

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [nova] [ironic] Input types for scheduler filters

2016-07-08 Thread Miles Gould

On 07/07/16 17:43, Miles Gould wrote:

Further evidence that this isn't the intended behaviour: if you remove
all the calls to str(), then the original tests still pass, but the
' e' (substring matching) one doesn't.


I've now proposed this as a patch: 
https://review.openstack.org/#/c/339576/ Please review!


Miles

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


[openstack-dev] [nova] [ironic] Input types for scheduler filters

2016-07-07 Thread Miles Gould

Hi everyone,

tl;dr: the tests for the  operator in 
nova.scheduler.filters.extra_specs_ops do not test what it looks like 
they're meant to test. This is confusing us, and holding up work in 
Ironic. Does it match its arguments against a list of strings, or 
against a single string?


--

Over in Ironic, we need a more flexible language for specifying 
root-device hints, and we thought it would be a Good Thing to adopt the 
scheduler filter language used in Nova. There's already a review in 
progress to move that code into oslo.utils 
(https://review.openstack.org/#/c/308398) and another to clean it up 
with a well-defined formal grammar 
(https://review.openstack.org/#/c/313699/), so this ought to be an easy 
win. But! We're not sure that we've correctly understood the semantics 
of the language, which in turn means we can't tell if it's suitable for 
our use.


Here are two representative tests for the  operator:

def test_extra_specs_matches_one_with_op_allin(self):
values = ['aes', 'mmx', 'aux']
self._do_extra_specs_ops_test(
value=str(values),
req=' aes mmx',
matches=True)

def test_extra_specs_fails_with_op_allin(self):
values = ['aes', 'mmx', 'aux']
self._do_extra_specs_ops_test(
value=str(values),
req='  txt',
matches=False)

So it looks like  takes a list of strings, and matches against a 
`value` which is also a list of strings, and returns True iff every 
argument is in `value`. But look closer. Those tests actually check 
matching against str(values), which is the literal string "['aes', 
'mmx', 'aux']". This is also a valid Python collection, to which the 
Python `in` operator applies just fine, so what these tests actually 
check is if every argument string is a *substring* of str(values). This 
means that the following test passes:


def test_extra_specs_matches_all_with_op_allin(self):
values = ['aes', 'mmx', 'aux']
self._do_extra_specs_ops_test(
value=str(values),
req=' e',
matches=True)

[Don't believe me? See https://review.openstack.org/#/c/336094]

Is this the intended behaviour? When this is called as part of a real 
scheduler filter, is the list of values stringified before matching like 
this?


Further evidence that this isn't the intended behaviour: if you remove 
all the calls to str(), then the original tests still pass, but the 
' e' (substring matching) one doesn't.


So, is  meant to be doing substring matching? Or are the tests 
in error?


Thanks!
Miles

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev