Re: [openstack-dev] [nova] jsonschema for scheduler hints

[Sean Dague]

On 12/08/2015 11:47 PM, Ken'ichi Ohmichi wrote:
> Hi Sylvain,
> 
> 2015-12-04 17:48 GMT+09:00 Sylvain Bauza :
>>>
>>> That leaves the out-of-tree discussion about custom filters and how we
>>> could have a consistent behaviour given that. Should we accept something in
>>> a specific deployment while another deployment could 401 against it ? Mmm,
>>> bad to me IMHO.
>>
>> We can have code to check the out-of-tree filters didn't expose any same
>> hints with in-tree filter.
>>
>> Sure, and thank you for that, that was missing in the past. That said, there
>> are still some interoperability concerns, let me explain : as a cloud
>> operator, I'm now providing a custom filter (say MyAwesomeFilter) which does
>> the lookup for an hint called 'my_awesome_hint'.
>>
>> If we enforce a strict validation (and not allow to accept any hint) it
>> would mean that this cloud would accept a request with 'my_awesome_hint'
>> while another cloud which wouldn't be running MyAwesomeFilter would then
>> deny the same request.
> 
> I am thinking the operator/vendor own filter should have some
> implementation code for registering its original hint to jsonschema to
> expose/validate available hints in the future.
> The way should be easy as possible so that they can implement the code easily.
> After that, we will be able to make the validation strict again.

Yeh, that was my thinking. As someone that did a lot of the jsonschema
work, is that something you could prototype?

-Sean

-- 
Sean Dague
http://dague.net

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [nova] jsonschema for scheduler hints

[Ken'ichi Ohmichi]

2015-12-09 21:20 GMT+09:00 Sean Dague :
> On 12/08/2015 11:47 PM, Ken'ichi Ohmichi wrote:
>> Hi Sylvain,
>>
>> 2015-12-04 17:48 GMT+09:00 Sylvain Bauza :

 That leaves the out-of-tree discussion about custom filters and how we
 could have a consistent behaviour given that. Should we accept something in
 a specific deployment while another deployment could 401 against it ? Mmm,
 bad to me IMHO.
>>>
>>> We can have code to check the out-of-tree filters didn't expose any same
>>> hints with in-tree filter.
>>>
>>> Sure, and thank you for that, that was missing in the past. That said, there
>>> are still some interoperability concerns, let me explain : as a cloud
>>> operator, I'm now providing a custom filter (say MyAwesomeFilter) which does
>>> the lookup for an hint called 'my_awesome_hint'.
>>>
>>> If we enforce a strict validation (and not allow to accept any hint) it
>>> would mean that this cloud would accept a request with 'my_awesome_hint'
>>> while another cloud which wouldn't be running MyAwesomeFilter would then
>>> deny the same request.
>>
>> I am thinking the operator/vendor own filter should have some
>> implementation code for registering its original hint to jsonschema to
>> expose/validate available hints in the future.
>> The way should be easy as possible so that they can implement the code 
>> easily.
>> After that, we will be able to make the validation strict again.
>
> Yeh, that was my thinking. As someone that did a lot of the jsonschema
> work, is that something you could prototype?

Yes.
On a prototype https://review.openstack.org/#/c/220440/ , each filter
needs to contain get_scheduler_hint_api_schema() which returns
available scheduler_hints parameter. Then stevedore detects these
parameters from each filter and extends jsonschema with them.
On current prototype, the detection and extension are implemented in nova-api.
but we need to change the prototype like:

  1. nova-sched detects available scheduler-hints from filters.
  2. nova-sched passes these scheduler-hints to nova-api via RPC.
  3. nova-api extends jsonschema with the gotten scheduler-hints.

After implementing the mechanism, the operator/vendor own filters just
need to implement get_scheduler_hint_api_schema(). That is not so
hard, I feel.

Thanks
Ken Ohmichi

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [nova] jsonschema for scheduler hints

[Andrew Laski]

On 12/06/15 at 02:49pm, Ken'ichi Ohmichi wrote:

Hi Sean,

2015-12-02 23:23 GMT+09:00 Sean Dague :

We have previously agreed that scheduler hints in Nova are an open ended
thing. It's expected for sites to have additional scheduler filters
which expose new hints. The way we handle that with our strict
jsonschema is that we allow additional properties -
https://github.com/openstack/nova/blob/1734ce7101982dd95f8fab1ab4815bd258a33744/nova/api/openstack/compute/schemas/scheduler_hints.py#L65

This means that if you specify some garbage hint, you don't get feedback
that it was garbage in your environment. That lost a couple of days
building multinode tests in the gate. Having gotten used to the hints
that "you've given us bad stuff", this was a stark change back to the
old world.

Would it be possible to make it so that the schema could be explicitly
extended (instead of implicitly extended). So that additional
properties=False, but a mechanism for a scheduler filter to register
it's jsonschema in?


We had a prototype for registering scheduler_hints to jsonschema from
available filters:

https://review.openstack.org/#/c/220440/

That was against current basic design of nova and abandoned.
We need more time for finding right implementation way for that.

BTW, I'd like to get feedback about scheduler_hints.
As the above jsonschema, nova just ignores unavailable scheduler_hint
if a client passes it.
So the client cannot know the specified scheduler_hint works or not
due to no feedback.
However, I feel that doesn't seem so bad because scheduler_hint is
just "hint" not "rule".
Nova can say "I consider this hint as possible, but sometimes ignore it".
In addition, Nova's purpose is for making physical environments abstract.
So it is not so bad to ignore the hint sometime.
Or should Nova handle scheduler_hint strictly?

I guess that depends on use cases.
So if needing to handle scheduler_hint strictly, we need the above
kind of mechanism for registering available hints to jsonschema.


Here's how it plays out in my mind:

There's been a lot of work put into isolating the scheduler within Nova, 
with a potential end goal of splitting it out.  With this in mind there 
are a couple of options.  Scheduler filters that accept hints could have 
an additional Nova component that adds in API schema information.  This 
is not a desirable approach IMO.  Or the Nova API could query the 
scheduler for a schema on startup and use that for validation.  This 
keeps scheduling extensibility completely within the scheduler.


The second part of this is that the schema only really tells you that 
you've used the correct hint name and format and nothing more.  There is 
no feedback on if the hint was used, and its presence in the schema 
does not mean the appropriate filter to use it is enabled.  Moving 
towards having filters register hints when enabled would allow for the 
schema to expose scheduling capabilities.  But that also means that 
there's no guarantee that in in-tree hint like "same_host" would be 
honored in every cloud.  And right now no way to query the schema to get 
the capabilities exposed.


So the right path to me would be for the scheduler to be able to 
construct a validation schema based on which filters are enabled, and 
then expose an RPC(and later HTTP) interface for getting that schema.  
Then nova-api could pull that schema at startup, and have a mechanism to 
re-acquire and load it.




Thanks
Ken Ohmichi

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [nova] jsonschema for scheduler hints

[Sylvain Bauza]

Le 08/12/2015 21:23, Andrew Laski a écrit :

On 12/06/15 at 02:49pm, Ken'ichi Ohmichi wrote:

Hi Sean,

2015-12-02 23:23 GMT+09:00 Sean Dague :
We have previously agreed that scheduler hints in Nova are an open 
ended

thing. It's expected for sites to have additional scheduler filters
which expose new hints. The way we handle that with our strict
jsonschema is that we allow additional properties -
https://github.com/openstack/nova/blob/1734ce7101982dd95f8fab1ab4815bd258a33744/nova/api/openstack/compute/schemas/scheduler_hints.py#L65 



This means that if you specify some garbage hint, you don't get 
feedback

that it was garbage in your environment. That lost a couple of days
building multinode tests in the gate. Having gotten used to the hints
that "you've given us bad stuff", this was a stark change back to the
old world.

Would it be possible to make it so that the schema could be explicitly
extended (instead of implicitly extended). So that additional
properties=False, but a mechanism for a scheduler filter to register
it's jsonschema in?


We had a prototype for registering scheduler_hints to jsonschema from
available filters:

https://review.openstack.org/#/c/220440/

That was against current basic design of nova and abandoned.
We need more time for finding right implementation way for that.

BTW, I'd like to get feedback about scheduler_hints.
As the above jsonschema, nova just ignores unavailable scheduler_hint
if a client passes it.
So the client cannot know the specified scheduler_hint works or not
due to no feedback.
However, I feel that doesn't seem so bad because scheduler_hint is
just "hint" not "rule".
Nova can say "I consider this hint as possible, but sometimes ignore 
it".
In addition, Nova's purpose is for making physical environments 
abstract.

So it is not so bad to ignore the hint sometime.
Or should Nova handle scheduler_hint strictly?

I guess that depends on use cases.
So if needing to handle scheduler_hint strictly, we need the above
kind of mechanism for registering available hints to jsonschema.


Here's how it plays out in my mind:

There's been a lot of work put into isolating the scheduler within 
Nova, with a potential end goal of splitting it out.  With this in 
mind there are a couple of options.  Scheduler filters that accept 
hints could have an additional Nova component that adds in API schema 
information.  This is not a desirable approach IMO.  Or the Nova API 
could query the scheduler for a schema on startup and use that for 
validation.  This keeps scheduling extensibility completely within the 
scheduler.


The second part of this is that the schema only really tells you that 
you've used the correct hint name and format and nothing more.  There 
is no feedback on if the hint was used, and its presence in the schema 
does not mean the appropriate filter to use it is enabled.  Moving 
towards having filters register hints when enabled would allow for the 
schema to expose scheduling capabilities.  But that also means that 
there's no guarantee that in in-tree hint like "same_host" would be 
honored in every cloud. And right now no way to query the schema to 
get the capabilities exposed.


So the right path to me would be for the scheduler to be able to 
construct a validation schema based on which filters are enabled, and 
then expose an RPC(and later HTTP) interface for getting that schema.  
Then nova-api could pull that schema at startup, and have a mechanism 
to re-acquire and load it.




I'm fine with a new RPC call for getting the schema, but as I explained, 
I think the schema should be accepting all the in-tree filter hints (ie. 
nova.scheduler.filters.all_filters method), not only those which are 
enabled (ie. scheduler_default_filters conf opt). Not sure if that's 
also what you said, I could have been misunderstanding.


Also, I'm not opiniated about out-of-tree filters. We could just provide 
the schema for scheduler_enabled_filters conf opt which would then 
accept custom hints, but I wonder if it would get an interop problem 
like I said.


-Sylvain




Thanks
Ken Ohmichi

__ 


OpenStack Development Mailing List (not for usage questions)
Unsubscribe: 
openstack-dev-requ...@lists.openstack.org?subject:unsubscribe

http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


__ 


OpenStack Development Mailing List (not for usage questions)
Unsubscribe: 
openstack-dev-requ...@lists.openstack.org?subject:unsubscribe

http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [nova] jsonschema for scheduler hints

[Andrew Laski]

On 12/08/15 at 09:43pm, Sylvain Bauza wrote:



Le 08/12/2015 21:23, Andrew Laski a écrit :

On 12/06/15 at 02:49pm, Ken'ichi Ohmichi wrote:

Hi Sean,

2015-12-02 23:23 GMT+09:00 Sean Dague :
We have previously agreed that scheduler hints in Nova are an 
open ended

thing. It's expected for sites to have additional scheduler filters
which expose new hints. The way we handle that with our strict
jsonschema is that we allow additional properties -
https://github.com/openstack/nova/blob/1734ce7101982dd95f8fab1ab4815bd258a33744/nova/api/openstack/compute/schemas/scheduler_hints.py#L65


This means that if you specify some garbage hint, you don't get 
feedback

that it was garbage in your environment. That lost a couple of days
building multinode tests in the gate. Having gotten used to the hints
that "you've given us bad stuff", this was a stark change back to the
old world.

Would it be possible to make it so that the schema could be explicitly
extended (instead of implicitly extended). So that additional
properties=False, but a mechanism for a scheduler filter to register
it's jsonschema in?


We had a prototype for registering scheduler_hints to jsonschema from
available filters:

https://review.openstack.org/#/c/220440/

That was against current basic design of nova and abandoned.
We need more time for finding right implementation way for that.

BTW, I'd like to get feedback about scheduler_hints.
As the above jsonschema, nova just ignores unavailable scheduler_hint
if a client passes it.
So the client cannot know the specified scheduler_hint works or not
due to no feedback.
However, I feel that doesn't seem so bad because scheduler_hint is
just "hint" not "rule".
Nova can say "I consider this hint as possible, but sometimes 
ignore it".
In addition, Nova's purpose is for making physical environments 
abstract.

So it is not so bad to ignore the hint sometime.
Or should Nova handle scheduler_hint strictly?

I guess that depends on use cases.
So if needing to handle scheduler_hint strictly, we need the above
kind of mechanism for registering available hints to jsonschema.


Here's how it plays out in my mind:

There's been a lot of work put into isolating the scheduler within 
Nova, with a potential end goal of splitting it out.  With this in 
mind there are a couple of options.  Scheduler filters that accept 
hints could have an additional Nova component that adds in API 
schema information.  This is not a desirable approach IMO.  Or the 
Nova API could query the scheduler for a schema on startup and use 
that for validation.  This keeps scheduling extensibility 
completely within the scheduler.


The second part of this is that the schema only really tells you 
that you've used the correct hint name and format and nothing more.  
There is no feedback on if the hint was used, and its presence in 
the schema does not mean the appropriate filter to use it is 
enabled.  Moving towards having filters register hints when enabled 
would allow for the schema to expose scheduling capabilities.  But 
that also means that there's no guarantee that in in-tree hint like 
"same_host" would be honored in every cloud. And right now no way 
to query the schema to get the capabilities exposed.


So the right path to me would be for the scheduler to be able to 
construct a validation schema based on which filters are enabled, 
and then expose an RPC(and later HTTP) interface for getting that 
schema.  Then nova-api could pull that schema at startup, and have 
a mechanism to re-acquire and load it.




I'm fine with a new RPC call for getting the schema, but as I 
explained, I think the schema should be accepting all the in-tree 
filter hints (ie. nova.scheduler.filters.all_filters method), not 
only those which are enabled (ie. scheduler_default_filters conf 
opt). Not sure if that's also what you said, I could have been 
misunderstanding.


I really didn't take a stance there, though I was leading towards only 
exposing hints for enabled filters because it moves towards exposing 
scheduling capabilities.  But just because it could be done that way 
doesn't mean it should be.  The validation schema and eventual exposure 
of capabilities could be done through two different mechanisms.




Also, I'm not opiniated about out-of-tree filters. We could just 
provide the schema for scheduler_enabled_filters conf opt which would 
then accept custom hints, but I wonder if it would get an interop 
problem like I said.


-Sylvain




Thanks
Ken Ohmichi

__

OpenStack Development Mailing List (not for usage questions)
Unsubscribe: 
openstack-dev-requ...@lists.openstack.org?subject:unsubscribe

http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


__

OpenStack Development Mailing List (not for usage questions)
Unsubscribe: 

Re: [openstack-dev] [nova] jsonschema for scheduler hints

[Ken'ichi Ohmichi]

2015-12-09 6:02 GMT+09:00 Andrew Laski :
>>
>> I'm fine with a new RPC call for getting the schema, but as I explained, I
>> think the schema should be accepting all the in-tree filter hints (ie.
>> nova.scheduler.filters.all_filters method), not only those which are enabled
>> (ie. scheduler_default_filters conf opt). Not sure if that's also what you
>> said, I could have been misunderstanding.
>
> I really didn't take a stance there, though I was leading towards only
> exposing hints for enabled filters because it moves towards exposing
> scheduling capabilities.  But just because it could be done that way doesn't
> mean it should be.  The validation schema and eventual exposure of
> capabilities could be done through two different mechanisms.

I am imaging swagger[1] will be a mechanisms for exposing the
capabilities in the future.
Swagger will provide available API parameters which contain
scheduler_hints also with json-schema format.
In addition, some servers(Kubernetes, etc.) provide swagger data via REST API.
So if we can add available scheduler_hints to json-schema and
implement swagger, users can know available scheduler_hints via REST
API.

Thanks
Ken Ohmichi

---
[1]: https://review.openstack.org/#/c/243365/ ,
https://review.openstack.org/#/c/233446/

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [nova] jsonschema for scheduler hints

[Ken'ichi Ohmichi]

Hi Sylvain,

2015-12-04 17:48 GMT+09:00 Sylvain Bauza :
>>
>> That leaves the out-of-tree discussion about custom filters and how we
>> could have a consistent behaviour given that. Should we accept something in
>> a specific deployment while another deployment could 401 against it ? Mmm,
>> bad to me IMHO.
>
> We can have code to check the out-of-tree filters didn't expose any same
> hints with in-tree filter.
>
> Sure, and thank you for that, that was missing in the past. That said, there
> are still some interoperability concerns, let me explain : as a cloud
> operator, I'm now providing a custom filter (say MyAwesomeFilter) which does
> the lookup for an hint called 'my_awesome_hint'.
>
> If we enforce a strict validation (and not allow to accept any hint) it
> would mean that this cloud would accept a request with 'my_awesome_hint'
> while another cloud which wouldn't be running MyAwesomeFilter would then
> deny the same request.

I am thinking the operator/vendor own filter should have some
implementation code for registering its original hint to jsonschema to
expose/validate available hints in the future.
The way should be easy as possible so that they can implement the code easily.
After that, we will be able to make the validation strict again.

Thanks
Ken Ohmichi

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [nova] jsonschema for scheduler hints

[Alex Xu]

2015-12-04 16:48 GMT+08:00 Sylvain Bauza :

>
>
> Le 04/12/2015 04:21, Alex Xu a écrit :
>
>
>
> 2015-12-02 23:12 GMT+08:00 Sylvain Bauza :
>
>>
>>
>> Le 02/12/2015 15:23, Sean Dague a écrit :
>>
>>> We have previously agreed that scheduler hints in Nova are an open ended
>>> thing. It's expected for sites to have additional scheduler filters
>>> which expose new hints. The way we handle that with our strict
>>> jsonschema is that we allow additional properties -
>>>
>>> https://github.com/openstack/nova/blob/1734ce7101982dd95f8fab1ab4815bd258a33744/nova/api/openstack/compute/schemas/scheduler_hints.py#L65
>>>
>>> This means that if you specify some garbage hint, you don't get feedback
>>> that it was garbage in your environment. That lost a couple of days
>>> building multinode tests in the gate. Having gotten used to the hints
>>> that "you've given us bad stuff", this was a stark change back to the
>>> old world.
>>>
>>> Would it be possible to make it so that the schema could be explicitly
>>> extended (instead of implicitly extended). So that additional
>>> properties=False, but a mechanism for a scheduler filter to register
>>> it's jsonschema in?
>>>
>>
>> I'm pretty +1 for that because we want to have in-tree filters clear for
>> the UX they provide when asking for scheduler hints.
>>
>
> +1 also, and we should have capability API for discovering what hints
> supported by current deployment.
>
>
>>
>> For the moment, it's possible to have 2 different filters asking for the
>> same hint without providing a way to explain the semantics so I would want
>> to make sure that one in-tree filter could just have the same behaviour for
>> *all the OpenStack deployments.*
>>
>> That said, I remember some discussion we had about that in the past, and
>> the implementation details we discussed about having the Nova API knowing
>> the list of filters and fitering by those.
>> To be clear, I want to make sure that we could not leak the deployment by
>> providing a 401 if a filter is not deployed, but rather just make sure that
>> all our in-tree filters are like checked, even if they aren't deployed.
>>
>
> There isn't any other Nova API return 401. So if you return 401, then
> everybody will know that is the only 401 in the nova, so I think there
> isn't any different. As we have capability API, it's fine let the user to
> know what is supported in the deployment.
>
>
>
> Sorry, I made a mistake by providing a wrong HTTP code for when the
> validation returns a ValidationError (due to the JSON schema not matched by
> the request).
> Here, my point is that if we enforce a per-enabled-filter basis for
> checking whether the hint should be enforced, it would mean that as an
> hacker, I could have some way to know what filters are enabled, or as an
> user, I could have different behaviours depending on the deployment.
>
> Let me give you an example: say that I'm not enabling the SameHostFilter
> which exposes the 'same_host' hint.
>
> For that specific cloud, if we allow to deny a request which could provide
> the 'same-host' hint (because the filter is not loaded by the
> 'scheduler_default_filters' option), it would make a difference with
> another cloud which enables SameHostFilter (because the request would pass).
>
> So, I'm maybe nitpicking, but I want to make clear that we shouldn't
> introspect the state of the filter, and just consider a static JSON schema
> (as we have today) which would reference all the hints, whether the
> corresponding filter is enabled or not.
>

yes, I see your concern, that is why I think we should have capabilities
API. User should query the capabilities API to ensure what filter enabled
in the current cloud.


>
>
>
>
>
>> That leaves the out-of-tree discussion about custom filters and how we
>> could have a consistent behaviour given that. Should we accept something in
>> a specific deployment while another deployment could 401 against it ? Mmm,
>> bad to me IMHO.
>>
>
> We can have code to check the out-of-tree filters didn't expose any same
> hints with in-tree filter.
>
>
>
> Sure, and thank you for that, that was missing in the past. That said,
> there are still some interoperability concerns, let me explain : as a cloud
> operator, I'm now providing a custom filter (say MyAwesomeFilter) which
> does the lookup for an hint called 'my_awesome_hint'.
>
> If we enforce a strict validation (and not allow to accept any hint) it
> would mean that this cloud would accept a request with 'my_awesome_hint'
> while another cloud which wouldn't be running MyAwesomeFilter would then
> deny the same request.
>

yes, same answer as above, capabilities API is used to discover enabled
'hints'.


>
> Hope I better explained my concerns,
> -Sylvain
>
>
>>
>> -Sylvain
>>
>> -Sean
>>>
>>>
>>
>> __
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe:
>> 

Re: [openstack-dev] [nova] jsonschema for scheduler hints

[Ken'ichi Ohmichi]

Hi Sean,

2015-12-02 23:23 GMT+09:00 Sean Dague :
> We have previously agreed that scheduler hints in Nova are an open ended
> thing. It's expected for sites to have additional scheduler filters
> which expose new hints. The way we handle that with our strict
> jsonschema is that we allow additional properties -
> https://github.com/openstack/nova/blob/1734ce7101982dd95f8fab1ab4815bd258a33744/nova/api/openstack/compute/schemas/scheduler_hints.py#L65
>
> This means that if you specify some garbage hint, you don't get feedback
> that it was garbage in your environment. That lost a couple of days
> building multinode tests in the gate. Having gotten used to the hints
> that "you've given us bad stuff", this was a stark change back to the
> old world.
>
> Would it be possible to make it so that the schema could be explicitly
> extended (instead of implicitly extended). So that additional
> properties=False, but a mechanism for a scheduler filter to register
> it's jsonschema in?

We had a prototype for registering scheduler_hints to jsonschema from
available filters:

https://review.openstack.org/#/c/220440/

That was against current basic design of nova and abandoned.
 We need more time for finding right implementation way for that.

BTW, I'd like to get feedback about scheduler_hints.
As the above jsonschema, nova just ignores unavailable scheduler_hint
if a client passes it.
So the client cannot know the specified scheduler_hint works or not
due to no feedback.
However, I feel that doesn't seem so bad because scheduler_hint is
just "hint" not "rule".
Nova can say "I consider this hint as possible, but sometimes ignore it".
In addition, Nova's purpose is for making physical environments abstract.
So it is not so bad to ignore the hint sometime.
Or should Nova handle scheduler_hint strictly?

I guess that depends on use cases.
So if needing to handle scheduler_hint strictly, we need the above
kind of mechanism for registering available hints to jsonschema.

Thanks
Ken Ohmichi

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [nova] jsonschema for scheduler hints

[Sylvain Bauza]

Le 04/12/2015 04:21, Alex Xu a écrit :



2015-12-02 23:12 GMT+08:00 Sylvain Bauza >:




Le 02/12/2015 15:23, Sean Dague a écrit :

We have previously agreed that scheduler hints in Nova are an
open ended
thing. It's expected for sites to have additional scheduler
filters
which expose new hints. The way we handle that with our strict
jsonschema is that we allow additional properties -

https://github.com/openstack/nova/blob/1734ce7101982dd95f8fab1ab4815bd258a33744/nova/api/openstack/compute/schemas/scheduler_hints.py#L65

This means that if you specify some garbage hint, you don't
get feedback
that it was garbage in your environment. That lost a couple of
days
building multinode tests in the gate. Having gotten used to
the hints
that "you've given us bad stuff", this was a stark change back
to the
old world.

Would it be possible to make it so that the schema could be
explicitly
extended (instead of implicitly extended). So that additional
properties=False, but a mechanism for a scheduler filter to
register
it's jsonschema in?


I'm pretty +1 for that because we want to have in-tree filters
clear for the UX they provide when asking for scheduler hints.


+1 also, and we should have capability API for discovering what hints 
supported by current deployment.



For the moment, it's possible to have 2 different filters asking
for the same hint without providing a way to explain the semantics
so I would want to make sure that one in-tree filter could just
have the same behaviour for *all the OpenStack deployments.*

That said, I remember some discussion we had about that in the
past, and the implementation details we discussed about having the
Nova API knowing the list of filters and fitering by those.
To be clear, I want to make sure that we could not leak the
deployment by providing a 401 if a filter is not deployed, but
rather just make sure that all our in-tree filters are like
checked, even if they aren't deployed.


There isn't any other Nova API return 401. So if you return 401, then 
everybody will know that is the only 401 in the nova, so I think there 
isn't any different. As we have capability API, it's fine let the user 
to know what is supported in the deployment.


Sorry, I made a mistake by providing a wrong HTTP code for when the 
validation returns a ValidationError (due to the JSON schema not matched 
by the request).
Here, my point is that if we enforce a per-enabled-filter basis for 
checking whether the hint should be enforced, it would mean that as an 
hacker, I could have some way to know what filters are enabled, or as an 
user, I could have different behaviours depending on the deployment.


Let me give you an example: say that I'm not enabling the SameHostFilter 
which exposes the 'same_host' hint.


For that specific cloud, if we allow to deny a request which could 
provide the 'same-host' hint (because the filter is not loaded by the 
'scheduler_default_filters' option), it would make a difference with 
another cloud which enables SameHostFilter (because the request would pass).


So, I'm maybe nitpicking, but I want to make clear that we shouldn't 
introspect the state of the filter, and just consider a static JSON 
schema (as we have today) which would reference all the hints, whether 
the corresponding filter is enabled or not.






That leaves the out-of-tree discussion about custom filters and
how we could have a consistent behaviour given that. Should we
accept something in a specific deployment while another deployment
could 401 against it ? Mmm, bad to me IMHO.


We can have code to check the out-of-tree filters didn't expose any 
same hints with in-tree filter.


Sure, and thank you for that, that was missing in the past. That said, 
there are still some interoperability concerns, let me explain : as a 
cloud operator, I'm now providing a custom filter (say MyAwesomeFilter) 
which does the lookup for an hint called 'my_awesome_hint'.


If we enforce a strict validation (and not allow to accept any hint) it 
would mean that this cloud would accept a request with 'my_awesome_hint' 
while another cloud which wouldn't be running MyAwesomeFilter would then 
deny the same request.


Hope I better explained my concerns,
-Sylvain




-Sylvain

-Sean



__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe:
openstack-dev-requ...@lists.openstack.org?subject:unsubscribe

http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [nova] jsonschema for scheduler hints

[Alex Xu]

2015-12-02 23:12 GMT+08:00 Sylvain Bauza :

>
>
> Le 02/12/2015 15:23, Sean Dague a écrit :
>
>> We have previously agreed that scheduler hints in Nova are an open ended
>> thing. It's expected for sites to have additional scheduler filters
>> which expose new hints. The way we handle that with our strict
>> jsonschema is that we allow additional properties -
>>
>> https://github.com/openstack/nova/blob/1734ce7101982dd95f8fab1ab4815bd258a33744/nova/api/openstack/compute/schemas/scheduler_hints.py#L65
>>
>> This means that if you specify some garbage hint, you don't get feedback
>> that it was garbage in your environment. That lost a couple of days
>> building multinode tests in the gate. Having gotten used to the hints
>> that "you've given us bad stuff", this was a stark change back to the
>> old world.
>>
>> Would it be possible to make it so that the schema could be explicitly
>> extended (instead of implicitly extended). So that additional
>> properties=False, but a mechanism for a scheduler filter to register
>> it's jsonschema in?
>>
>
> I'm pretty +1 for that because we want to have in-tree filters clear for
> the UX they provide when asking for scheduler hints.
>

+1 also, and we should have capability API for discovering what hints
supported by current deployment.


>
> For the moment, it's possible to have 2 different filters asking for the
> same hint without providing a way to explain the semantics so I would want
> to make sure that one in-tree filter could just have the same behaviour for
> *all the OpenStack deployments.*
>
> That said, I remember some discussion we had about that in the past, and
> the implementation details we discussed about having the Nova API knowing
> the list of filters and fitering by those.
> To be clear, I want to make sure that we could not leak the deployment by
> providing a 401 if a filter is not deployed, but rather just make sure that
> all our in-tree filters are like checked, even if they aren't deployed.
>

There isn't any other Nova API return 401. So if you return 401, then
everybody will know that is the only 401 in the nova, so I think there
isn't any different. As we have capability API, it's fine let the user to
know what is supported in the deployment.


>
> That leaves the out-of-tree discussion about custom filters and how we
> could have a consistent behaviour given that. Should we accept something in
> a specific deployment while another deployment could 401 against it ? Mmm,
> bad to me IMHO.
>

We can have code to check the out-of-tree filters didn't expose any same
hints with in-tree filter.


>
>
> -Sylvain
>
> -Sean
>>
>>
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [nova] jsonschema for scheduler hints

[Sylvain Bauza]

Le 02/12/2015 15:23, Sean Dague a écrit :

We have previously agreed that scheduler hints in Nova are an open ended
thing. It's expected for sites to have additional scheduler filters
which expose new hints. The way we handle that with our strict
jsonschema is that we allow additional properties -
https://github.com/openstack/nova/blob/1734ce7101982dd95f8fab1ab4815bd258a33744/nova/api/openstack/compute/schemas/scheduler_hints.py#L65

This means that if you specify some garbage hint, you don't get feedback
that it was garbage in your environment. That lost a couple of days
building multinode tests in the gate. Having gotten used to the hints
that "you've given us bad stuff", this was a stark change back to the
old world.

Would it be possible to make it so that the schema could be explicitly
extended (instead of implicitly extended). So that additional
properties=False, but a mechanism for a scheduler filter to register
it's jsonschema in?


I'm pretty +1 for that because we want to have in-tree filters clear for 
the UX they provide when asking for scheduler hints.


For the moment, it's possible to have 2 different filters asking for the 
same hint without providing a way to explain the semantics so I would 
want to make sure that one in-tree filter could just have the same 
behaviour for *all the OpenStack deployments.*


That said, I remember some discussion we had about that in the past, and 
the implementation details we discussed about having the Nova API 
knowing the list of filters and fitering by those.
To be clear, I want to make sure that we could not leak the deployment 
by providing a 401 if a filter is not deployed, but rather just make 
sure that all our in-tree filters are like checked, even if they aren't 
deployed.


That leaves the out-of-tree discussion about custom filters and how we 
could have a consistent behaviour given that. Should we accept something 
in a specific deployment while another deployment could 401 against it ? 
Mmm, bad to me IMHO.



-Sylvain


-Sean




__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev