Re: [openstack-dev] [nova] The strange case of osapi_compute_unique_server_name_scope

2015-02-23 Thread Matthew Booth
On 20/02/15 20:15, Andrew Bogott wrote:
 On 2/20/15 9:06 AM, Mike Dorman wrote:
 I can report that we do use this option (‘global' setting.)  We have to
 enforce name uniqueness for instances’ integration with some external
 systems (namely AD and Spacewalk) which require unique naming.

 However, we also do some external name validation which I think
 effectively enforces uniqueness as well.  So if this were deprecated, I
 don’t know if it would directly affect us for our specific situation.

 Other operators, anyone else using
 osapi_compute_unique_server_name_scope?
 I use it!  And, in fact, added it in the first place :(
 
 I have no real recall of what we concluded when originally discussing
 the associated race.  The feature is useful to me and I'd love it if it
 could be moved into the db to fix the race, but I concede that it's a
 pretty big can o' worms, so if no one else cries out in pain I can live
 with it being deprecated.

Ok, with 2 identified users I'm going to abandon my patch to deprecate
with no replacement. My current plan is to leave the race there with a
comment in the code and the config documentation. Perhaps we can fix
this properly at some point when we get the online schema changes, but
for the moment it seems like a lot of complication for a relatively
small problem.

Do you use the global or project scope, btw?

Matt
-- 
Matthew Booth
Red Hat Engineering, Virtualisation Team

Phone: +442070094448 (UK)
GPG ID:  D33C3490
GPG FPR: 3733 612D 2D05 5458 8A8A 1600 3441 EA19 D33C 3490

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [nova] The strange case of osapi_compute_unique_server_name_scope

2015-02-20 Thread Matthew Booth
On 19/02/15 18:57, Jay Pipes wrote:
 On 02/19/2015 05:18 AM, Matthew Booth wrote:
 Nova contains a config variable osapi_compute_unique_server_name_scope.
 Its help text describes it pretty well:

 When set, compute API will consider duplicate hostnames invalid within
 the specified scope, regardless of case. Should be empty, project or
 global.

 So, by default hostnames are not unique, but depending on this setting
 they could be unique either globally or in the scope of a project.

 Ideally a unique constraint would be enforced by the database but,
 presumably because this is a config variable, that isn't the case here.
 Instead it is enforced in code, but the code which does this predictably
 races. My first attempt to fix this using the obvious SQL solution
 appeared to work, but actually fails in MySQL as it doesn't support that
 query structure[1][2]. SQLite and PostgreSQL do support it, but they
 don't support the query structure which MySQL supports. Note that this
 isn't just a syntactic thing. It looks like it's still possible to do
 this if we compound the workaround with a second workaround, but I'm
 starting to wonder if we'd be better fixing the design.

 First off, do we need this config variable? Is anybody actually using
 it? I suspect the answer's going to be yes, but it would be extremely
 convenient if it's not.

 Assuming this configurability is required, is there any way we can
 instead use it to control a unique constraint in the db at service
 startup? This would be something akin to a db migration. How do we
 manage those?

 Related to the above, I'm not 100% clear on which services run this
 code. Is it possible for different services to have a different
 configuration of this variable, and does that make sense? If so, that
 would preclude a unique constraint in the db.
 
 Is there a bug that you are attempting to fix here? If not, I'd suggest
 just leaving this code as it is...

The bug is the race. If a user sets
osapi_compute_unique_server_name_scope they're presumably expecting the
associating uniqueness guarantees, but we're not providing them.

John suggested I deprecate it and see who complains:

https://review.openstack.org/157731

In the mean time, I'm starting to think the most prudent course of
action would be to not fix the race: it's not the most important race in
that function, it's been broken for a long time, and I suspect few
people are using it. We could document that it's broken... In fact, I
might add that to the deprecation notice.

Matt
-- 
Matthew Booth
Red Hat Engineering, Virtualisation Team

Phone: +442070094448 (UK)
GPG ID:  D33C3490
GPG FPR: 3733 612D 2D05 5458 8A8A 1600 3441 EA19 D33C 3490

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [nova] The strange case of osapi_compute_unique_server_name_scope

2015-02-20 Thread Johannes Erdfelt
On Thu, Feb 19, 2015, Matthew Booth mbo...@redhat.com wrote:
 Assuming this configurability is required, is there any way we can
 instead use it to control a unique constraint in the db at service
 startup? This would be something akin to a db migration. How do we
 manage those?

Ignoring if this particular feature is useful or not, this is possible.

With sqlalchemy-migrate, there could be code to check the config
option at startup and add/remove the unique constraint. This would leave
some schema management out of the existing scripts, which would be
mildly ugly.

With my online schema changes patch, this is all driven by the model.
Similar code could add/remove the unique constraint to the model. At
startup, the schema could be compared against the model to ensure
everything matches.

Adding/removing a unique constraint at any time leaves open some user
experience problems with data that violates the constraint preventing it
from being created.

Presumably a tool could help operators deal with that.

All that said, it's kind of messy and nontrivial work, so I'd avoid
trying to support a feature like this if we really don't need to :)

JE


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [nova] The strange case of osapi_compute_unique_server_name_scope

2015-02-20 Thread Mike Dorman
I can report that we do use this option (‘global' setting.)  We have to 
enforce name uniqueness for instances’ integration with some external 
systems (namely AD and Spacewalk) which require unique naming.

However, we also do some external name validation which I think 
effectively enforces uniqueness as well.  So if this were deprecated, I 
don’t know if it would directly affect us for our specific situation.

Other operators, anyone else using osapi_compute_unique_server_name_scope?

Mike





On 2/19/15, 3:18 AM, Matthew Booth mbo...@redhat.com wrote:

Nova contains a config variable osapi_compute_unique_server_name_scope.
Its help text describes it pretty well:

When set, compute API will consider duplicate hostnames invalid within
the specified scope, regardless of case. Should be empty, project or
global.

So, by default hostnames are not unique, but depending on this setting
they could be unique either globally or in the scope of a project.

Ideally a unique constraint would be enforced by the database but,
presumably because this is a config variable, that isn't the case here.
Instead it is enforced in code, but the code which does this predictably
races. My first attempt to fix this using the obvious SQL solution
appeared to work, but actually fails in MySQL as it doesn't support that
query structure[1][2]. SQLite and PostgreSQL do support it, but they
don't support the query structure which MySQL supports. Note that this
isn't just a syntactic thing. It looks like it's still possible to do
this if we compound the workaround with a second workaround, but I'm
starting to wonder if we'd be better fixing the design.

First off, do we need this config variable? Is anybody actually using
it? I suspect the answer's going to be yes, but it would be extremely
convenient if it's not.

Assuming this configurability is required, is there any way we can
instead use it to control a unique constraint in the db at service
startup? This would be something akin to a db migration. How do we
manage those?

Related to the above, I'm not 100% clear on which services run this
code. Is it possible for different services to have a different
configuration of this variable, and does that make sense? If so, that
would preclude a unique constraint in the db.

Thanks,

Matt

[1] Which has prompted me to get the test_db_api tests running on MySQL.
See this series if you're interested:
https://review.openstack.org/#/c/156299/

[2] For specifics, see my ramblings here:
https://review.openstack.org/#/c/141115/7/nova/db/sqlalchemy/api.py,cm
line 2547
-- 
Matthew Booth
Red Hat Engineering, Virtualisation Team

Phone: +442070094448 (UK)
GPG ID:  D33C3490
GPG FPR: 3733 612D 2D05 5458 8A8A 1600 3441 EA19 D33C 3490

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [nova] The strange case of osapi_compute_unique_server_name_scope

2015-02-20 Thread Andrew Bogott

On 2/20/15 9:06 AM, Mike Dorman wrote:

I can report that we do use this option (‘global' setting.)  We have to
enforce name uniqueness for instances’ integration with some external
systems (namely AD and Spacewalk) which require unique naming.

However, we also do some external name validation which I think
effectively enforces uniqueness as well.  So if this were deprecated, I
don’t know if it would directly affect us for our specific situation.

Other operators, anyone else using osapi_compute_unique_server_name_scope?

I use it!  And, in fact, added it in the first place :(

I have no real recall of what we concluded when originally discussing 
the associated race.  The feature is useful to me and I'd love it if it 
could be moved into the db to fix the race, but I concede that it's a 
pretty big can o' worms, so if no one else cries out in pain I can live 
with it being deprecated.


-Andrew




Mike





On 2/19/15, 3:18 AM, Matthew Booth mbo...@redhat.com wrote:


Nova contains a config variable osapi_compute_unique_server_name_scope.
Its help text describes it pretty well:

When set, compute API will consider duplicate hostnames invalid within
the specified scope, regardless of case. Should be empty, project or
global.

So, by default hostnames are not unique, but depending on this setting
they could be unique either globally or in the scope of a project.

Ideally a unique constraint would be enforced by the database but,
presumably because this is a config variable, that isn't the case here.
Instead it is enforced in code, but the code which does this predictably
races. My first attempt to fix this using the obvious SQL solution
appeared to work, but actually fails in MySQL as it doesn't support that
query structure[1][2]. SQLite and PostgreSQL do support it, but they
don't support the query structure which MySQL supports. Note that this
isn't just a syntactic thing. It looks like it's still possible to do
this if we compound the workaround with a second workaround, but I'm
starting to wonder if we'd be better fixing the design.

First off, do we need this config variable? Is anybody actually using
it? I suspect the answer's going to be yes, but it would be extremely
convenient if it's not.

Assuming this configurability is required, is there any way we can
instead use it to control a unique constraint in the db at service
startup? This would be something akin to a db migration. How do we
manage those?

Related to the above, I'm not 100% clear on which services run this
code. Is it possible for different services to have a different
configuration of this variable, and does that make sense? If so, that
would preclude a unique constraint in the db.

Thanks,

Matt

[1] Which has prompted me to get the test_db_api tests running on MySQL.
See this series if you're interested:
https://review.openstack.org/#/c/156299/

[2] For specifics, see my ramblings here:
https://review.openstack.org/#/c/141115/7/nova/db/sqlalchemy/api.py,cm
line 2547
--
Matthew Booth
Red Hat Engineering, Virtualisation Team

Phone: +442070094448 (UK)
GPG ID:  D33C3490
GPG FPR: 3733 612D 2D05 5458 8A8A 1600 3441 EA19 D33C 3490

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


[openstack-dev] [nova] The strange case of osapi_compute_unique_server_name_scope

2015-02-19 Thread Matthew Booth
Nova contains a config variable osapi_compute_unique_server_name_scope.
Its help text describes it pretty well:

When set, compute API will consider duplicate hostnames invalid within
the specified scope, regardless of case. Should be empty, project or
global.

So, by default hostnames are not unique, but depending on this setting
they could be unique either globally or in the scope of a project.

Ideally a unique constraint would be enforced by the database but,
presumably because this is a config variable, that isn't the case here.
Instead it is enforced in code, but the code which does this predictably
races. My first attempt to fix this using the obvious SQL solution
appeared to work, but actually fails in MySQL as it doesn't support that
query structure[1][2]. SQLite and PostgreSQL do support it, but they
don't support the query structure which MySQL supports. Note that this
isn't just a syntactic thing. It looks like it's still possible to do
this if we compound the workaround with a second workaround, but I'm
starting to wonder if we'd be better fixing the design.

First off, do we need this config variable? Is anybody actually using
it? I suspect the answer's going to be yes, but it would be extremely
convenient if it's not.

Assuming this configurability is required, is there any way we can
instead use it to control a unique constraint in the db at service
startup? This would be something akin to a db migration. How do we
manage those?

Related to the above, I'm not 100% clear on which services run this
code. Is it possible for different services to have a different
configuration of this variable, and does that make sense? If so, that
would preclude a unique constraint in the db.

Thanks,

Matt

[1] Which has prompted me to get the test_db_api tests running on MySQL.
See this series if you're interested:
https://review.openstack.org/#/c/156299/

[2] For specifics, see my ramblings here:
https://review.openstack.org/#/c/141115/7/nova/db/sqlalchemy/api.py,cm
line 2547
-- 
Matthew Booth
Red Hat Engineering, Virtualisation Team

Phone: +442070094448 (UK)
GPG ID:  D33C3490
GPG FPR: 3733 612D 2D05 5458 8A8A 1600 3441 EA19 D33C 3490

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [nova] The strange case of osapi_compute_unique_server_name_scope

2015-02-19 Thread Jay Pipes

On 02/19/2015 05:18 AM, Matthew Booth wrote:

Nova contains a config variable osapi_compute_unique_server_name_scope.
Its help text describes it pretty well:

When set, compute API will consider duplicate hostnames invalid within
the specified scope, regardless of case. Should be empty, project or
global.

So, by default hostnames are not unique, but depending on this setting
they could be unique either globally or in the scope of a project.

Ideally a unique constraint would be enforced by the database but,
presumably because this is a config variable, that isn't the case here.
Instead it is enforced in code, but the code which does this predictably
races. My first attempt to fix this using the obvious SQL solution
appeared to work, but actually fails in MySQL as it doesn't support that
query structure[1][2]. SQLite and PostgreSQL do support it, but they
don't support the query structure which MySQL supports. Note that this
isn't just a syntactic thing. It looks like it's still possible to do
this if we compound the workaround with a second workaround, but I'm
starting to wonder if we'd be better fixing the design.

First off, do we need this config variable? Is anybody actually using
it? I suspect the answer's going to be yes, but it would be extremely
convenient if it's not.

Assuming this configurability is required, is there any way we can
instead use it to control a unique constraint in the db at service
startup? This would be something akin to a db migration. How do we
manage those?

Related to the above, I'm not 100% clear on which services run this
code. Is it possible for different services to have a different
configuration of this variable, and does that make sense? If so, that
would preclude a unique constraint in the db.


Is there a bug that you are attempting to fix here? If not, I'd suggest 
just leaving this code as it is...


Best,
-jay

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev