Re: [openstack-dev] [keystone][nova][cinder][horizon][all] properties / metadata for resources
On Tue, Nov 8, 2016 at 4:12 PM, Gage Hugo wrote: > The idea is that a cloud admin could define a list of keys that they need > for their setup within keystone's configuration file, then only those keys > will be valid for storing values in the project properties table. Then each > call would check against the list of valid keys and deny any calls that are > sent with an invalid key. Please do not do this, it throws fuel onto the interoperability file we still have not put out. One actual technical drawback to doing this is that none of the OpenStack services can depend on any of those keys actually being defined, so this is still effectively just a single-deployment 'extras' field, with the only advance being that all rows may have the same set of keys, depending on how the configuration changes over time. > This idea seems to help with the issue to avoid allowing anyone to throw any > arbitrary values into these project properties vs just a set number of > values. It may feel that way, but it really does not help at all. From a cloud consumer point of view (including tools developed and used by deployers, possibly across multiple cloud deployments) this is no help at all. dt -- Dean Troyer dtro...@gmail.com __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [keystone][nova][cinder][horizon][all] properties / metadata for resources
On 11/8/2016 7:14 PM, Adrian Turjak wrote: On 09/11/16 11:12, Gage Hugo wrote: This spec was discussed at the keystone meeting today and during the conversation that continued afterwards, an idea of using the keystone configuration to set a list of keys was mentioned. The idea is that a cloud admin could define a list of keys that they need for their setup within keystone's configuration file, then only those keys will be valid for storing values in the project properties table. Then each call would check against the list of valid keys and deny any calls that are sent with an invalid key. This idea seems to help with the issue to avoid allowing anyone to throw any arbitrary values into these project properties vs just a set number of values. That feels far more restricting than it needs to be... If done like this, the list should be optional, as having to restarting Keystone to register the new config if you decide you need to add additional values is a terrible approach. __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev Agree, whitelisting this in config sounds like a really bad idea. -- Thanks, Matt Riedemann __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [keystone][nova][cinder][horizon][all] properties / metadata for resources
On 09/11/16 11:12, Gage Hugo wrote: > This spec was discussed at the keystone meeting today and during the > conversation that continued afterwards, an idea of using the keystone > configuration to set a list of keys was mentioned. > > The idea is that a cloud admin could define a list of keys that they > need for their setup within keystone's configuration file, then only > those keys will be valid for storing values in the project properties > table. Then each call would check against the list of valid keys and > deny any calls that are sent with an invalid key. > > This idea seems to help with the issue to avoid allowing anyone to > throw any arbitrary values into these project properties vs just a set > number of values. That feels far more restricting than it needs to be... If done like this, the list should be optional, as having to restarting Keystone to register the new config if you decide you need to add additional values is a terrible approach. __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [keystone][nova][cinder][horizon][all] properties / metadata for resources
On 11/8/2016 4:12 PM, Gage Hugo wrote: This spec was discussed at the keystone meeting today and during the conversation that continued afterwards, an idea of using the keystone configuration to set a list of keys was mentioned. The idea is that a cloud admin could define a list of keys that they need for their setup within keystone's configuration file, then only those keys will be valid for storing values in the project properties table. Then each call would check against the list of valid keys and deny any calls that are sent with an invalid key. This idea seems to help with the issue to avoid allowing anyone to throw any arbitrary values into these project properties vs just a set number of values. So...completely undiscoverable and cloud-specific? That doesn't sound very interoperable. -- Thanks, Matt Riedemann __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [keystone][nova][cinder][horizon][all] properties / metadata for resources
On 04/11/16 08:15 PM, Steve Martinelli wrote: > > We have somewhat had support for this, we have an "extras" column > defined in our database schema, whatever a user puts in a request that > doesn't match up with our API, those key-values are dumped into the > "extras" column. It's not a pleasant user experience, since you can't > really "unset" the data easily, or grab it, or update it. There's > actually been patches to keystoneclient for getting around this, but its > rather hacky and hardcodes a lot of values [2] [3] we've been storing metadata/attributes/properties in Ceilometer and Gnocchi. in Ceilometer, we just flattened the json and built keys based on that which allowed you to index and unset/set things. that said, it wasn't that great in Ceilometer because allowing it to be completely free-form just encouraged the practice of dumping useless information in it. in Gnocchi, we support dynamically addign attributes as well but you must explicitly tell it to create add the attribute to the resource. i won't lie, i don't know exactly how the magic works (you'll have to ask sileht), but it basically creates columns/tables to the db based on the request. cheers, -- gord __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [keystone][nova][cinder][horizon][all] properties / metadata for resources
This spec was discussed at the keystone meeting today and during the conversation that continued afterwards, an idea of using the keystone configuration to set a list of keys was mentioned. The idea is that a cloud admin could define a list of keys that they need for their setup within keystone's configuration file, then only those keys will be valid for storing values in the project properties table. Then each call would check against the list of valid keys and deny any calls that are sent with an invalid key. This idea seems to help with the issue to avoid allowing anyone to throw any arbitrary values into these project properties vs just a set number of values. On Sun, Nov 6, 2016 at 6:25 PM, Matt Riedemann wrote: > On 11/4/2016 7:15 PM, Steve Martinelli wrote: > >> The keystone team has a new spec being proposed for the Ocata release, >> it essentially boils down to adding properties / metadata for projects >> (for now) [1]. >> >> We have somewhat had support for this, we have an "extras" column >> defined in our database schema, whatever a user puts in a request that >> doesn't match up with our API, those key-values are dumped into the >> "extras" column. It's not a pleasant user experience, since you can't >> really "unset" the data easily, or grab it, or update it. There's >> actually been patches to keystoneclient for getting around this, but its >> rather hacky and hardcodes a lot of values [2] [3] >> >> I've added nova and cinder here since the APIs that are being proposed >> are more or less carbon copies of what is available through their APIs >> (for server and volumes, respectively). What has been your project's >> experience with handling metadata / properties? I assume that its been >> there a while and you can't remove it. If you could go back and redo >> things, would you do it another way? Would you take a more purist stance >> and enforce more strict APIs, metadata be damned? >> >> I also added horizon because i'm curious about the impact this causes >> when representing a resource. >> >> Personally, I am for the idea, we've had numerous requests from >> operators about providing this support and I like to make them happy. >> >> [1] https://review.openstack.org/#/c/36/ >> [2] https://review.openstack.org/#/c/375239/ >> [3] https://review.openstack.org/#/c/296246/ >> >> >> >> __ >> OpenStack Development Mailing List (not for usage questions) >> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscrib >> e >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >> >> > If you're going to do it, restrict the case and characters in the keys > because if you don't you can get fits in the backend database wrinkles. See > this nova spec for details: > > https://specs.openstack.org/openstack/nova-specs/specs/newto > n/approved/lowercase-metadata-keys.html > > -- > > Thanks, > > Matt Riedemann > > > __ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [keystone][nova][cinder][horizon][all] properties / metadata for resources
On 11/4/2016 7:15 PM, Steve Martinelli wrote: The keystone team has a new spec being proposed for the Ocata release, it essentially boils down to adding properties / metadata for projects (for now) [1]. We have somewhat had support for this, we have an "extras" column defined in our database schema, whatever a user puts in a request that doesn't match up with our API, those key-values are dumped into the "extras" column. It's not a pleasant user experience, since you can't really "unset" the data easily, or grab it, or update it. There's actually been patches to keystoneclient for getting around this, but its rather hacky and hardcodes a lot of values [2] [3] I've added nova and cinder here since the APIs that are being proposed are more or less carbon copies of what is available through their APIs (for server and volumes, respectively). What has been your project's experience with handling metadata / properties? I assume that its been there a while and you can't remove it. If you could go back and redo things, would you do it another way? Would you take a more purist stance and enforce more strict APIs, metadata be damned? I also added horizon because i'm curious about the impact this causes when representing a resource. Personally, I am for the idea, we've had numerous requests from operators about providing this support and I like to make them happy. [1] https://review.openstack.org/#/c/36/ [2] https://review.openstack.org/#/c/375239/ [3] https://review.openstack.org/#/c/296246/ __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev If you're going to do it, restrict the case and characters in the keys because if you don't you can get fits in the backend database wrinkles. See this nova spec for details: https://specs.openstack.org/openstack/nova-specs/specs/newton/approved/lowercase-metadata-keys.html -- Thanks, Matt Riedemann __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [keystone][nova][cinder][horizon][all] properties / metadata for resources
On 06/11/16 13:17, Steve Martinelli wrote: > > Interesting, I'll add this to the review and see how some if the folks > proposing the new APIs would find that as suitable for their use > cases. For reference: http://developer.openstack.org/api-ref/compute/ For our use case, I need key:value pairings, so something like the tags system wouldn't quite work. That said, I kind of like the hardcoded limit of: "Each server can have up to 50 tags." The examples currently that we use (or would like to use if we could limit access to certain roles): Project (in extra): created_at: created_by: @ terminated_at: terminated_by: : @ terminated_reason: sign_up_type: <"individual" or "organisation"> organisation: partner_id: User (in extra): created_at: created_by: @ invited_by: @ of person who sent invitation. terminated_at: terminated_by: : @ terminated_reason: Chances are we will be adding more as well. Right now part of the problem is that a user can do project get for their own project, and will see values in extra, which means we can't store anything in there we don't want the clients or their users to see. I will point out though that some of this stuff we keep in our ERP system as well, but it is far less flexible than OpenStack and much of that info we'd like to keep synced in both places so that it is easy to query from either direction. This makes audit trails easier and allows a "project show" to tell us what we need to know about a project without going to the ERP system as well (which not everyone has access to anyway). Also worth noting is that the reason most of this works, and is actually enforced, is that we don't use Keystone directly for project/user creation/management. We have a service that handles the automation of admin tasks and automates most of this via the Keystoneclient. We do still have people with actual admin access who do occasionally change things manually, but we are doing more and more via this service both for consistency, and to track who did what when. Doing all of the above via the proposed new API would be easy, and while the timedate values won't themselves be queryable, the "created_at"/"updated_at" values on the property will be. So I can do a query along the lines of: projects where properties have "terminated_at" and property updated_at >= ; Doing this via swift is... I guess I could store a list of each property in a file, and then parse the contents. For straightforward tags, that would be fine, but not for key:value pairs where the contents of the value will be different. I could probably do it by making the files instead be more of a reverse mapping, where I make a container for each resource type and have the file name as "_" (eg "terminated_by_@") with the file itself containing a list of resource ids. That would at least make things less awful to search for, but it would still be MUCH slower than if these were proper Keystone database entries. Not to mention doing it in swift would make it hard to expose to anything but the project where the swift data is stored in. I'd need to build a service to handle these queries for me, and it would need to be built in a service project so it has access to swift, but exposes its API to OpenStack. So not Swift I think. > > I am most concerned actually about the resistance from some in the > Keystone contributor community to storing quota *limits* [1] for > users and projects. Right now, every service project needs to > store information about quota limits for all users and projects, > and the services each do this annoyingly differently. Keystone is > the thing that stores attributes of a user or a project. Limits of > various quantitative resources in the system are an attribute of a > user or a project. This information belongs in Keystone, IMHO, > with a good REST API that other services can use to grab this > information. > > > Actually, this summit was the first I've heard of it (more so than > just a passing idea with no one up for doing the work). We talked > about it at our unconference session and Boris Bobrov (breton) has a > few TODOs on the topic (post to ML and create a > spec https://etherpad.openstack.org/p/ocata-keystone-unconference ) > Storing limits (quotas) in Keystone feels wrong, although I can't place my finger on why. While yes they are sort of attributes of a project, they aren't exactly identity or access attributes. I do think we need to centralise them, I just don't know if Keystone is exactly the place for it, although I agree that there isn't a better place right now. Plus centralising them might actually mean we can do hierarchical quotas! As odd as it may sound, what if we considered that limits are a form of dynamic policy? And migrate to treating resource limits as such. Combine that with a general shift to centralised dynamic policies in Keystone, and then it sort of feels better. It would be a massive effort, but it could mean per us
Re: [openstack-dev] [keystone][nova][cinder][horizon][all] properties / metadata for resources
On Sat, Nov 5, 2016 at 6:15 PM, Jay Pipes wrote: > On 11/05/2016 01:15 AM, Steve Martinelli wrote: > >> The keystone team has a new spec being proposed for the Ocata release, >> it essentially boils down to adding properties / metadata for projects >> (for now) [1]. >> > > Yes, I'd seen that particular spec review and found it interesting in a > couple ways. Please comment on it :) I've added nova and cinder here since the APIs that are being proposed >> are more or less carbon copies of what is available through their APIs >> (for server and volumes, respectively). What has been your project's >> experience with handling metadata / properties? I assume that its been >> there a while and you can't remove it. If you could go back and redo >> things, would you do it another way? Would you take a more purist stance >> and enforce more strict APIs, metadata be damned? >> > > Yes. I would get rid of the server metadata API that is in the Compute > API. I believe the server tags API in the Compute API is appropriate for > user-defined taxonomy of servers. For non user-defined things like system > metadata, I prefer to have schema-defined attributes that are standardize > and typed but a structured "properties" API can be useful as long as the > key and value fields are indexable and reasonably sized. > Interesting, I'll add this to the review and see how some if the folks proposing the new APIs would find that as suitable for their use cases. For reference: http://developer.openstack.org/api-ref/compute/ > I also added horizon because i'm curious about the impact this causes >> when representing a resource. >> >> Personally, I am for the idea, we've had numerous requests from >> operators about providing this support and I like to make them happy. >> > > I am most concerned actually about the resistance from some in the > Keystone contributor community to storing quota *limits* [1] for users and > projects. Right now, every service project needs to store information about > quota limits for all users and projects, and the services each do this > annoyingly differently. Keystone is the thing that stores attributes of a > user or a project. Limits of various quantitative resources in the system > are an attribute of a user or a project. This information belongs in > Keystone, IMHO, with a good REST API that other services can use to grab > this information. > Actually, this summit was the first I've heard of it (more so than just a passing idea with no one up for doing the work). We talked about it at our unconference session and Boris Bobrov (breton) has a few TODOs on the topic (post to ML and create a spec https://etherpad.openstack.org/p/ocata-keystone-unconference ) __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [keystone][nova][cinder][horizon][all] properties / metadata for resources
On 11/05/2016 01:15 AM, Steve Martinelli wrote: The keystone team has a new spec being proposed for the Ocata release, it essentially boils down to adding properties / metadata for projects (for now) [1]. Yes, I'd seen that particular spec review and found it interesting in a couple ways. We have somewhat had support for this, we have an "extras" column defined in our database schema, whatever a user puts in a request that doesn't match up with our API, those key-values are dumped into the "extras" column. It's not a pleasant user experience, since you can't really "unset" the data easily, or grab it, or update it. There's actually been patches to keystoneclient for getting around this, but its rather hacky and hardcodes a lot of values [2] [3] "not a pleasant user experience" would be an understatement :) In addition to the unpleasant user experience, there is the additional problem that jamming such information into a JSON BLOB and storing it in a TEXT field in a relational database means none of the information stored in the field can be indexed which means there's no ability to search on particular key or value information. I've added nova and cinder here since the APIs that are being proposed are more or less carbon copies of what is available through their APIs (for server and volumes, respectively). What has been your project's experience with handling metadata / properties? I assume that its been there a while and you can't remove it. If you could go back and redo things, would you do it another way? Would you take a more purist stance and enforce more strict APIs, metadata be damned? Yes. I would get rid of the server metadata API that is in the Compute API. I believe the server tags API in the Compute API is appropriate for user-defined taxonomy of servers. For non user-defined things like system metadata, I prefer to have schema-defined attributes that are standardize and typed but a structured "properties" API can be useful as long as the key and value fields are indexable and reasonably sized. I also added horizon because i'm curious about the impact this causes when representing a resource. Personally, I am for the idea, we've had numerous requests from operators about providing this support and I like to make them happy. I am most concerned actually about the resistance from some in the Keystone contributor community to storing quota *limits* [1] for users and projects. Right now, every service project needs to store information about quota limits for all users and projects, and the services each do this annoyingly differently. Keystone is the thing that stores attributes of a user or a project. Limits of various quantitative resources in the system are an attribute of a user or a project. This information belongs in Keystone, IMHO, with a good REST API that other services can use to grab this information. Best, -jay [1] limits, not usages. __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [keystone][nova][cinder][horizon][all] properties / metadata for resources
The keystone team has a new spec being proposed for the Ocata release, it essentially boils down to adding properties / metadata for projects (for now) [1]. We have somewhat had support for this, we have an "extras" column defined in our database schema, whatever a user puts in a request that doesn't match up with our API, those key-values are dumped into the "extras" column. It's not a pleasant user experience, since you can't really "unset" the data easily, or grab it, or update it. There's actually been patches to keystoneclient for getting around this, but its rather hacky and hardcodes a lot of values [2] [3] I've added nova and cinder here since the APIs that are being proposed are more or less carbon copies of what is available through their APIs (for server and volumes, respectively). What has been your project's experience with handling metadata / properties? I assume that its been there a while and you can't remove it. If you could go back and redo things, would you do it another way? Would you take a more purist stance and enforce more strict APIs, metadata be damned? I also added horizon because i'm curious about the impact this causes when representing a resource. Personally, I am for the idea, we've had numerous requests from operators about providing this support and I like to make them happy. [1] https://review.openstack.org/#/c/36/ [2] https://review.openstack.org/#/c/375239/ [3] https://review.openstack.org/#/c/296246/ __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
