[openstack-dev] Keystone PTL Candidacy
Hello, everyone! I'd like to keep my name in the hat as PTL for Keystone during the Juno release cycle. As I'm not looking to shake things up for Juno, I'm going to direct you to my Icehouse PTL candidacy email [1], and promise that I will continue to deliver on that philosophy in Juno. Specifically, it's worth reiterating that my primary focus is in ... supporting our outstanding community of contributors in any way that I can so that they can be as productive as possible. Never mind the PTL, Keystone's success would not be possible without the community behind it. I actually view the client-side pieces to be the most important parts of keystone (and keystoneclient.middleware.auth_token, in particular) due to the more immediate impact on our stakeholders. In terms of Juno, I'm especially looking forward to landing support for ephemeral, signed tokens backed by revocation events on the client-side, along with increasing adoption for keystoneclient.session (and therefore the v3 API) across OpenStack. To all of our contributors during Icehouse: THANK YOU! -Dolph [1] http://lists.openstack.org/pipermail/openstack-dev/2013-September/015387.html ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [All][Keystone] Deprecation of the v2 API
On Tue, Apr 1, 2014 at 7:40 PM, Anne Gentle a...@openstack.org wrote: On Wed, Mar 26, 2014 at 5:30 AM, Thierry Carrez thie...@openstack.orgwrote: Russell Bryant wrote: [...] First, it seems there isn't a common use of deprecated. To me, marking something deprecated means that the deprecated feature: - has been completely replaced by something else - end users / deployers should take action to migrate to the new thing immediately. - The project has provided a documented migration path - the old thing will be removed at a specific date/release Agreed, IMHO we need to reserve the use the deprecated terminology for the idea of moving end users, deployers, external client library developers (people outside of OpenStack direct reach) off a given API version. Deprecation is about giving them a fair heads-up about something that is about to be removed, so that they are encouraged to move off it. It needs to be discussed and announced with the user community, and come with a precise plan. Internal consumption of APIs between OpenStack projects is a different beast: (1) it's under our control and (2) we really can't remove an API until all our internal pieces have migrated off it. So I wouldn't use deprecation warnings to encourage other OpenStack projects to move off an API. They can't come with a precise date since if projects don't comply with this suggestion we just can't remove that API support. I would therefore go this way: 1. API vN is stable and supported 2. API vN+1 is being developed and experimental 3. API vN+1 is marked stable and supported 4. Engage with other consuming OpenStack projects to migrate to vN+1 5. Migration is completed 6. Deprecation plan (and removal date) is discussed with stakeholders 7. Deprecation plan (and removal date) is decided and announced 8. Deprecation messages are added to code for API vN users 9. At removal date, API vN is removed Keystone is at step 4. It shouldn't use deprecation terminology before step 6. Hi all, I wanted to double-check some wording. I'm chasing down how to update docs based on a DocImpact flag[1] with related bugs and the nova blueprint to deprecate XML. [2] I'm pretty sure deprecate is the term we're using as the message says XML support has been deprecated and will be removed in the Juno release. [3] I don't see any issue with using the term deprecate here. However, I think it's important to say it may be removed, not will. Deprecations are frequently delayed, so the wording has potential to be inaccurate as-is. You could even go so far as to say it *may* be removed *as early as* the Juno release. Oslo's deprecation decorator (the deprecator) does not use the wording today, though. To me, the same arguments that caused a different message to be substituted for Keystone's v2.0 API could be used here. Is that inaccurate? Has the discussion been carried to the logical conclusion? Should deprecated be used yet? I just want to point out that Keystone followed Nova's lead on this one - Keystone's XML support, which is only rigorously tested against a very small portion of the v2 API, is deprecated as of Icehouse as well. I think XML is a broader community issue, not one that needs to be discussed per-project. I think we could follow a similar pattern for XML in the Compute API v2 -- we need to define discussed with stakeholders and how/when we say the discussion is done. I wanted to put up a set of ideas for the channels (push) and inputs (pull): - posted a collaborative statement expressing need to deprecate to the openstack-dev mailing list - posted a request for comment to the openstack mailing list In the case of XML, I think we already have a sufficiently long history of relevant mailing list discussions suggesting a focus on a single serialization format (JSON, in our case). - questions added to the user survey that indicate amount of use and desire to move away from - responses to user survey compiled showing a certain percent of agreement on deprecation This would be really interesting to see, at least for XML. I wouldn't recommend it for all the smaller features that face deprecation each release. Would that be sufficient for the discussed with stakeholders step? Any other channels I'm missing? If the discussion requires six months of discussion at a minimum, is that acceptable? One of keystone's summit sessions in Hong Kong partly focused on producing a list of things to deprecate during Icehouse. We ended up deprecating things that there was a clear consensus on (redundant middleware, etc). Although there were certainly deployers represented in that session, I like the idea of making deprecations a regular topic in a project-specific design session focused on collecting deployer feedback (using the session format suggested in a recent openstack-dev thread for Atlanta). Thanks, Anne [1]
Re: [openstack-dev] Operators Design Summit ideas for Atlanta
On Mon, Mar 31, 2014 at 10:40 PM, Adam Young ayo...@redhat.com wrote: On 03/28/2014 03:01 AM, Tom Fifield wrote: Thanks to those projects that responded. I've proposed sessions in swift, ceilometer, tripleO and horizon. Keystone would also be interested in user feedback, of course. Crossing openstack-dev threads [1] here, gathering feedback on proposed deprecations would be a great topic for such a session. [1] http://lists.openstack.org/pipermail/openstack-dev/2014-April/031652.html On 17/03/14 07:54, Tom Fifield wrote: All, Many times we've heard a desire for more feedback and interaction from users. However, their attendance at design summit sessions is met with varied success. However, last summit, by happy accident, a swift session turned into a something a lot more user driven. A competent user was able to describe their use case, and the developers were able to stage a number of question to them. In this way, some of the assumptions about the way certain things were implemented, and the various priorities of future plans became clearer. It worked really well ... perhaps this is something we'd like to have happen for all the projects? *Idea*: Add an ops session for each project in the design summit https://etherpad.openstack.org/p/ATL-ops-dedicated- design-summit-sessions Most operators running OpenStack tend to treat it more holistically than those coding it. They are aware of, but don't necessarily think or work in terms of project breakdowns. To this end, I'd imagine the such sessions would: * have a primary purpose for developers to ask the operators to answer questions, and request information * allow operators to tell the developers things (give feedback) as a secondary purpose that could potentially be covered better in a cross-project session * need good moderation, for example to push operator-to-operator discussion into forums with more time available (eg https://etherpad.openstack.org/p/ATL-ops-unconference-RFC ) * be reinforced by having volunteer good users in potentially every design summit session (https://etherpad.openstack.org/p/ATL-ops-in-design-sessions ) Anyway, just a strawman - please jump on the etherpad (https://etherpad.openstack.org/p/ATL-ops-dedicated- design-summit-sessions) or leave your replies here! Regards, Tom ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [All][Keystone] Deprecation of the v2 API
On Wed, Apr 2, 2014 at 8:43 AM, Russell Bryant rbry...@redhat.com wrote: On 04/02/2014 09:20 AM, Dolph Mathews wrote: On Tue, Apr 1, 2014 at 7:40 PM, Anne Gentle a...@openstack.org mailto:a...@openstack.org wrote: On Wed, Mar 26, 2014 at 5:30 AM, Thierry Carrez thie...@openstack.org mailto:thie...@openstack.org wrote: Russell Bryant wrote: [...] First, it seems there isn't a common use of deprecated. To me, marking something deprecated means that the deprecated feature: - has been completely replaced by something else - end users / deployers should take action to migrate to the new thing immediately. - The project has provided a documented migration path - the old thing will be removed at a specific date/release Agreed, IMHO we need to reserve the use the deprecated terminology for the idea of moving end users, deployers, external client library developers (people outside of OpenStack direct reach) off a given API version. Deprecation is about giving them a fair heads-up about something that is about to be removed, so that they are encouraged to move off it. It needs to be discussed and announced with the user community, and come with a precise plan. Internal consumption of APIs between OpenStack projects is a different beast: (1) it's under our control and (2) we really can't remove an API until all our internal pieces have migrated off it. So I wouldn't use deprecation warnings to encourage other OpenStack projects to move off an API. They can't come with a precise date since if projects don't comply with this suggestion we just can't remove that API support. I would therefore go this way: 1. API vN is stable and supported 2. API vN+1 is being developed and experimental 3. API vN+1 is marked stable and supported 4. Engage with other consuming OpenStack projects to migrate to vN+1 5. Migration is completed 6. Deprecation plan (and removal date) is discussed with stakeholders 7. Deprecation plan (and removal date) is decided and announced 8. Deprecation messages are added to code for API vN users 9. At removal date, API vN is removed Keystone is at step 4. It shouldn't use deprecation terminology before step 6. Hi all, I wanted to double-check some wording. I'm chasing down how to update docs based on a DocImpact flag[1] with related bugs and the nova blueprint to deprecate XML. [2] I'm pretty sure deprecate is the term we're using as the message says XML support has been deprecated and will be removed in the Juno release. [3] I don't see any issue with using the term deprecate here. However, I think it's important to say it may be removed, not will. Deprecations are frequently delayed, so the wording has potential to be inaccurate as-is. You could even go so far as to say it *may* be removed *as early as* the Juno release. Oslo's deprecation decorator (the deprecator) does not use the wording today, though. Agree that the wording should be changed. https://review.openstack.org/84725 https://bugs.launchpad.net/nova/+bug/1301384 I'll get this into icehouse-rc2 for Nova. To me, the same arguments that caused a different message to be substituted for Keystone's v2.0 API could be used here. Is that inaccurate? Has the discussion been carried to the logical conclusion? Should deprecated be used yet? I just want to point out that Keystone followed Nova's lead on this one - Keystone's XML support, which is only rigorously tested against a very small portion of the v2 API, is deprecated as of Icehouse as well. I think XML is a broader community issue, not one that needs to be discussed per-project. In terms of the the project-wide issue, the TC at least formalized that the only thing we expect from a project is a REST API with JSON support. We didn't want to *prevent* XML support. http://git.openstack.org/cgit/openstack/governance/tree/reference/incubation-integration-requirements#n40 Project must have a REST API with at least a JSON entity representation I think we could follow a similar pattern for XML in the Compute API v2 -- we need to define discussed with stakeholders and how/when we say the discussion is done. I wanted to put up a set of ideas for the channels (push) and inputs (pull): - posted a collaborative statement expressing need to deprecate to the openstack-dev mailing list - posted a request for comment
Re: [openstack-dev] [keystone] [oslo] Using oslo.cache in keystoneclient.middleware.auth_token
dogpile.cache would be substantially lighter on the client-side as it only has a hard dependency on dogpile.core. It supports plenty of backends beyond memcached and we already use it in keystone quite heavily. http://dogpilecache.readthedocs.org/en/latest/ On Mon, Mar 31, 2014 at 11:35 AM, Doug Hellmann doug.hellm...@dreamhost.com wrote: On Mon, Mar 31, 2014 at 12:18 PM, Kurt Griffiths kurt.griffi...@rackspace.com wrote: Hi folks, has there been any discussion on using oslo.cache within the auth_token middleware to allow for using other cache backends besides memcached? I didn’t find a Keystone blueprint for it, and was considering registering one for Juno if the team thinks this feature makes sense. I’d be happy to put some time into the implementation. That does make sense. We need to look at the dependency graph between the keystoneclient and oslo.cache, though. It appears the current version of oslo.cache is going to bring in quite a few oslo libraries that we would not want keystoneclient to depend on [1]. Moving the middleware to a separate library would solve that. [1] https://wiki.openstack.org/wiki/Oslo/Dependencies Doug ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [keystone] [horizon] [nova]
FWIW, that issue is tracked here: https://bugs.launchpad.net/keystone/+bug/967832 On Fri, Mar 28, 2014 at 1:02 PM, Ryan Hallisey rhall...@redhat.com wrote: Currently, when you delete a tenant that has 1 or more running instances, the tenant will be deleted without warning and the running instance(s) will be left in place. Should there be a warning and should the admin be allowed to do this? Or should it be left be? Thanks, -Ryan Hallisey ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Ironic][Keystone] Move drivers credentials to Keystone
On Tue, Mar 25, 2014 at 12:49 PM, Jay Pipes jaypi...@gmail.com wrote: On Tue, 2014-03-25 at 17:39 +, Miller, Mark M (EB SW Cloud - RD - Corvallis) wrote: Why not use Barbican? It stores credentials after encrypting them. No reason not to add a Barbican driver as well. If Keystone's /v3/credentials API has any legs, it should be backed by barbican, if not superseded by it completely. Best, -jay ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [All][Keystone] Deprecation of the v2 API
On Tue, Mar 25, 2014 at 5:50 PM, Russell Bryant rbry...@redhat.com wrote: We discussed the deprecation of the v2 keystone API in the cross-project meeting today [1]. This thread is to recap and bring that discussion to some consensus. The issue is that Keystone has marked the v2 API as deprecated in Icehouse: https://blueprints.launchpad.net/keystone/+spec/deprecate-v2-api If you use the API, deployments will get this in their logs: WARNING keystone.openstack.common.versionutils [-] Deprecated: v2 API is deprecated as of Icehouse in favor of v3 API and may be removed in K. The deprecation status is reflected in the API for end users, as well. For example, from the CLI: $ keystone discover Keystone found at http://172.16.12.38:5000/v2.0 - supports version v2.0 (deprecated) here http://172.16.12.38:5000/v2.0/ My proposal is that this deprecation be reverted. Here's why: First, it seems there isn't a common use of deprecated. To me, marking something deprecated means that the deprecated feature: - has been completely replaced by something else - end users / deployers should take action to migrate to the new thing immediately. - The project has provided a documented migration path - the old thing will be removed at a specific date/release Agree on all points. Unfortunately, we have yet to succeed on the documentation front: https://blueprints.launchpad.net/keystone/+spec/document-v2-to-v3-transition The problem with the above is that most OpenStack projects do not support the v3 API yet. From talking to Dolph in the meeting, it sounds like the intention is: - fully support v2, just don't add features - signal to other projects that they should be migrating to v3 Above all else, this was our primary goal: to raise awareness about our path forward, and to identify the non-obvious stakeholders that we needed to work with in order to drop support for v2. With today's discussion as evidence, I think we've succeeded in that regard :) Given that intention, I believe the proper thing to do is to actually leave the API marked as fully supported / stable. Keystone should be working with other OpenStack projects to migrate them to v3. Once that is complete, deprecation can be re-visited. Happy to! Revert deprecation of the v2 API: https://review.openstack.org/#/c/82963/ Although I'd prefer to apply this patch directly to milestone-proposed, so we can continue into Juno with the deprecation in master. In summary, until we have completed v3 support within OpenStack itself, it's premature to mark the API deprecated since that's a signal to end users and deployers that says action is required. Thoughts? [1] http://eavesdrop.openstack.org/meetings/project/2014/project.2014-03-25-21.01.log.html#l-103 -- Russell Bryant ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [keystone] python-keystoneclient unit tests only if python-memcache is installed
FWIW, I opened a bug [1] and proposed a fix [2]. [1]: https://bugs.launchpad.net/python-keystoneclient/+bug/1296794 [2]: https://review.openstack.org/#/c/82527/ On Fri, Mar 21, 2014 at 12:38 AM, Thomas Goirand z...@debian.org wrote: On 03/20/2014 11:48 PM, Dolph Mathews wrote: Yes, those tests are conditionally executed if https://pypi.python.org/pypi/python-memcached/ is installed and if so, memcached is assumed to be accessible on localhost. Unfortunately the test suite doesn't have a sanity check for that following assumption, so the test failures aren't particularly helpful. Oh, ok, thanks! I've added python-memcache *AND* memcached as build-dependency, then everything is back to working! :) Thanks again for the above help, Cheers, Thomas Goirand (zigo) ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [keystone] python-keystoneclient unit tests only if python-memcache is installed
Yes, those tests are conditionally executed if https://pypi.python.org/pypi/python-memcached/ is installed and if so, memcached is assumed to be accessible on localhost. Unfortunately the test suite doesn't have a sanity check for that following assumption, so the test failures aren't particularly helpful. On Thu, Mar 20, 2014 at 10:10 AM, Thomas Goirand z...@debian.org wrote: Hi, I've noticed that if there's python-memcache installed, building the python-keystoneclient (last version: 0.6.0) will produce some unit tests errors (see below the first 2 errors, there's 7 of them in total). If I apt-get purge python-memcache, the errors don't show. I'm worried that this is the symptoms of a problem with some backends. Can someone confirm if there's a real problem or if I should just declare a build-conflict? Cheers, Thomas Goirand (zigo) == FAIL: keystoneclient.tests.test_auth_token_middleware.v2AuthTokenMiddlewareTest.test_encrypt_cache_data keystoneclient.tests.test_auth_token_middleware.v2AuthTokenMiddlewareTest.test_encrypt_cache_data -- testtools.testresult.real._StringException: Traceback (most recent call last): File /home/zigo/sources/openstack/icehouse/python-keystoneclient/build-area/python-keystoneclient-0.6.0/keystoneclient/tests/test_auth_token_middleware.py, line 784, in test_encryp self.assertEqual(self.middleware._cache_get(token), data[0]) File /usr/lib/python2.7/dist-packages/testtools/testcase.py, line 321, in assertEqual self.assertThat(observed, matcher, message) File /usr/lib/python2.7/dist-packages/testtools/testcase.py, line 406, in assertThat raise mismatch_error MismatchError: None != 'this_data' == FAIL: keystoneclient.tests.test_auth_token_middleware.v2AuthTokenMiddlewareTest.test_no_memcache_protection keystoneclient.tests.test_auth_token_middleware.v2AuthTokenMiddlewareTest.test_no_memcache_protection -- testtools.testresult.real._StringException: Traceback (most recent call last): File /home/zigo/sources/openstack/icehouse/python-keystoneclient/build-area/python-keystoneclient-0.6.0/keystoneclient/tests/test_auth_token_middleware.py, line 815, in test_no_mem self.assertEqual(self.middleware._cache_get(token), data[0]) File /usr/lib/python2.7/dist-packages/testtools/testcase.py, line 321, in assertEqual self.assertThat(observed, matcher, message) File /usr/lib/python2.7/dist-packages/testtools/testcase.py, line 406, in assertThat raise mismatch_error MismatchError: None != 'this_data' ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Nova] Updates to Juno blueprint review process
On Thu, Mar 20, 2014 at 10:49 AM, Russell Bryant rbry...@redhat.com wrote: We recently discussed the idea of using gerrit to review blueprint specifications [1]. There was a lot of support for the idea so we have proceeded with putting this together before the start of the Juno development cycle. We now have a new project set up, openstack/nova-specs. You submit changes to it just like any other project in gerrit. Find the README and a template for specifications here: http://git.openstack.org/cgit/openstack/nova-specs/tree/README.rst http://git.openstack.org/cgit/openstack/nova-specs/tree/template.rst This is great! This is the same basic process we've used for API-impacting changes in keystone and it has worked really well for us, and we're eager to adopt the same thing on a more general level. The process seems overly complicated to me, however. As a blueprint proposer, I find it odd that I have to propose my blueprint as part of approved/ -- why not just have a single directory to file things away that have been implemented? Is it even necessary to preserve them? (why not just git rm when implemented?) Gerrit already provides a permalink (to the review). The blueprint process wiki page has also been updated to reflect that we will be using this for Nova: https://wiki.openstack.org/wiki/Blueprints#Nova Note that *all* Juno blueprints, including ones that were previously approved, must go through this new process. This will help ensure that blueprints previously approved still make sense, as well as ensure that all Juno specs follow a more complete and consistent format. Before the flood of spec reviews start, we would really like to get feedback on the content of the spec template. It includes things like deployer impact which could use more input. Feel free to provide feedback on list, or just suggest updates via proposed changes in gerrit. I suspect this process to evolve a bit throughout Juno, but I'm very excited about the positive impact it is likely to have on our overall result. Thanks! [1] http://lists.openstack.org/pipermail/openstack-dev/2014-March/029232.html -- Russell Bryant ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Heat] [Keystone] Where are the strings in the keystone API's defined?
For posterity, I assume this thread is related to: http://lists.openstack.org/pipermail/openstack-dev/2014-February/028125.html Anyway, keystone itself has issued 36-char tenant ID's in the past (diablo, I believe, if not essex as well). Something like this: $ python -c import uuid; s = str(uuid.uuid4()); print s; print len(s); 1b54024b-7c62-494e-9a34-6138e04e3dc7 36 Migrations to essex also pulled in existing tenant ID's from other pre-existing data sources. Most importantly, keystone is able to read tenants from backends such as LDAP, and you're welcome to write your own driver and return whatever data you want as an ID. From an API perspective, keystone assumes that tenant ID's are URL-safe and globally unique, but does nothing to enforce either of those. Perhaps that's somewhere keystone could start (emit a warning if that's not the case?), before other services make stricter assumptions about the constraints around tenant ID's. Note: all of the above applies equally to user ID's, as well! On Mon, Mar 10, 2014 at 12:53 PM, Clint Byrum cl...@fewbar.com wrote: While looking into this bug: https://bugs.launchpad.net/heat/+bug/1290274 I was trying to find out why the original developers felt that tenant_ids should be a 'varchar(256)'. In addition to moving from a regular varchar into a text field, varchar(256) in MySQL will become a tinytext which causes all sorts of performance issues, this just seems a _lot_ bigger than anything I've ever seen shown as a tenant/project id. I'd expect at worst varchar(64). Also it is probably safe to store as varbinary since we don't ever sort on it or store case-insensitive equivalents to it. So I was wondering where users can go to find out what to expect in that field. I dug through the API documentation for Keystone and I see nothing that really would constituate a format or even length. But maybe I'm just not looking in the right place. Thanks! ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [all][keystone] Increase of USER_ID length maximum from 64 to 255
On Mon, Mar 3, 2014 at 8:48 AM, Jay Pipes jaypi...@gmail.com wrote: On Sun, 2014-03-02 at 12:05 -0800, Morgan Fainberg wrote: Having done some work with MySQL (specifically around similar data sets) and discussing the changes with some former coworkers (MySQL experts) I am inclined to believe the move from varchar to binary absolutely would increase performance like this. However, I would like to get some real benchmarks around it and if it really makes a difference we should get a smart UUID type into the common SQLlibs (can pgsql see a similar benefit? Db2?) I think this conversation. Should be split off from the keystone one at hand - I don't want valuable information / discussions to get lost. No disagreement on either point. However, this should be done after the standardization to a UUID user_id in Keystone, as a separate performance improvement patch. Agree? Best, -jay ++ https://blueprints.launchpad.net/keystone/+spec/sql-id-binary-16 ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [keystone] Notification When Creating/Deleting a Tenant in openstack
On Fri, Feb 28, 2014 at 12:48 PM, Nader Lahouti nader.laho...@gmail.comwrote: The idea behind this when we originally implemented notifications in Keystone was to provide the resource being changed, such as 'user', 'project', 'trust' and the uuid of that resource. From there your plugin and could request more information from Keystone by doing a GET on that resource. This way would could keep the payload of the notification sent minimal in case all the information on the resource wasn't required. The issue is that, notification is send after project is deleted, so no additional information can be fetched (i.e. project name,...). The GET request fails. As there is only project ID is in resource_info in the notification. In my case I at least need the name of the project. What's the use case for needing the name of a project after it's been deleted? It's mutable in keystone anyway, so you certainly shouldn't be keeping references to project names. Thanks, Nader. On Mon, Feb 24, 2014 at 10:50 AM, Lance D Bragstad ldbra...@us.ibm.comwrote: Response below. Best Regards, Lance Bragstad ldbra...@us.ibm.com Nader Lahouti nader.laho...@gmail.com wrote on 02/24/2014 11:31:10 AM: From: Nader Lahouti nader.laho...@gmail.com To: OpenStack Development Mailing List (not for usage questions) openstack-dev@lists.openstack.org, Date: 02/24/2014 11:37 AM Subject: Re: [openstack-dev] [keystone] Notification When Creating/ Deleting a Tenant in openstack Hi Swann, I was able to listen to keystone notification by setting notifications in the keystone.conf file. I only needed the notification (CURD) for project and handle it in my plugin code so don't need ceilometer to handle them. The other issue is that the notification is only for limited to resource_id and don't have other information such as project name. The idea behind this when we originally implemented notifications in Keystone was to provide the resource being changed, such as 'user', 'project', 'trust' and the uuid of that resource. From there your plugin and could request more information from Keystone by doing a GET on that resource. This way would could keep the payload of the notification sent minimal in case all the information on the resource wasn't required. The issue that I'm facing is that GET fails as the project is deleted from database, so cannot get any info from the resource_info in the notification from Thanks, Nader. On Mon, Feb 24, 2014 at 2:10 AM, Swann Croiset swan...@gmail.com wrote: Hi Nader, These notifications must be handled by Ceilometer like others [1]. it is surprising that it does not already identity meters indeed... probably nobody needs them before you. I guess it remains to open a BP and code them like I recently did for Heat [2] http://docs.openstack.org/developer/ceilometer/measurements.html https://blueprints.launchpad.net/ceilometer/+spec/handle-heat-notifications 2014-02-20 19:10 GMT+01:00 Nader Lahouti nader.laho...@gmail.com: Thanks Dolph for link. The document shows the format of the message and doesn't give any info on how to listen to the notification. Is there any other document showing the detail on how to listen or get these notifications ? Regards, Nader. On Feb 20, 2014, at 9:06 AM, Dolph Mathews dolph.math...@gmail.com wrote: Yes, see: http://docs.openstack.org/developer/keystone/event_notifications.html On Thu, Feb 20, 2014 at 10:54 AM, Nader Lahouti nader.laho...@gmail.com wrote: Hi All, I have a question regarding creating/deleting a tenant in openstack (using horizon or CLI). Is there any notification mechanism in place so that an application get informed of such an event? If not, can it be done using plugin to send create/delete notification to an application? Appreciate your suggestion and help. Regards, Nader. ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev
Re: [openstack-dev] [all][keystone] Increase of USER_ID length maximum from 64 to 255
, Dolph Mathews wrote: On Tue, Feb 25, 2014 at 2:38 PM, Jay Pipes jaypi...@gmail.com wrote: On Tue, 2014-02-25 at 11:47 -0800, Morgan Fainberg wrote: For purposes of supporting multiple backends for Identity (multiple LDAP, mix of LDAP and SQL, federation, etc) Keystone is planning to increase the maximum size of the USER_ID field from an upper limit of 64 to an upper limit of 255. This change would not impact any currently assigned USER_IDs (they would remain in the old simple UUID format), however, new USER_IDs would be increased to include the IDP identifier (e.g. USER_ID@@IDP_IDENTIFIER). -1 I think a better solution would be to have a simple translation table only in Keystone that would store this longer identifier (for folks using federation and/or LDAP) along with the Keystone user UUID that is used in foreign key relations and other mapping tables through Keystone and other projects. Morgan and I talked this suggestion through last night and agreed it's probably the best approach, and has the benefit of zero impact on other services, which is something we're obviously trying to avoid. I imagine it could be as simple as a user_id to domain_id lookup table. All we really care about is given a globally unique user ID, which identity backend is the user from? On the downside, it would likely become bloated with unused ephemeral user IDs, so we'll need enough metadata about the mapping to implement a purging behavior down the line. UUIDs are 32 chars long. Its really just uuid@@uuid that pushes us over the 64 character limit. If we can shorten up the IDP_ID we can fit everything in 64 chars (which means only Nova needs to expand its column size) What if we enumerated IDPs by index, from 1000 to or something comparable, and then use the new domain_index (or prot domain id to not be a uuid). Then the above scheme would work and no migration would be required. The only identifiers that would ever be communicated to any non-Keystone OpenStack endpoint would be the UUID user and tenant IDs. There is the obvious concern that projects are utilizing (and storing) the user_id in a field that cannot accommodate the increased upper limit. Before this change is merged in, it is important for the Keystone team to understand if there are any places that would be overflowed by the increased size. I would go so far as to say the user_id and tenant_id fields should be *reduced* in size to a fixed 16-char BINARY or 32-char CHAR field for performance reasons. Lengthening commonly-used and frequently-joined identifier fields is not a good option, IMO. Best, -jay The review that would implement this change in size is https://review.openstack.org/#/c/74214 and is actively being worked on/reviewed. I have already spoken with the Nova team, and a single instance has been identified that would require a migration (that will have a fix proposed for the I3 timeline). If there are any other known locations that would have issues with an increased USER_ID size, or any concerns with this change to USER_ID format, please respond so that the issues/concerns can be addressed. Again, the plan is not to change current USER_IDs but that new ones could be up to 255 characters in length. Cheers, Morgan Fainberg — Morgan Fainberg Principal Software Engineer Core Developer, Keystone m...@metacloud.com ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [all][keystone] Increase of USER_ID length maximum from 64 to 255
On Wed, Feb 26, 2014 at 4:23 AM, Marco Fargetta marco.farge...@ct.infn.itwrote: Hi Morgan, On Tue, Feb 25, 2014 at 11:47:43AM -0800, Morgan Fainberg wrote: For purposes of supporting multiple backends for Identity (multiple LDAP, mix of LDAP and SQL, federation, etc) Keystone is planning to increase the maximum size of the USER_ID field from an upper limit of 64 to an upper limit of 255. This change would not impact any currently assigned USER_IDs (they would remain in the old simple UUID format), however, new USER_IDs would be increased to include the IDP identifier (e.g. USER_ID@@IDP_IDENTIFIER). in this case if a user would access with different systems (e.g. SAML with portal, LDAP with CLI) it is mapped to two different identities inside keystone. Is this correct? If so, is there any way to map an individual person with two identities sharing resources? That's correct - they'd result in different identities and keystone has no means of presuming they are the same person. But I think you answered it yourself, in that they would effectively be sharing resources with themselves, so you'd just have to ensure they had the same authorization on the same projects using both identities. Cheers, Marco There is the obvious concern that projects are utilizing (and storing) the user_id in a field that cannot accommodate the increased upper limit. Before this change is merged in, it is important for the Keystone team to understand if there are any places that would be overflowed by the increased size. The review that would implement this change in size is https:// review.openstack.org/#/c/74214 and is actively being worked on/reviewed. I have already spoken with the Nova team, and a single instance has been identified that would require a migration (that will have a fix proposed for the I3 timeline). If there are any other known locations that would have issues with an increased USER_ID size, or any concerns with this change to USER_ID format, please respond so that the issues/concerns can be addressed. Again, the plan is not to change current USER_IDs but that new ones could be up to 255 characters in length. Cheers, Morgan Fainberg — Morgan Fainberg Principal Software Engineer Core Developer, Keystone m...@metacloud.com ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev -- Eng. Marco Fargetta, PhD Istituto Nazionale di Fisica Nucleare (INFN) Catania, Italy EMail: marco.farge...@ct.infn.it ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [all][keystone] Increase of USER_ID length maximum from 64 to 255
On Tue, Feb 25, 2014 at 2:38 PM, Jay Pipes jaypi...@gmail.com wrote: On Tue, 2014-02-25 at 11:47 -0800, Morgan Fainberg wrote: For purposes of supporting multiple backends for Identity (multiple LDAP, mix of LDAP and SQL, federation, etc) Keystone is planning to increase the maximum size of the USER_ID field from an upper limit of 64 to an upper limit of 255. This change would not impact any currently assigned USER_IDs (they would remain in the old simple UUID format), however, new USER_IDs would be increased to include the IDP identifier (e.g. USER_ID@@IDP_IDENTIFIER). -1 I think a better solution would be to have a simple translation table only in Keystone that would store this longer identifier (for folks using federation and/or LDAP) along with the Keystone user UUID that is used in foreign key relations and other mapping tables through Keystone and other projects. Morgan and I talked this suggestion through last night and agreed it's probably the best approach, and has the benefit of zero impact on other services, which is something we're obviously trying to avoid. I imagine it could be as simple as a user_id to domain_id lookup table. All we really care about is given a globally unique user ID, which identity backend is the user from? On the downside, it would likely become bloated with unused ephemeral user IDs, so we'll need enough metadata about the mapping to implement a purging behavior down the line. The only identifiers that would ever be communicated to any non-Keystone OpenStack endpoint would be the UUID user and tenant IDs. There is the obvious concern that projects are utilizing (and storing) the user_id in a field that cannot accommodate the increased upper limit. Before this change is merged in, it is important for the Keystone team to understand if there are any places that would be overflowed by the increased size. I would go so far as to say the user_id and tenant_id fields should be *reduced* in size to a fixed 16-char BINARY or 32-char CHAR field for performance reasons. Lengthening commonly-used and frequently-joined identifier fields is not a good option, IMO. Best, -jay The review that would implement this change in size is https://review.openstack.org/#/c/74214 and is actively being worked on/reviewed. I have already spoken with the Nova team, and a single instance has been identified that would require a migration (that will have a fix proposed for the I3 timeline). If there are any other known locations that would have issues with an increased USER_ID size, or any concerns with this change to USER_ID format, please respond so that the issues/concerns can be addressed. Again, the plan is not to change current USER_IDs but that new ones could be up to 255 characters in length. Cheers, Morgan Fainberg — Morgan Fainberg Principal Software Engineer Core Developer, Keystone m...@metacloud.com ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [keystone] Notification When Creating/Deleting a Tenant in openstack
Yes, see: http://docs.openstack.org/developer/keystone/event_notifications.html On Thu, Feb 20, 2014 at 10:54 AM, Nader Lahouti nader.laho...@gmail.comwrote: Hi All, I have a question regarding creating/deleting a tenant in openstack (using horizon or CLI). Is there any notification mechanism in place so that an application get informed of such an event? If not, can it be done using plugin to send create/delete notification to an application? Appreciate your suggestion and help. Regards, Nader. ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [keystone] SAML consumption Blueprints
On Thu, Feb 20, 2014 at 4:18 AM, Marco Fargetta marco.farge...@ct.infn.itwrote: Dear all, I am interested to the integration of SAML with keystone and I am analysing the following blueprint and its implementation: https://blueprints.launchpad.net/keystone/+spec/saml-id https://review.openstack.org/#/c/71353/ Looking at the code there is something I cannot undertand. In the code it seems you will use apache httpd with mod_shib (or other alternatives) to parse saml assertion and the code inside keystone will read only the values extrapolated by the front-end server. That's correct (for icehouse development, at least). If this is the case, it is not clear to me why you need to register the IdPs, with its certificate, in keystone using the new federation API. You can filter the IdP in the server so why do you need this extra list? What is the use of the IdP list and the certificate? This reflects our original design, and it has evolved a bit from the original design to be a bit more simplified. With the additional dependency on mod_shib / mod_mellon, we are no longer implementing the certificates API, but we do still need the IdP API. The IdP API specifically allows us to track the source of an identity, and apply the correct authorization mapping (producing the project- and domain-based role assignments that OpenStack is accostomed to to) to the federated attributes coming from mod_shib / mod_mellon. The benefit is that federated identities from one source can have a different level of authorization than those identities from a different source, even if they (theoretically) had the exact same SAML assertions. Is still this implementation open to discussion or the design is frozen for the icehouse release? It is certainly still open to discussion (and the implementation open to review!), but we're past feature proposal freeze; anything that would require new development (beyond what is already in review) will have to wait a few weeks for Juno. Thanks in advance, Marco ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] Keystone working with V3
On Wed, Feb 19, 2014 at 10:21 AM, Vinod Kumar Boppanna vinod.kumar.boppa...@cern.ch wrote: Dear All, I am doing some development in Nova and in this regard, i have to write a code where Nova requests some date through V3 API of keystone. But the keystoneclient is always falling back to V2 (keystoneclient/v3/client.py). Due to this the keystone V3 API which i am using is failing as it is not available in V2. I did a hack to solve it https://review.openstack.org/#/c/74678/1/keystoneclient/v3/client.py But i was told that it is not acceptable doing this way. So, does anybody from Keystone doing any thing on allowing V3 APIs in keystone client (not falling back to V2). If so, please let me know as my code in nova requires this feature and then only i can submit the code to Gerrit. There's another mailing list discussion on the same topic: http://lists.openstack.org/pipermail/openstack-dev/2014-February/026177.html The short of it is that we probably need such a built-in hack for icehouse. I'm going to experiment with your patch today, thanks! Thanks Regards, Vinod Kumar Boppanna ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] Gerrit co-authors and ticket stealing
On Wed, Feb 19, 2014 at 12:33 PM, Dan Prince dpri...@redhat.com wrote: Perhaps one of the lesser know Gerrit features is the ability to overwrite someone else's patchset/review with a new revision. This can be a handy thing for collaboration, or perhaps to make minor edits (spelling fixes for example) to help expedite the review process. Generally I think things are fine and friendly on this front. There are a couple side effect behaviors that can occur. o/ I do this regularly to help authors land their intended changes (hopefully with less frustration than they would otherwise experience). Most frequently, if the only thing holding me back from a +1 / +2 is a few nits, I'll leave some brief review feedback on the current patchset, and submit a subsequent patchset with the nits fixed, and leave a +1 / +2. Things like: Changing the author or adding yourself as a co-author. Changing the original author should almost never happen (I'm not sure that it has). Adding yourself as a co-author is less of an issue, but is also somewhat questionable if for example all you've done is re-worded something or fixed a spelling issue. So long as the original author is in the know here I think it is probably fine to add yourself as a co-author. But making more meaningful changes, even to a commit message should be checked ahead of time so as not to disrupt the intent of the original authors patch IMO. +1 absolutely agree with these guidelines. Continuing the above, when I want to make more meaningful changes, I either A) suggest a pastebin's diff to the author, or B) go ahead and make the changes but ask that the original author review the latest patchset themselves and express a +1 to acknowledge the result. Leaving clear Gerrit feedback on the most recent patchset/commit with a -1 should do just fine in most cases if you would like a meaningful change and aren't closely collaborating (already) on the fix... It has also come to my attention that co-authoring a patch steals the Launchpad ticket. I believe this is something that we should watch closely (and perhaps fix if we can). +1 I used to make a habit of jumping to the bug and assigning the bug back, but depending on your definition of steal (what does it actually impact?), I'm not sure it's worth the effort? Regardless, I'd appreciate it if the LP bot implementing this behavior used the Author (which as you alluded, must be manually revised, e.g. `git commit --amend --author`) on the commit rather than the Committer. Not trying to point the finger at anyone specifically here. I've probably been guilty of clobbering violations and/or accidental ticket stealing myself. We just need to be careful with these more advanced collaborative coding workflows so as not to step on each others toes. Thanks for bringing this up! Gerrit provides for some powerful workflows and I'd love it if the community was more comfortable taking advantage of them. Dan ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Neutron]Do you think tanent_id should be verified
There's an open bug [1] against nova neutron to handle notifications [2] from keystone about such events. I'd love to see that happen during Juno! [1] https://bugs.launchpad.net/nova/+bug/967832 [2] http://docs.openstack.org/developer/keystone/event_notifications.html On Mon, Feb 17, 2014 at 6:35 AM, Yongsheng Gong gong...@unitedstack.comwrote: It is not easy to enhance it. If we check the tenant_id on creation, if should we also to do some job when keystone delete tenant? On Mon, Feb 17, 2014 at 6:41 AM, Dolph Mathews dolph.math...@gmail.comwrote: keystoneclient.middlware.auth_token passes a project ID (and name, for convenience) to the underlying application through the WSGI environment, and already ensures that this value can not be manipulated by the end user. Project ID's (redundantly) passed through other means, such as URLs, are up to the service to independently verify against keystone (or equivalently, against the WSGI environment), but can be directly manipulated by the end user if no checks are in place. Without auth_token in place to manage multitenant authorization, I'd still expect services to blindly trust the values provided in the environment (useful for both debugging the service and alternative deployment architectures). On Sun, Feb 16, 2014 at 8:52 AM, Dong Liu willowd...@gmail.com wrote: Hi stackers: I found that when creating network subnet and other resources, the attribute tenant_id can be set by admin tenant. But we did not verify that if the tanent_id is real in keystone. I know that we could use neutron without keystone, but do you think tenant_id should be verified when we using neutron with keystone. thanks ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] Sent the first batch of invitations to Atlanta's Summit
I just noticed the subject of this email referred to the first batch of invitations -- are there going to be subsequent batches of invites? If so, who was not included in the first batch that will be in subsequent batches? On Tue, Jan 28, 2014 at 2:45 PM, Stefano Maffulli stef...@openstack.orgwrote: A few minutes ago we sent the first batch of invites to people who contributed to any of the official OpenStack programs[1] from 00:00 UTC on April 4, 2014 (Grizzly release day) until present. We'll send more invites *after each milestone* from now on and until feature freeze (March 6th, according to release schedule[2]) IMPORTANT CHANGE Contrary to previous times, the code is a *$600 discount*. If you don't use it before March 22, when registration prices will increase, *you will be charged*. Use it! Now! And apply for the Travel Support Program if you need to: https://wiki.openstack.org/wiki/Travel_Support_Program Cheers, stef [1] http://git.openstack.org/cgit/openstack/governance/tree/reference/programs.yaml [2] https://wiki.openstack.org/wiki/Icehouse_Release_Schedule -- Ask and answer questions on https://ask.openstack.org ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [keystone][all] Keystone V2 and V3 support in icehouse
On Mon, Feb 10, 2014 at 5:23 PM, Frittoli, Andrea (Cloud Services) fritt...@hp.com wrote: Hi, I’m working on a tempest blueprint to make tempest able to run 100% on keystone v3 (or later versions) – the auth version to be used will be available via a configuration switch. The rationale is that Keystone V2 is deprecated in icehouse, V3 being the primary version. Thus it would be good to have (at least) one of the gate jobs running entirely with keystone v3. Much appreciated! Have a link to that bp? There are other components beyond tempest that would need some changes to make this happen. Nova and cinder python bindings work only with keystone v2 [0], and as far as I know all core services work with keystone v2 (at least by default). Is there a plan to support identity v3 there until the end of icehouse? Yes (but maybe not by the end of icehouse)- we'd like to make all other client libraries depend on keystoneclient's library for authentication in the long run. Jamie Lennox has done a ton of great work to prepare keystoneclient for that responsibility during Icehouse. If not I think we may have to consider still supporting v2 in icehouse. v2 should certainly be supported in icehouse; which version other projects default to is up to them, but I'd like to see *all* projects at least defaulting to v3 by the end of Juno. Andrea [0] https://bugs.launchpad.net/python-novaclient/+bug/1262843 -- Andrea Frittoli IaaS Systems Engineering Team HP Cloud ☁ ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [keystone][all] Keystone V2 and V3 support in icehouse
On Tue, Feb 18, 2014 at 10:21 AM, Frittoli, Andrea (HP Cloud) fritt...@hp.com wrote: Hi, thanks for the update Link to the tempest bp I’m working on: https://blueprints.launchpad.net/tempest/+spec/multi-keystone-api-version-tests The update of the python binding to use the keystone binding is targeted for icehouse or juno? Clients are tracked against the same release milestones of the services, so the integration can happen whenever someone wants to tackle it and we can release them when they're ready. andrea *From:* Dolph Mathews [mailto:dolph.math...@gmail.com] *Sent:* 18 February 2014 13:54 *To:* OpenStack Development Mailing List (not for usage questions) *Cc:* Jamie Lennox *Subject:* Re: [openstack-dev] [keystone][all] Keystone V2 and V3 support in icehouse On Mon, Feb 10, 2014 at 5:23 PM, Frittoli, Andrea (Cloud Services) fritt...@hp.com wrote: Hi, I’m working on a tempest blueprint to make tempest able to run 100% on keystone v3 (or later versions) – the auth version to be used will be available via a configuration switch. The rationale is that Keystone V2 is deprecated in icehouse, V3 being the primary version. Thus it would be good to have (at least) one of the gate jobs running entirely with keystone v3. Much appreciated! Have a link to that bp? There are other components beyond tempest that would need some changes to make this happen. Nova and cinder python bindings work only with keystone v2 [0], and as far as I know all core services work with keystone v2 (at least by default). Is there a plan to support identity v3 there until the end of icehouse? Yes (but maybe not by the end of icehouse)- we'd like to make all other client libraries depend on keystoneclient's library for authentication in the long run. Jamie Lennox has done a ton of great work to prepare keystoneclient for that responsibility during Icehouse. If not I think we may have to consider still supporting v2 in icehouse. v2 should certainly be supported in icehouse; which version other projects default to is up to them, but I'd like to see *all* projects at least defaulting to v3 by the end of Juno. Andrea [0] https://bugs.launchpad.net/python-novaclient/+bug/1262843 -- Andrea Frittoli IaaS Systems Engineering Team HP Cloud ☁ ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Neutron]Do you think tanent_id should be verified
keystoneclient.middlware.auth_token passes a project ID (and name, for convenience) to the underlying application through the WSGI environment, and already ensures that this value can not be manipulated by the end user. Project ID's (redundantly) passed through other means, such as URLs, are up to the service to independently verify against keystone (or equivalently, against the WSGI environment), but can be directly manipulated by the end user if no checks are in place. Without auth_token in place to manage multitenant authorization, I'd still expect services to blindly trust the values provided in the environment (useful for both debugging the service and alternative deployment architectures). On Sun, Feb 16, 2014 at 8:52 AM, Dong Liu willowd...@gmail.com wrote: Hi stackers: I found that when creating network subnet and other resources, the attribute tenant_id can be set by admin tenant. But we did not verify that if the tanent_id is real in keystone. I know that we could use neutron without keystone, but do you think tenant_id should be verified when we using neutron with keystone. thanks ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] Interested in attracting new contributors?
On Wed, Feb 12, 2014 at 8:30 AM, Julie Pichon jpic...@redhat.com wrote: I can definitely sympathise with the comment in Stefano's article that there are not enough easy tasks / simple issues for newcomers. There's a lot to learn already when you're starting out (git, gerrit, python, devstack, ...) and simple bugs are so hard to find - something that will take a few minutes to an existing contributor will take much longer for someone who's still figuring out where to get the code from. My counterargument to this is to jump straight into http://review.openstack.org/ (which happens to be publicly available to newcomers). Easy tasks / simple issues (i.e. nits!) are *frequently* cited in code review, and although our community tends to get hung up on seeing them fixed prior merging the patchset in question (sometimes with good reason, sometimes due to arbitrary opinion), that doesn't always happen (for example, it's not worth delaying approval of an important patch to see a typo fixed in an inline comment) and isn't always appropriate (such as, this other thing over here should be refactored). There's a lot of such scenarios where new contributors can quickly find things to contribute, or at lest provide incredibly valuable feedback to the project in the form of reviews! As a bonus, new contributors jumping straight into reviews tend to get up to speed on the code base *much* more quickly than they otherwise would (IMO), as they become directly involved in design discussions, etc. [1] http://opensource.com/business/14/2/analyzing-contributions-to-openstack [2] http://openhatch.org/ [3] http://openhatch.org/+projects/OpenStack%20dashboard%20%28Horizon%29 [4] https://openhatch.org/wiki/Contacting_new_contributors ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [keystone] Integrating with 3rd party DB
On Fri, Feb 7, 2014 at 3:29 AM, Jamie Lennox jamielen...@redhat.com wrote: - Original Message - From: Noorul Islam K M noo...@noorul.com To: Jamie Lennox jamielen...@redhat.com Cc: OpenStack Development Mailing List (not for usage questions) openstack-dev@lists.openstack.org Sent: Friday, 7 February, 2014 7:13:20 PM Subject: Re: [openstack-dev] [keystone] Integrating with 3rd party DB Jamie Lennox jamielen...@redhat.com writes: - Original Message - From: Noorul Islam K M noo...@noorul.com To: Dolph Mathews dolph.math...@gmail.com Cc: OpenStack Development Mailing List (not for usage questions) openstack-dev@lists.openstack.org Sent: Friday, 7 February, 2014 2:00:34 PM Subject: Re: [openstack-dev] [keystone] Integrating with 3rd party DB Dolph Mathews dolph.math...@gmail.com writes: On Thu, Feb 6, 2014 at 6:38 AM, Noorul Islam Kamal Malmiyoda noo...@noorul.com wrote: Hello stackers, We have a database with tables users, projects, roles, etc. Is there any reference implementation or best practices to make keystone use this DB instead of its own? What's the problem you're having? Does the schema in this database differ from what keystone expects? What have you tried so far? I am trying to figure out the best way of integrating keystone with 3rd party database. I have been reading but I would like to get expert opinion on which is the best way of doing it. Regards, Noorul How obscure is this database? If it can integrate with SQLAlchemy then it's going to be relatively trivial and BY FAR the best approach. That database is accessible only using APIs. We have APIs to authenticate users against this DB, read projects to which user has access to, and roles to which user belongs to. If that's not going to work then your only other option is to implement the database as its own backend for each of the managers. If you look through the folders in keystone (identity, credentials etc) you'll see a Driver class for most of them that you will have to implement for your database. There are examples of the sqlalchemy (and some LDAP) backends there that you can work from. I will look at LDAP back-end code. I'd try to avoid the second case if you can avoid it. We've gotten better at keeping the driver interface stable but you're going to have a constant battle keeping interfaces and functionality up to date with the keystone code. So, if I understand correctly, in any case we need to modify keystone code. No, you can write the driver and then load it from outside of keystone via the config file. However you will need to closely look at the Driver calls within keystone, i'm pretty sure that these aren't documented anywhere. ++ It sounds like the data at least loosely follows keystone's existing schema... you could adjust for any differences using SQL views to present the schema that keystone expects. Thank you for the explanation. Regards, Noorul I have been reading https://wiki.openstack.org/wiki/Keystone/Federation/Blueprint but I could not find a open reference implementation for the same. Regards, Noorul ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [PTL] Designating required use upstream code
On Wed, Feb 5, 2014 at 10:22 AM, Thierry Carrez thie...@openstack.orgwrote: (This email is mostly directed to PTLs for programs that include one integrated project) The DefCore subcommittee from the OpenStack board of directors asked the Technical Committee yesterday about which code sections in each integrated project should be designated sections in the sense of [1] (code you're actually needed to run or include to be allowed to use the trademark). That determines where you can run alternate code (think: substitute your own private hypervisor driver) and still be able to call the result openstack. [1] https://wiki.openstack.org/wiki/Governance/CoreDefinition PTLs and their teams are obviously the best placed to define this, so it seems like the process should be: PTLs propose designated sections to the TC, which blesses them, combines them and forwards the result to the DefCore committee. We could certainly leverage part of the governance repo to make sure the lists are kept up to date. Comments, thoughts ? I'm curious about the level of granularity that's envisioned in each definition. Designated sections could be as broad as keystone.* or as narrow as keystone.token.controllers.Auth.validate_token_head(). It could be defined in terms of executables, package paths, or line numbers. The definition is likely to change over time (i.e. per stable release). For example, where support for password-based authentication might have been mandated for an essex deployment, a havana deployment has the ability to remove the password auth plugin and replace it with something else. The definition may also be conditional, and require either A or B. In havana for example, where keystone shipped two stable APIs side by side, I wouldn't expect all deployments to enable both (or even bother to update their paste pipeline from a previous release). -- Thierry Carrez (ttx) ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [keystone] Integrating with 3rd party DB
On Thu, Feb 6, 2014 at 6:38 AM, Noorul Islam Kamal Malmiyoda noo...@noorul.com wrote: Hello stackers, We have a database with tables users, projects, roles, etc. Is there any reference implementation or best practices to make keystone use this DB instead of its own? What's the problem you're having? Does the schema in this database differ from what keystone expects? What have you tried so far? I have been reading https://wiki.openstack.org/wiki/Keystone/Federation/Blueprint but I could not find a open reference implementation for the same. Regards, Noorul ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] keystone-manage db_sync doesn't work if [database] connection points to IPv6 address
Can you open a bug for this at https://bugs.launchpad.net/keystone ? Thanks! On Sun, Feb 2, 2014 at 9:15 AM, Martinx - ジェームズ thiagocmarti...@gmail.comwrote: Guys, I'm trying to install IceHouse-2 in a dual-stacked environment (Ubuntu 14.04) but, keystone-manage db_sync doesn't work if db connection points to a IPv6 address, like this: My /etc/network/interfaces looks like: --- # The loopback network interface auto lo iface lo inet loopback iface lo inet6 loopback auto eth0 # IPv6 iface eth0 inet6 static address 2001:1291::fffa:: netmask 64 gateway 2001:1291::fffa::1 # dns-* options are implemented by the resolvconf package, if installed dns-search domain.com dns-nameservers 2001:4860:4860::8844 # IPv4 iface eth0 inet static address 192.168.XX.100 netmask 24 gateway 192.168.XX.1 # dns-* options are implemented by the resolvconf package, if installed dns-nameservers 8.8.4.4 8.8.8.8 dns-search domain.com --- My /etc/hosts contains: --- 2001:1291::fffa::controller-1.domain.com controller-1 192.168.XX.100 controller-1.domain.com controller-1 --- MySQL binds on both IPv4 and IPv6, my.cnf like this: --- bind-address = :: --- My /etc/keystone/keystone.conf contains: --- connection = mysql:// keystoneUser:keystonep...@controller-1.domain.com/keystone --- So, this way, keystone-manage db_sync does not work but, if I replace keystone.conf connection line into this: --- connection = mysql://keystoneUser:keystonep...@192.168.xx.100/keystone --- It works! Then, after db_sync, I return it back to FQDN, where it resolves to IPv6 address and it works fine... Cheers! Thiago ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [keystone][heat] Migration to keystone v3 API questions
On Sat, Feb 1, 2014 at 12:33 PM, Anne Gentle a...@openstack.org wrote:
On Thu, Jan 23, 2014 at 5:21 AM, Steven Hardy sha...@redhat.com wrote:
Hi all,
I've recently been working on migrating the heat internal interfaces to
use
the keystone v3 API exclusively[1].
This work has mostly been going well, but I've hit a couple of issues
which
I wanted to discuss, so we agree the most appropriate workarounds:
1. keystoneclient v3 functionality not accessible when catalog contains a
v2 endppoint:
In my test environment my keystone endpoint looks like:
http://127.0.0.1:5000/v2.0
And I'd guess this is similar to the majority of real deployments atm?
Yes, I was just researching this for the Ops Guide O'Reilly edition, and
don't see evidence of deployments doing otherwise in their endpoint
definition.
Also I didn't uncover many (any?) deployments going from Identity v2 to v3
yet. Meaning, if they're already running v2, when they upgrade to havana,
they do not move to Identity v3.
So when creating a keystoneclient object I've been doing:
from keystoneclient.v3 import client as kc_v3
v3_endpoint = self.context.auth_url.replace('v2.0', 'v3')
client = kc_v3.Client(auth_url=v3_endpoint, ...
Which, assuming the keystone service has both v2 and v3 API's enabled
works, but any attempt to use v3 functionality fails with 404 because
keystoneclient falls back to using the v2.0 endpoint from the catalog.
So to work around this I do this:
client = kc_v3.Client(auth_url=v3_endpoint, endpoint=v3_endpoint, ...
client.authenticate()
Which results in the v3 features working OK.
So my questions are:
- Is this a reasonable workaround for production environments?
- What is the roadmap for moving keystone endpoints to be version
agnostic?
- Is there work ongoing to make the client smarter in terms of figuring
out
what URL to use (version negotiation or substituting the appropriate
path
when we are in an environment with a legacy v2.0 endpoint..)
I'd like to understand the ramifications of
https://review.openstack.org/#/c/62801/ so I have a few questions:
- If keystone service catalog endpoints become version agnostic, does that
make other project's support of multiple versions of the Identity API
easier?
Yes, because they can discover the versioned API endpoint they need at
runtime (which may differ from that required by another identity client),
rather than requiring additional external configuration (or adding further
bloat to the service catalog; every service that's overloading the service
type with versioning is doing it *terribly* wrong).
- If the client gets smarter, does that automatically let Heat support
Identity v2? Or is more work required in Heat after your blueprint at [1]
is complete?
I saw a brief discussion at project meeting Jan 14 [3] but I didn't see
any questioning of whether it's premature to preclude the use of Identity
v2 in any integrated project.
Can we discuss implications and considerations at the project meeting next
week?
Sure!
Thanks,
Anne
[3]
http://eavesdrop.openstack.org/meetings/project/2014/project.2014-01-14-21.02.log.html
2. Client (CLI) support for v3 API
What is the status re porting keystoneclient to provide access to the v3
functionality on the CLI?
In particular, Heat is moving towards using domains to encapsulate the
in-instance users it creates[2], so administrators will require some way
to
manage users in a non-default domain, e.g to get visibility of what Heat
is
doing in that domain and debug in the event of any issues.
If anyone can provide any BP links or insight that would be much
appreciated!
Thanks,
Steve
[1] https://blueprints.launchpad.net/heat/+spec/keystone-v3-only
[2] https://wiki.openstack.org/wiki/Heat/Blueprints/InstanceUsers
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] Hierarchicical Multitenancy Discussion
CC'd Adam Young Several of us were very much in favor of this around the Folsom release, but we settled on domains as a solution to the most immediate use case (isolation between flat collections of tenants, without impacting the rest of openstack). I don't think it has been discussed much in the keystone community since, but it's still a concept that I'm very much interested in, as it's much more powerful than domains when it comes to issues like granular delegation. On Tue, Jan 28, 2014 at 12:35 PM, Vishvananda Ishaya vishvana...@gmail.comwrote: Hi Everyone, I apologize for the obtuse title, but there isn't a better succinct term to describe what is needed. OpenStack has no support for multiple owners of objects. This means that a variety of private cloud use cases are simply not supported. Specifically, objects in the system can only be managed on the tenant level or globally. The key use case here is to delegate administration rights for a group of tenants to a specific user/role. There is something in Keystone called a “domain” which supports part of this functionality, but without support from all of the projects, this concept is pretty useless. In IRC today I had a brief discussion about how we could address this. I have put some details and a straw man up here: https://wiki.openstack.org/wiki/HierarchicalMultitenancy I would like to discuss this strawman and organize a group of people to get actual work done by having an irc meeting this Friday at 1600UTC. I know this time is probably a bit tough for Europe, so if we decide we need a regular meeting to discuss progress then we can vote on a better time for this meeting. https://wiki.openstack.org/wiki/Meetings#Hierarchical_Multitenancy_Meeting Please note that this is going to be an active team that produces code. We will *NOT* spend a lot of time debating approaches, and instead focus on making something that works and learning as we go. The output of this team will be a MultiTenant devstack install that actually works, so that we can ensure the features we are adding to each project work together. Vish ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] extending keystone identity
On Tue, Jan 28, 2014 at 12:54 PM, Simon Perfer simon.per...@hotmail.comwrote:
Thanks again, Dolph.
First, is there some good documentation on how to write a custom driver?
I'm wondering specifically about how a keystone user-list is mapped to a
specific function in identity/backend/mydriver.py.
I believe it's calling list_users() in your implementation of the Driver
interface (or raising Not Implemented from the Driver abstract base class
itself).
I suppose this mapping is why I was getting the 500 error about the action
not being implemented.
(501 Not Implemented - 500 is for unhandled exceptions)
Secondly, before poking around with writing a custom driver, I was decided
to simply inherit ldap.Identity, as follows:
class Identity(ldap.Identity):
def __init__(self):
super(Identity, self).__init__()
LOG.debug('My authentication module loaded')
def authenticate(self, user_id, password):
LOG.debug('in auth function')
The basic structure of that looks good to me.
def __init__(self, *args, **kwargs):
super(Identity, self).__init__(*args, **kwargs)
When I get a list of users, I never get the debug output.
What debug output are you expecting? The above code snippet doesn't
override list_users(), so I wouldn't expect any output, except what
ldap.Identity already provides.
Further, I removed the authenticate method from the Identity class in
ldap.py and list-users STILL worked.
Unsure how this is possible. It seems we're never hitting the authenticate
method, which is why overridin it in my custom driver doesn't make much of
a difference in reaching my goal for local users.
Correct - list_users() shouldn't require authenticate() ... or vice versa.
Is there another method I'm supposed to be overriding?
Not if you only want to change the behavior of authentication. list_users()
should only called by the administrative API.
I appreciate the help -- I know these are likely silly questions to
seasoned keystone developers.
--
From: dolph.math...@gmail.com
Date: Mon, 27 Jan 2014 22:35:18 -0600
To: openstack-dev@lists.openstack.org
Subject: Re: [openstack-dev] extending keystone identity
From your original email, it sounds like you want to extend the existing
LDAP identity driver implementation, rather than writing a custom driver
from scratch, which is what you've written. The TemplatedCatalog driver
sort of follows that pattern with the KVS catalog driver, although it's not
a spectacular example.
On Mon, Jan 27, 2014 at 9:11 PM, Simon Perfer simon.per...@hotmail.comwrote:
I dug a bit more and found this in the logs:
(keystone.common.wsgi): 2014-01-27 19:07:13,851 WARNING The action you
have requested has not been implemented.
Despite basing my (super simple) code on the SQL or LDAP backends, I must
be doing something wrong.
-- I've placed my backend code in
/usr/share/pyshared/keystone/identity/backends/nicira.py
or /usr/share/pyshared/keystone/common/nicira.py
-- I DO see the my authenticate module loaded in the log
I would appreciate any help in figuring out what I'm missing. Thanks!
--
From: simon.per...@hotmail.com
To: openstack-dev@lists.openstack.org
Date: Mon, 27 Jan 2014 21:58:43 -0500
Subject: Re: [openstack-dev] extending keystone identity
Dolph, I appreciate the response and pointing me in the right direction.
Here's what I have so far:
imports here
CONF = config.CONF
LOG = logging.getLogger(__name__)
class Identity(identity.Driver):
def __init__(self):
super(Identity, self).__init__()
LOG.debug('My authentication module loaded')
def authenticate(self, user_id, password, domain_scope=None):
LOG.debug('in authenticate method')
When I request a user-list via the python-keystoneclient, we never make it
into the authenticate method (as is evident by the missing debug log).
Any thoughts on why I'm not hitting this method?
--
From: dolph.math...@gmail.com
Date: Mon, 27 Jan 2014 18:14:50 -0600
To: openstack-dev@lists.openstack.org
Subject: Re: [openstack-dev] extending keystone identity
_check_password() is a private/internal API, so we make no guarantees
about it's stability. Instead, override the public authenticate() method
with something like this:
def authenticate(self, user_id, password, domain_scope=None):
if user_id in SPECIAL_LIST_OF_USERS:
# compare against value from keystone.conf
pass
else:
return super(CustomIdentityDriver, self).authenticate(user_id,
password, domain_scope)
On Mon, Jan 27, 2014 at 3:27 PM, Simon Perfer simon.per...@hotmail.comwrote:
I'm looking to create a simple Identity driver that will look at
usernames. A small number of specific users should be authenticated by
looking at a hard-coded password in keystone.conf, while any
Re: [openstack-dev] extending keystone identity
_check_password() is a private/internal API, so we make no guarantees about
it's stability. Instead, override the public authenticate() method with
something like this:
def authenticate(self, user_id, password, domain_scope=None):
if user_id in SPECIAL_LIST_OF_USERS:
# compare against value from keystone.conf
pass
else:
return super(CustomIdentityDriver, self).authenticate(user_id,
password, domain_scope)
On Mon, Jan 27, 2014 at 3:27 PM, Simon Perfer simon.per...@hotmail.comwrote:
I'm looking to create a simple Identity driver that will look at
usernames. A small number of specific users should be authenticated by
looking at a hard-coded password in keystone.conf, while any other users
should fall back to LDAP authentication.
I based my original driver on what's found here:
http://waipeng.wordpress.com/2013/09/30/openstack-ldap-authentication/
As can be seen in the github code (
https://raw.github.com/waipeng/keystone/8c18917558bebbded0f9c588f08a84b0ea33d9ae/keystone/identity/backends/ldapauth.py),
there's a _check_password() method which is supposedly called at some point.
I've based my driver on this ldapauth.py file, and created an Identity
class which subclasses sql.Identity. Here's what I have so far:
CONF = config.CONF
LOG = logging.getLogger(__name__)
class Identity(sql.Identity):
def __init__(self):
super(Identity, self).__init__()
LOG.debug('My authentication module loaded')
def _check_password(self, password, user_ref):
LOG.debug('Authenticating via my custom hybrid authentication')
username = user_ref.get('name')
LOG.debug('Username = %s' % username)
I can see from the syslog output that we never enter the _check_password()
function.
Can someone point me in the right direction regarding which function calls
the identity driver? Also, what is the entry function in the identity
drivers? Why wouldn't check_password() be called, as we see in the github /
blog example above?
THANKS!
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] extending keystone identity
From your original email, it sounds like you want to extend the existing
LDAP identity driver implementation, rather than writing a custom driver
from scratch, which is what you've written. The TemplatedCatalog driver
sort of follows that pattern with the KVS catalog driver, although it's not
a spectacular example.
On Mon, Jan 27, 2014 at 9:11 PM, Simon Perfer simon.per...@hotmail.comwrote:
I dug a bit more and found this in the logs:
(keystone.common.wsgi): 2014-01-27 19:07:13,851 WARNING The action you
have requested has not been implemented.
Despite basing my (super simple) code on the SQL or LDAP backends, I must
be doing something wrong.
-- I've placed my backend code in
/usr/share/pyshared/keystone/identity/backends/nicira.py
or /usr/share/pyshared/keystone/common/nicira.py
-- I DO see the my authenticate module loaded in the log
I would appreciate any help in figuring out what I'm missing. Thanks!
--
From: simon.per...@hotmail.com
To: openstack-dev@lists.openstack.org
Date: Mon, 27 Jan 2014 21:58:43 -0500
Subject: Re: [openstack-dev] extending keystone identity
Dolph, I appreciate the response and pointing me in the right direction.
Here's what I have so far:
imports here
CONF = config.CONF
LOG = logging.getLogger(__name__)
class Identity(identity.Driver):
def __init__(self):
super(Identity, self).__init__()
LOG.debug('My authentication module loaded')
def authenticate(self, user_id, password, domain_scope=None):
LOG.debug('in authenticate method')
When I request a user-list via the python-keystoneclient, we never make it
into the authenticate method (as is evident by the missing debug log).
Any thoughts on why I'm not hitting this method?
--
From: dolph.math...@gmail.com
Date: Mon, 27 Jan 2014 18:14:50 -0600
To: openstack-dev@lists.openstack.org
Subject: Re: [openstack-dev] extending keystone identity
_check_password() is a private/internal API, so we make no guarantees
about it's stability. Instead, override the public authenticate() method
with something like this:
def authenticate(self, user_id, password, domain_scope=None):
if user_id in SPECIAL_LIST_OF_USERS:
# compare against value from keystone.conf
pass
else:
return super(CustomIdentityDriver, self).authenticate(user_id,
password, domain_scope)
On Mon, Jan 27, 2014 at 3:27 PM, Simon Perfer simon.per...@hotmail.comwrote:
I'm looking to create a simple Identity driver that will look at
usernames. A small number of specific users should be authenticated by
looking at a hard-coded password in keystone.conf, while any other users
should fall back to LDAP authentication.
I based my original driver on what's found here:
http://waipeng.wordpress.com/2013/09/30/openstack-ldap-authentication/
As can be seen in the github code (
https://raw.github.com/waipeng/keystone/8c18917558bebbded0f9c588f08a84b0ea33d9ae/keystone/identity/backends/ldapauth.py),
there's a _check_password() method which is supposedly called at some point.
I've based my driver on this ldapauth.py file, and created an Identity
class which subclasses sql.Identity. Here's what I have so far:
CONF = config.CONF
LOG = logging.getLogger(__name__)
class Identity(sql.Identity):
def __init__(self):
super(Identity, self).__init__()
LOG.debug('My authentication module loaded')
def _check_password(self, password, user_ref):
LOG.debug('Authenticating via my custom hybrid authentication')
username = user_ref.get('name')
LOG.debug('Username = %s' % username)
I can see from the syslog output that we never enter the _check_password()
function.
Can someone point me in the right direction regarding which function calls
the identity driver? Also, what is the entry function in the identity
drivers? Why wouldn't check_password() be called, as we see in the github /
blog example above?
THANKS!
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
___ OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
___ OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [All] Code proposal deadline for Icehouse
On Thu, Jan 23, 2014 at 4:02 PM, Russell Bryant rbry...@redhat.com wrote: Greetings, Last cycle we had A feature proposal deadline across some projects. This was the date that code associated with blueprints had to be posted for review to make the release. This was in advance of the official feature freeze (merge deadline). Last time this deadline was used by 5 projects across 3 different dates [1]. I would like to add a deadline for this again for Nova. I'm thinking 2 weeks ahead of the feature freeze right now, which would be February 18th. I'm wondering if it's worth coordinating on this so the schedule is less confusing. Thoughts on picking a single date? How's Feb 18? +1 for Feb 18th If this isn't clearly resolved on the list before the next cross-project meeting (Jan 28), let's discuss it then to nail it down. Thanks, [1] https://wiki.openstack.org/wiki/Havana_Release_Schedule -- Russell Bryant ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] new keystone developer
First of all, welcome! As Steve suggested, feel free to ask questions in #openstack-dev ... it seems there's almost always someone online with deep knowledge of keystone. On Wed, Jan 22, 2014 at 8:28 PM, Mario Adessi mario.ade...@live.com wrote: I'd like to begin contributing to the keystone project. Keystone, along with all the other major infrastructure components in OpenStack, is a rather large project. I've read over the developer documentationhttp://docs.openstack.org/developer/keystone/#developers-documentation, but was hoping to get help with some questions. (1) Are there diagrams that describe how various classes, functions, etc. interact with one another? I've seen a few in the past, but they tend to get out of date quickly. At a high level, the application is structured as follows: Paste Pipeline - Routers - Controllers - Managers - Drivers / Backends (SQL, LDAP, KVS, etc) And that's been true since the essex release. There are a few code paths that deviate from this naming convention (such as keystone.auth), but they follow the same basic pattern regardless. Database migrations have relatively flat call stacks. Some tests have a rather challenging inheritance hierarchy that we're working to unwind. (2) What's the best way to debug keystone when editing existing code or adding? Tips from those who do this every day would be greatly appreciated. I feel like I'm one of the remaining few that doesn't use any fancy tools. I write tests, hammer on them repeatedly, and read tracebacks. (3) Is there a way to import large chunks (or, preferably, all) of keystone into iPython? This makes debugging super easy and would fit in nicely with my existing workflow with other projects. Sounds like Yuriy has the answer you're looking for here! (4) Any other tips / tricks to help jumpstart tinkering with code? I always encourage people to jump straight into gerrit and participate in some code reviews. It's the best way to get a sense of the direction of the project as it evolves, and at the end of the day, you'll be much better prepared to produce your own patch(es). Many thanks. -mario ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Keystone] bp proposal: quotas on users and projects per domain
On Thu, Jan 23, 2014 at 9:59 AM, Florent Flament florent.flament-...@cloudwatt.com wrote: Hi, Although it is true that projects and users don't consume a lot of resources, I think that there may be cases where setting quotas (possibly large) may be useful. For instance, a cloud provider may wish to prevent domain administrators to mistakingly create an infinite number of users and/or projects, by calling APIs in a bugging loop. That sounds like it would be better solved by API rate limiting, not quotas. Moreover, if quotas can be disabled, I don't see any reason not to allow cloud operators to set quotas on users and/or projects if they wishes to do so for whatever marketing reason (e.g. charging more to allow more users or projects). That's the shallow business decision I was alluding to, which I don't think we have any reason to support in-tree. Regards, Florent Flament -- *From: *Dolph Mathews dolph.math...@gmail.com *To: *OpenStack Development Mailing List (not for usage questions) openstack-dev@lists.openstack.org *Sent: *Thursday, January 23, 2014 3:09:51 PM *Subject: *Re: [openstack-dev] [Keystone] bp proposal: quotas on users and projects per domain ... why? It strikes me as a rather shallow business decision to limit the number of users or projects in a system, as neither are actually cost-consuming resources. On Thu, Jan 23, 2014 at 6:43 AM, Matthieu Huin matthieu.h...@enovance.com wrote: Hello, I'd be interested in opinions and feedback on the following blueprint: https://blueprints.launchpad.net/keystone/+spec/tenants-users-quotas The idea is to add a mechanism preventing the creation of users or projects once a quota per domain is met. I believe this could be interesting for cloud providers who delegate administrative rights under domains to their customers. I'd like to hear the community's thoughts on this, especially in terms of viability. Many thanks, Matthieu Huin m...@enovance.com http://www.enovance.com eNovance SaS - 10 rue de la Victoire 75009 Paris - France ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [keystoneclient] old keystone-client package on pypi
Ooh, I meant to get this done last week as I agree that keystoneclient needed to see a new release, but it totally slipped my mind. python-keystoneclient 0.4.2 is now available on pypi! https://pypi.python.org/pypi/python-keystoneclient/0.4.2 What's included in the milestone: https://launchpad.net/python-keystoneclient/+milestone/0.4.2 On Mon, Jan 13, 2014 at 7:17 AM, Sylvain Bauza sylvain.ba...@bull.netwrote: Le 13/01/2014 13:49, Thierry Carrez a écrit : Sylvain Bauza wrote: Le 27/12/2013 10:24, Nikolay Starodubtsev a écrit : Hi all, Guys, I want to say that keystoneclient package on pypi is too old. For example it hadn't Client func in keystoneclient/client.py. May be someone can help me with this? Speaking of python-keystoneclient, the latest release is 0.4.1, which is indeed pretty old (Havana release timeframe). Any chance to get a fresher release soon ? The only solution as of now is pointing to the master eggfile, which is really bad... The solution is for the PTL to tag a new version, and then it will appear on PyPI. Thanks Thierry, my question was indeed when a new tag would be delivered (and consequently a package) ? 0.4.1 is 3 months old, and a lot of features have been implemented meanwhile : https://github.com/openstack/python-keystoneclient/compare/0.4.1...master In particular, Climate would use the keystoneclient.client.Client class for automatic discovery of the Keystone API, which is not part of the latest release. -Sylvain ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] Keystone Hashing MD5 to SHA256
On Tue, Jan 7, 2014 at 11:01 AM, Adam Young ayo...@redhat.com wrote: On 01/06/2014 01:10 PM, Jeremy Stanley wrote: On 2014-01-06 10:19:39 -0500 (-0500), Adam Young wrote: If it were as easy as just replaceing hteh hash algorithm, we would have done it a year + ago. I'm guessing you figured that by now. [...] With the lack of In-Reply-To header and not finding any previous messages to the list in the past few months with a similar subject line, I'm lacking some context (so forgive me if I'm off the mark). If the goal is to thwart offline brute-forcing of the hashed data, shouldn't we be talking about switching away from a plain hash to a key derivation function anyway (PBKDF2, bcrypt, scrypt, et cetera)? MD5 is still resistant to preimage and second preimage attacks as far as I've seen, and SHA256 doesn't take too many orders of magnitude more operations to calculate than MD5. Sorry to all for the cryptic (ha) nature of this mail. It was in response to an expired code review: https://review.openstack.org/#/c/61445/ But I thought would benefit from a larger audience. Note that the Hashes in question are not vulnerable to many of the attecks, as they are used primarily on very strict sets of data (the keystone tokens) and only between the keystone clients and servers. It is not possible to create an arbitrary token, hash it, and have that at all usable in Keystone. Which is why MD5 has not been deemed to be a problem for us. I like the Idea of prefixing the hashes with the algorithm, but we still need a way to integrate that in. A specific Keystone server will only use one Hash alrgorithm, since it is useing the Hash as the unique ID for a database lookup. Thus, in order for the clients and server to be in sync, they need a way to agree on the hash algorithm. Identifying it in the hash is probably too late for most uses. ++ the current architecture requires *clients* to perform the hash, and make requests against the server using the hashed token. So, the client needs to know which hash to use, not just communicate the hash it chose to the service (or have the service published hashed tokens?). Ideally, the service would only have to index on a single hash, so it should be able to communicate which algorithm it expects back to clients in order to provide an upgrade path from MD5. ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev -- -Dolph ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [keystone] Changes to keystone-core!
Hello everyone! We've been talking this for a long while, and we finally have a bunch of changes to make to keystone-core all at once. A few people have moved on, the project has grown a bit, and our review queue grows ever longer. As ayoung phrased it in today's keystone meeting, with entirely selfish motivations, we'd like to welcome the following new Guardians of the Gate to keystone-core: + Steve Martinelli (stevemar) + Jamie Lennox (jamielennox) + David Stanek (dstanek) And I'll be removing the following inactive members from keystone-core (which hopefully won't come as a surprise to anyone!): - Andy Smith (termie) - Devin Carlen (devcamcar) - Gabriel Hurley (gabrielhurley) - Joe Heck (heckj) I'll be making these changes today. Thanks to EVERYONE for your contributions! Happy code reviewing, -Dolph ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] Process for proposing patches attached to launchpad bugs?
In the past, I've been able to get authors of bug fixes attached to Launchpad bugs to sign the CLA and submit the patch through gerrit... although, in one case it took quite a bit of time (and thankfully it wasn't a critical fix or anything). This scenario just came up again (example: [1]), so I'm asking preemptively... what if the author is unwilling / unable in signing the CLA and propose through gerrit, or it's a critical bug fix and waiting on an author to go through the CLA process is undesirable for the community? Obviously that's a bit of a fail on our part, but what's the most appropriate expedient way to handle it? Can we propose the patch to gerrit ourselves? If so, who should appear as the --author of the commit? Who should appear as Co-Authored-By, especially when the committer helps to evolve the patch evolves further in review? Alternatively, am I going about this all wrong? Thanks! [1]: https://bugs.launchpad.net/keystone/+bug/1198171/comments/8 -Dolph ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] Incubation Request for Barbican
On Thu, Dec 12, 2013 at 4:48 PM, Morgan Fainberg m...@metacloud.com wrote: On December 12, 2013 at 14:32:36, Dolph Mathews (dolph.math...@gmail.com//dolph.math...@gmail.com) wrote: On Thu, Dec 12, 2013 at 2:58 PM, Adam Young ayo...@redhat.com wrote: On 12/04/2013 08:58 AM, Jarret Raim wrote: While I am all for adding a new program, I think we should only add one if we rule out all existing programs as a home. With that in mind why not add this to the keystone program? Perhaps that may require a tweak to keystones mission statement, but that is doable. I saw a partial answer to this somewhere but not a full one. From our point of view, Barbican can certainly help solve some problems related to identity like SSH key management and client certs. However, there is a wide array of functionality that Barbican will handle that is not related to identity. Some examples, there is some additional detail in our application if you want to dig deeper [1]. * Symmetric key management - These keys are used for encryption of data at rest in various places including Swift, Nova, Cinder, etc. Keys are resources that roll up to a project, much like servers or load balancers, but they have no direct relationship to an identity. * SSL / TLS certificates - The management of certificate authorities and the issuance of keys for SSL / TLS. Again, these are resources rather than anything attached to identity. * SSH Key Management - These could certainly be managed through keystone if we think that¹s the right way to go about it, but from Barbican¹s point of view, these are just another type of a key to be generated and tracked that roll up to an identity. * Client certificates - These are most likely tied to an identity, but again, just managed as resources from a Barbican point of view. * Raw Secret Storage - This functionality is usually used by applications residing on an Cloud. An app can use Barbican to store secrets such as sensitive configuration files, encryption keys and the like. This data belongs to the application rather than any particular user in Keystone. For example, some Rackspace customers don¹t allow their application dev / maintenance teams direct access to the Rackspace APIs. * Boot Verification - This functionality is used as part of the trusted boot functionality for transparent disk encryption on Nova. * Randomness Source - Barbican manages HSMs which allow us to offer a source of true randomness. In short (ha), I would encourage everyone to think of keys / certificates as resources managed by an API in much the same way we think of VMs being managed by the Nova API. A consumer of Barbican (either as an OpenStack service or a consumer of an OpenStack cloud) will have an API to create and manage various types of secrets that are owned by their project. My reason for keeping them separate is more practical: the Keystone team is already somewhat overloaded. I know that a couple of us have interest in contributing to Barbican, the question is time and prioritization. Unless there is some benefit to having both projects in the same program with essentially different teams, I think Barbican should proceed as is. I personally plan on contributing to Barbican. /me puts PTL hat on ++ I don't want Russel's job. Keystone has a fairly narrow mission statement in my mind (come to think of it, I need to propose it to governance..), and that's basically to abstract away the problem of authenticating and authorizing the API users of other openstack services. Everything else, including identity management, key management, key distribution, quotas, etc, is just secondary fodder that we tend to help with along the way... but they should be first class problems in someone else's mind. If we rolled everything together that kind of looks related to keystone under a big keystone program for the sake of organizational tidiness, I know I would be less effective as a PTL and that's a bit disheartening. That said, I'm always happy to help where I can. The long and the short of it is that I can’t argue that Barbican couldn’t be considered a mechanism of “Identity” (in most everything keys end up being a form of Identity, and the management of that would fit nicely under the “Identity Program”). That being said I also can’t argue that Barbican shouldn’t be it’s own top-level program. It comes down to the best fit for OpenStack as a whole. From a deployer standpoint, I don’t think it will make any real difference if Barbican is in Identity or it’s own program. Basically, it’ll be a separate process to run in either case. It will have it’s own rules and quirks. From a developer standpoint, I don’t think it will make a significant difference (besides, perhaps where documentation lies). The contributors to Keystone will contribute (or not) to Barbican and vice-versa based upon interest/time/needs. From a community
Re: [openstack-dev] API spec for OS-NS-ROLES extension
Services already own their own policy enforcement, and therefore own their
own definitions of roles. A service deployment can already require roles
that are prefixed by a specific string (compute-*), and can already map
actual capabilities onto those roles ({compute-create:
role:compute-manager}).
Centralizing or duplicating some aspect of that enforcement into keystone
does nothing for OpenStack, AFAICT.
At this point, if you've somehow changed your proposal as the message below
implies, I'd appreciate a summary of the revision that addresses the
questions and issues that have repeatedly been raised against it and
ignored.
On Wed, Dec 18, 2013 at 11:19 AM, Tiwari, Arvind arvind.tiw...@hp.comwrote:
Hi Adam,
I would like to request you to revisit the below link and provide your
opinion, so that we can move forward and try to find a common ground where
everyone.
https://review.openstack.org/#/c/61897
*Below is my justification for service_id in role model:*
In a public cloud deployment model, service teams (or service deployers)
defines the roles along with other artifacts (service and endpoint) and
they need full control on these artifacts including roles. This way they
can control the life cycle of these artifacts without depending on IAM
service providers. (more details in
https://blueprints.launchpad.net/keystone/+spec/name-spaced-roles)
As an IAM service provider in a public cloud deployment, it is our
responsibility to facilitate them so that they can control full life cycle
of their service specific artifacts. To make it happen we need tight access
control on these artifacts, so that a service deployer accidently or
maliciously not able to mess-up with other services.
To achieve that level fine granularity and to isolate service deployers
from artifacts, we need to associate entity models (service, endpoints and
roles) with a service. This way we can define entity ownership and define
access control policy based on service. Currently, role data model does
not support any association and that is why I am requesting to introduce
some way to associate a role with domain, project and service. This
association also helps to define a namespace for making the role name
globally unique.
Previously I was trying achieve tight linking of roles with service_id and
that might be offending for some community members. Now after much effort
and help from David Chadwick, we have generalized the role model and come
up with generic design, so that it can fit in with every once use case. As
I mentioned in the spec it will be backward compatible so that it won’t
break the existing deployments
I would appreciate if you can revisit the link and provide comments and
suggestions, there might be still some room for improvements and I am open
for them.
Dolph, I would also like you to review the specs, so that we can make some
progress.
Regards,
Arvind
--
-Dolph
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [keystone] domain admin role query
On Wed, Dec 18, 2013 at 12:48 PM, Ravi Chunduru ravi...@gmail.com wrote: Thanks all for the information. I have now v3 policies in place, the issue is that as a domain admin I could not create a project in the domain. I get 403 unauthorized status. I see that when as a 'domain admin' request a token, the response did not have any roles. In the token request, I couldnt specify the project - as we are about to create the project in next step. Specify a domain as the scope to obtain domain-level authorization in the resulting token. See the third example under Scope: https://github.com/openstack/identity-api/blob/master/openstack-identity-api/v3/src/markdown/identity-api-v3.md#scope-scope Here is the complete request/response of all the steps done. https://gist.github.com/kumarcv/8015275 I am assuming its a bug. Please let me know your opinions. Thanks, -Ravi. On Thu, Dec 12, 2013 at 3:00 PM, Henry Nash hen...@linux.vnet.ibm.comwrote: Hi So the idea wasn't the you create a domain with the id of 'domain_admin_id', rather that you create the domain that you plan to use for your admin domain, and then paste its (auto-generated) domain_id into the policy file. Henry On 12 Dec 2013, at 03:11, Paul Belanger paul.belan...@polybeacon.com wrote: On 13-12-11 11:18 AM, Lyle, David wrote: +1 on moving the domain admin role rules to the default policy.json -David Lyle From: Dolph Mathews [mailto:dolph.math...@gmail.com] Sent: Wednesday, December 11, 2013 9:04 AM To: OpenStack Development Mailing List (not for usage questions) Subject: Re: [openstack-dev] [keystone] domain admin role query On Tue, Dec 10, 2013 at 10:49 PM, Jamie Lennox jamielen...@redhat.com wrote: Using the default policies it will simply check for the admin role and not care about the domain that admin is limited to. This is partially a left over from the V2 api when there wasn't domains to worry about. A better example of policies are in the file etc/policy.v3cloudsample.json. In there you will see the rule for create_project is: identity:create_project: rule:admin_required and domain_id:%(project.domain_id)s, as opposed to (in policy.json): identity:create_project: rule:admin_required, This is what you are looking for to scope the admin role to a domain. We need to start moving the rules from policy.v3cloudsample.json to the default policy.json =) Jamie - Original Message - From: Ravi Chunduru ravi...@gmail.com To: OpenStack Development Mailing List openstack-dev@lists.openstack.org Sent: Wednesday, 11 December, 2013 11:23:15 AM Subject: [openstack-dev] [keystone] domain admin role query Hi, I am trying out Keystone V3 APIs and domains. I created an domain, created a project in that domain, created an user in that domain and project. Next, gave an admin role for that user in that domain. I am assuming that user is now admin to that domain. Now, I got a scoped token with that user, domain and project. With that token, I tried to create a new project in that domain. It worked. But, using the same token, I could also create a new project in a 'default' domain too. I expected it should throw authentication error. Is it a bug? Thanks, -- Ravi One of the issues I had this week while using the policy.v3cloudsample.json was I had no easy way of creating a domain with the id of 'admin_domain_id'. I basically had to modify the SQL directly to do it. Any chance we can create a 2nd domain using 'admin_domain_id' via keystone-manage sync_db? -- Paul Belanger | PolyBeacon, Inc. Jabber: paul.belan...@polybeacon.com | IRC: pabelanger (Freenode) Github: https://github.com/pabelanger | Twitter: https://twitter.com/pabelanger ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev -- Ravi ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev -- -Dolph ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [keystone] domain admin role query
On Thu, Dec 12, 2013 at 8:50 AM, Adam Young ayo...@redhat.com wrote: On 12/11/2013 10:11 PM, Paul Belanger wrote: On 13-12-11 11:18 AM, Lyle, David wrote: +1 on moving the domain admin role rules to the default policy.json -David Lyle From: Dolph Mathews [mailto:dolph.math...@gmail.com] Sent: Wednesday, December 11, 2013 9:04 AM To: OpenStack Development Mailing List (not for usage questions) Subject: Re: [openstack-dev] [keystone] domain admin role query On Tue, Dec 10, 2013 at 10:49 PM, Jamie Lennox jamielen...@redhat.com wrote: Using the default policies it will simply check for the admin role and not care about the domain that admin is limited to. This is partially a left over from the V2 api when there wasn't domains to worry about. A better example of policies are in the file etc/policy.v3cloudsample.json. In there you will see the rule for create_project is: identity:create_project: rule:admin_required and domain_id:%(project.domain_id)s, as opposed to (in policy.json): identity:create_project: rule:admin_required, This is what you are looking for to scope the admin role to a domain. We need to start moving the rules from policy.v3cloudsample.json to the default policy.json =) Jamie - Original Message - From: Ravi Chunduru ravi...@gmail.com To: OpenStack Development Mailing List openstack-dev@lists. openstack.org Sent: Wednesday, 11 December, 2013 11:23:15 AM Subject: [openstack-dev] [keystone] domain admin role query Hi, I am trying out Keystone V3 APIs and domains. I created an domain, created a project in that domain, created an user in that domain and project. Next, gave an admin role for that user in that domain. I am assuming that user is now admin to that domain. Now, I got a scoped token with that user, domain and project. With that token, I tried to create a new project in that domain. It worked. But, using the same token, I could also create a new project in a 'default' domain too. I expected it should throw authentication error. Is it a bug? Thanks, -- Ravi One of the issues I had this week while using the policy.v3cloudsample.json was I had no easy way of creating a domain with the id of 'admin_domain_id'. I basically had to modify the SQL directly to do it. You should not have to edit the SQL. You should be able, at a minimum, to re-enable the ADMIN_TOKEN in the config file to create any object inside of Keystone. open a bug for the problem, and describe what you did step by step? Any chance we can create a 2nd domain using 'admin_domain_id' via keystone-manage sync_db? I totally forgot about this piece -- this is just another incarnation of this bug at the domain level which we should avoid furthering: https://bugs.launchpad.net/keystone/+bug/968696 But, to answer your question: no. It's intended to be a placeholder in the policy file for an actual domain ID (modify the policy file, don't hack at the SQL backend). ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev -- -Dolph ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Keystone] policy has no effect because of hard coded assert_admin?
The policy file is protecting v3 API calls at the controller layer, but you're calling the v2 API. The policy decorators should be moved to the manager layer to protect both APIs equally... but we'd have to be very careful not to break deployments depending on the trivial assert_admin behavior (hence the reason we only wrapped v3 with the new policy decorators). On Thu, Dec 12, 2013 at 1:41 AM, Qiu Yu unic...@gmail.com wrote: Hi, I was trying to fine tune some keystone policy rules. Basically I want to grant create_project action to user in ops role. And following are my steps. 1. Adding a new user usr1 2. Creating new role ops 3. Granting this user a ops role in service tenant 4. Adding new lines to keystone policy file ops_required: [[role:ops]], admin_or_ops: [[rule:admin_required], [rule:ops_required]], 5. Change identity:create_project: [[rule:admin_required]], to identity:create_project: [[rule:admin_or_ops]], 6. Restart keystone service keystone tenant-create with credential of user usr1 still returns 403 Forbidden error. “You are not authorized to perform the requested action, admin_required. (HTTP 403)” After some quick scan, it seems that create_project function has a hard-coded assert_admin call[1], which does not respect settings in the policy file. Any ideas why? Is it a bug to fix? Thanks! BTW, I'm running keystone havana release with V2 API. [1] https://github.com/openstack/keystone/blob/master/keystone/identity/controllers.py#L105 Thanks, -- Qiu Yu ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev -- -Dolph ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] How to best make User Experience a priority in every project
On Wed, Dec 11, 2013 at 4:25 PM, Stefano Maffulli stef...@openstack.orgwrote: On 12/06/2013 02:19 AM, Jaromir Coufal wrote: We are growing. At the moment we are 4 core members and others are coming in. But honestly, contributors are not coming to specific projects - they go to reach UX community in a sense - OK this is awesome effort, how can I help? What can I work on? It seems to me that from the comments in the thread, we may have these fresh energies directed at reviewing code from the UX perspective. Do you think that code reviews across all projects are something in scope for the UX team? If so, how do you think we can make it easier for the UX team to discover reviews that may require comments? Unfortunately, that's probably most patches. However, I imagine most commits with DocImpact also have very obvious UX impact - so I'd start by directing attention to those patches before they merge. /stef -- Ask and answer questions on https://ask.openstack.org ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev -- -Dolph ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] Incubation Request for Barbican
On Thu, Dec 12, 2013 at 2:58 PM, Adam Young ayo...@redhat.com wrote: On 12/04/2013 08:58 AM, Jarret Raim wrote: While I am all for adding a new program, I think we should only add one if we rule out all existing programs as a home. With that in mind why not add this to the keystone program? Perhaps that may require a tweak to keystones mission statement, but that is doable. I saw a partial answer to this somewhere but not a full one. From our point of view, Barbican can certainly help solve some problems related to identity like SSH key management and client certs. However, there is a wide array of functionality that Barbican will handle that is not related to identity. Some examples, there is some additional detail in our application if you want to dig deeper [1]. * Symmetric key management - These keys are used for encryption of data at rest in various places including Swift, Nova, Cinder, etc. Keys are resources that roll up to a project, much like servers or load balancers, but they have no direct relationship to an identity. * SSL / TLS certificates - The management of certificate authorities and the issuance of keys for SSL / TLS. Again, these are resources rather than anything attached to identity. * SSH Key Management - These could certainly be managed through keystone if we think that¹s the right way to go about it, but from Barbican¹s point of view, these are just another type of a key to be generated and tracked that roll up to an identity. * Client certificates - These are most likely tied to an identity, but again, just managed as resources from a Barbican point of view. * Raw Secret Storage - This functionality is usually used by applications residing on an Cloud. An app can use Barbican to store secrets such as sensitive configuration files, encryption keys and the like. This data belongs to the application rather than any particular user in Keystone. For example, some Rackspace customers don¹t allow their application dev / maintenance teams direct access to the Rackspace APIs. * Boot Verification - This functionality is used as part of the trusted boot functionality for transparent disk encryption on Nova. * Randomness Source - Barbican manages HSMs which allow us to offer a source of true randomness. In short (ha), I would encourage everyone to think of keys / certificates as resources managed by an API in much the same way we think of VMs being managed by the Nova API. A consumer of Barbican (either as an OpenStack service or a consumer of an OpenStack cloud) will have an API to create and manage various types of secrets that are owned by their project. My reason for keeping them separate is more practical: the Keystone team is already somewhat overloaded. I know that a couple of us have interest in contributing to Barbican, the question is time and prioritization. Unless there is some benefit to having both projects in the same program with essentially different teams, I think Barbican should proceed as is. I personally plan on contributing to Barbican. /me puts PTL hat on ++ I don't want Russel's job. Keystone has a fairly narrow mission statement in my mind (come to think of it, I need to propose it to governance..), and that's basically to abstract away the problem of authenticating and authorizing the API users of other openstack services. Everything else, including identity management, key management, key distribution, quotas, etc, is just secondary fodder that we tend to help with along the way... but they should be first class problems in someone else's mind. If we rolled everything together that kind of looks related to keystone under a big keystone program for the sake of organizational tidiness, I know I would be less effective as a PTL and that's a bit disheartening. That said, I'm always happy to help where I can. Keystone plays a critical role for us (as it does with every service) in authenticating the user to a particular project and storing the roles that the user has for that project. Barbican then enforces these restrictions. However, keys / secrets are fundamentally divorced from identities in much the same way that databases in Trove are, they are owned by a project, not a particular user. Hopefully our thought process makes some sense, let me know if I can provide more detail. Jarret [1] https://wiki.openstack.org/wiki/Barbican/Incubation ___ OpenStack-dev mailing listOpenStack-dev@lists.openstack.orghttp://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev -- -Dolph ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org
Re: [openstack-dev] [bugs] definition of triaged
On Thu, Dec 12, 2013 at 3:46 PM, Robert Collins robe...@robertcollins.netwrote: Hi, I'm trying to overhaul the bug triage process for nova (initially) to make it much lighter and more effective. I'll be sending a more comprehensive mail shortly but one thing that has been giving me pause is this: Confirmed The bug was reproduced or confirmed as a genuine bug Triaged The bug comments contain a full analysis on how to properly fix the issue From wiki.openstack.org/wiki/Bugs Putting aside the difficulty of complete reproduction sometimes, I don't understand the use of Triaged here. In LP they mean: Confirmed Verified by someone other than the reporter. Triaged Verified by the bug supervisor. So our meaning is very divergent. I'd like us to consolidate on the standard meaning - which is that the relative priority of having a doctor [developer] attack the problem has been assessed. Specifically: - we should use Triaged to indicate that: - we have assigned a priority - we believe it's a genuine bug - we have routed[tagged] it to what is probably the right place ++ that's exactly how I use it, with some emphasis on believe which I use to differentiate from this from Confirmed ... [vendor driver/low-hanging-fruit etc] - we should use Incomplete if we aren't sure that its a bug and need the reporter to tell us more to be sure - triagers shouldn't ever set 'confirmed' - thats reserved solely for end users to tell us that more than one user is encountering the problem. As a triager, if I put my user hat on and am able to reproduce a bug, I'll mark Confirmed, otherwise it's just Triaged. -Rob -- Robert Collins rbtcoll...@hp.com Distinguished Technologist HP Converged Cloud ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev -- -Dolph ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Keystone] policy has no effect because of hard coded assert_admin?
On Thu, Dec 12, 2013 at 12:40 PM, Morgan Fainberg m...@metacloud.com wrote: As Dolph stated, V3 is where the policy file protects. This is one of the many reasons why I would encourage movement to using V3 Keystone over V2. The V2 API is officially deprecated in the Icehouse cycle, I think that moving the decorator potentially could cause more issues than not as stated for compatibility. I would be very concerned about breaking compatibility with deployments and maintaining the security behavior with the encouragement to move from V2 to V3. I am also not convinced passing the context down to the manager level is the right approach. Making a move on where the protection occurs likely warrants a deeper discussion (perhaps in Atlanta?). ++ I *should* have written could be moved to the manager layer. I don't actually think they should, at least at the moment. With v2.0 gone, it would be a more interesting, more approachable discussion. Cheers, Morgan Fainberg On December 12, 2013 at 10:32:40, Dolph Mathews (dolph.math...@gmail.com//dolph.math...@gmail.com) wrote: The policy file is protecting v3 API calls at the controller layer, but you're calling the v2 API. The policy decorators should be moved to the manager layer to protect both APIs equally... but we'd have to be very careful not to break deployments depending on the trivial assert_admin behavior (hence the reason we only wrapped v3 with the new policy decorators). On Thu, Dec 12, 2013 at 1:41 AM, Qiu Yu unic...@gmail.com wrote: Hi, I was trying to fine tune some keystone policy rules. Basically I want to grant create_project action to user in ops role. And following are my steps. 1. Adding a new user usr1 2. Creating new role ops 3. Granting this user a ops role in service tenant 4. Adding new lines to keystone policy file ops_required: [[role:ops]], admin_or_ops: [[rule:admin_required], [rule:ops_required]], 5. Change identity:create_project: [[rule:admin_required]], to identity:create_project: [[rule:admin_or_ops]], 6. Restart keystone service keystone tenant-create with credential of user usr1 still returns 403 Forbidden error. “You are not authorized to perform the requested action, admin_required. (HTTP 403)” After some quick scan, it seems that create_project function has a hard-coded assert_admin call[1], which does not respect settings in the policy file. Any ideas why? Is it a bug to fix? Thanks! BTW, I'm running keystone havana release with V2 API. [1] https://github.com/openstack/keystone/blob/master/keystone/identity/controllers.py#L105 Thanks, -- Qiu Yu ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev -- -Dolph ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev -- -Dolph ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [keystone] domain admin role query
On Tue, Dec 10, 2013 at 10:49 PM, Jamie Lennox jamielen...@redhat.comwrote: Using the default policies it will simply check for the admin role and not care about the domain that admin is limited to. This is partially a left over from the V2 api when there wasn't domains to worry about. A better example of policies are in the file etc/policy.v3cloudsample.json. In there you will see the rule for create_project is: identity:create_project: rule:admin_required and domain_id:%(project.domain_id)s, as opposed to (in policy.json): identity:create_project: rule:admin_required, This is what you are looking for to scope the admin role to a domain. We need to start moving the rules from policy.v3cloudsample.json to the default policy.json =) Jamie - Original Message - From: Ravi Chunduru ravi...@gmail.com To: OpenStack Development Mailing List openstack-dev@lists.openstack.org Sent: Wednesday, 11 December, 2013 11:23:15 AM Subject: [openstack-dev] [keystone] domain admin role query Hi, I am trying out Keystone V3 APIs and domains. I created an domain, created a project in that domain, created an user in that domain and project. Next, gave an admin role for that user in that domain. I am assuming that user is now admin to that domain. Now, I got a scoped token with that user, domain and project. With that token, I tried to create a new project in that domain. It worked. But, using the same token, I could also create a new project in a 'default' domain too. I expected it should throw authentication error. Is it a bug? Thanks, -- Ravi ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev -- -Dolph ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [keystone][heat] ec2tokens, v3 credentials and request signing
On Mon, Dec 9, 2013 at 9:08 PM, Adam Young ayo...@redhat.com wrote: On 12/09/2013 05:34 PM, Steven Hardy wrote: Hi all, I have some queries about what the future of the ec2tokens API is for keystone, context as we're looking to move Heat from a horrible mixture of v2/v3 keystone to just v3, currently I'm not sure we can: - The v3/credentials API allows ec2tokens to be stored (if you create the access/secret key yourself), but it requires admin, which creating an ec2-keypair via the v2 API does not? - There is no v3 interface for validating signed requests like you can via POST v2.0/ec2tokens AFAICT? I thought both of these were accidentally introduced in v3 by including the ec2 middleware in the v3 pipeline by default back in grizzly? Essentially making them undocumented APIs. I haven't tested it myself. - Validating requests signed with ec2 credentials stored via v3/credentials does not work, if you try to use v2.0/ec2tokens, should it? What's the blocker / what doesn't work? So my question is basically, what's the future of ec2tokens, is there some alternative in the pipeline for satisfying the same use-case? The main issues we have in Heat: - We want to continue supporting AWS style signed requests for our cloudformation-compatible API, which is currently done via ec2tokens. - ec2 keypairs are currently the only method of getting a non-expiring credential which we can deploy in-instance, that is no longer possible via the v3 API for the reasons above. As mentioned below, access keys don't necessarily expire, and can be utilized without storing a user identity (you just need to store an OAuth access key secret key). When generating OpenStack tokens, they impersonate they user who delegated authorization. What is the recommended way for us to deploy a (non expiring) credential in an instance (ideally derived from a trust or otherwise role-limited), then use that credential to authenticate against our API? X509. The issue, as I understand it, is that there is no user object to back that credential. You don't have a user to execute the trust. Note that you should not be deriving a credential from a trust, you should be linking a trust to a credential. The KDS code base has a similar problem. We need a longer term credential service for internal components of Open Stack. KDS is going to do it with symmetric keys, which might serve your needs. This is usually done via Kerberos in enterprise deployments. My first thought is that the easiest solution would be to allow trust scoped tokens to optionally be configured to not expire (until we delete the trust when we delete the Heat stack)? Can anyone offer any suggestions on a v3 compatible way to do this? I did start looking at oauth as a possible solution, but it seems the client support is not yet there, and there's no auth middleware we can use for authenticating requests containing oauth credentials, any ideas on the status of this would be most helpful! The current implementation basically requires you to exchange an OAuth access token for an identity v3 token -- we don't have middleware to validate OAuth-signed requests like we do for identity API tokens (auth_token). OAuth is short term delegation, not what you need. Expiration of access tokens is optional: https://github.com/openstack/identity-api/blob/master/openstack-identity-api/v3/src/markdown/identity-api-v3-os-oauth1-ext.md#create-access-token-post-os-oauth1access_token Thanks! Steve ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev -- -Dolph ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [qa][keystone] Keystoneclient tests to tempest
On Sun, Dec 8, 2013 at 5:20 PM, Monty Taylor mord...@inaugust.com wrote: Hi! Thanks - I've been wanting to kill this for a long time. Thanks for starting the discussion... On 12/08/2013 07:26 PM, Brant Knudson wrote: We'd like to get the keystoneclient tests out of keystone. They're serving a useful purpose of catching problems with non-backwards compatible changes in keystoneclient so we still want them run. Problem is they're running at the wrong time -- only on changes to keystone and not changes to keystoneclient. The tests need to be run: When keystoneclient changes - run the tests against the change When the tests change - run the change against the current keystoneclient and also old clients When keystone changes - run the tests against the change with current client You're talking about this: https://review.openstack.org/#/c/41931/ Which is almost ready to go. So here's what I think we need to do to get keystone client tests out of keystone: 1) Figure out where to put the tests - is it tempest or something else? Tempest. They should be moved to tempest. 2) Write up a test and put it there 3) Have a job that when there's a change in the tests it runs against current client lib Don't need most of that - the generalized test clients against stable versions matrix is about to land - as long as tempest has your tests in the scenarios, you'll be fine. 4) Expand the job to also run against old clients - or is there 1 job per version? - what versions? (keystone does master, essex-3, and 0.1.1) - e.g. tox -e master,essex-3,0.1.1 essex and 0.1.1 are both older than dirt. We just removed folsom from the gate. I'd got ahead and remove these. - suggest start with these versions and then consider what to use in future OpenStack has a clear support policy - the gate infrastructure will be covering that. Done! 5) Now we can start adding tests Tempest scenarios. 6) Have a job that when there's a change in keystoneclient it runs these tests against the change 7) When there's a change in keystone, run these tests against the change Yup. Already accounted for. 8) Copy the keystoneclient tests from keystone to the new location -- will require some changes 9) Remove the tests from keystone \o/ 10) Move tests back to keystone where makes sense -- use webtest like v3 tests I created an etherpad with this same info so it's easier to discuss: https://etherpad.openstack.org/p/KeystoneTestsToTempest - Brant ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev THANK YOU TO EVERYONE HELPING TO MAKE THIS HAPPEN! To cross-link, there was also a summit discussion on the topic: https://etherpad.openstack.org/p/icehouse-summit-qa-keystone -Dolph ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Keystoneclient] [Keystone] [Solum] Last released version of keystoneclient does not work with python33
On Wed, Dec 4, 2013 at 7:48 PM, David Stanek dsta...@dstanek.com wrote: On Wed, Dec 4, 2013 at 6:44 PM, Adrian Otto adrian.o...@rackspace.comwrote: Jamie, Thanks for the guidance here. I am checking to see if any of our developers might take an interest in helping with the upstream work. At the very least, it might be nice to have some understanding of how much work there is to be done in HTTPretty. (Dolph correct me if I am wrong, but...) I don't think that there is much work to be done beyond getting that pull request merged upstream. Dolph ran the tests using the code from the pull request somewhat successfully. The errors that we saw were just in keystoneclient code. ++ and the other errors I was hitting all have open patches in gerrit to see them fixed. It didn't seem like we were far off, but I haven't tested all these patches together yet to find out if they're just hiding even more problems. Either way, a py33 test run for keystoneclient will look very different very soon. -- David blog: http://www.traceback.org twitter: http://twitter.com/dstanek www: http://dstanek.com ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev -- -Dolph ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Keystone][Marconi][Oslo] Discoverable home document for APIs (Was: Re: [Nova][Glance] Support of v1 and v2 glance APIs in Nova)
On Mon, Nov 25, 2013 at 4:25 PM, Jamie Lennox jamielen...@redhat.comwrote:
To most of your questions i don't know the answer as the format was in
place before i started with the project. I know that it is similar (though
not exactly the same) as nova's but not where they are documented (as they
are version independent)
I can tell you it looks like:
{
versions: {
values: [
{
status: stable,
updated: 2013-03-06T00:00:00Z,
media-types: [
{
base: application\/json,
type: application\/vnd.openstack.identity-v3+json
},
{
base: application\/xml,
type: application\/vnd.openstack.identity-v3+xml
}
],
id: v3.0,
links: [
{
href: http:\/\/localhost:5000\/v3\/,
rel: self
}
]
},
{
status: stable,
updated: 2013-03-06T00:00:00Z,
media-types: [
{
base: application\/json,
type: application\/vnd.openstack.identity-v2.0+json
},
{
base: application\/xml,
type: application\/vnd.openstack.identity-v2.0+xml
}
],
id: v2.0,
links: [
{
href: http:\/\/localhost:5000\/v2.0\/,
rel: self
},
{
href: http:\/\/docs.openstack.org
\/api\/openstack-identity-service\/2.0\/content\/,
type: text\/html,
rel: describedby
},
{
href: http:\/\/docs.openstack.org
\/api\/openstack-identity-service\/2.0\/identity-dev-guide-2.0.pdf,
type: application\/pdf,
rel: describedby
}
]
}
]
}
}
The above is keystone's unversioned multiple choice response. I just wrote
docs for v3's existing version description response, which is closely based
on the above:
https://review.openstack.org/#/c/60576/
- Original Message -
From: Flavio Percoco fla...@redhat.com
To: OpenStack Development Mailing List (not for usage questions)
openstack-dev@lists.openstack.org
Sent: Monday, 25 November, 2013 6:41:42 PM
Subject: [openstack-dev] [Keystone][Marconi][Oslo] Discoverable home
document for APIs (Was: Re: [Nova][Glance]
Support of v1 and v2 glance APIs in Nova)
On 25/11/13 09:28 +1000, Jamie Lennox wrote:
So the way we have this in keystone at least is that querying GET / will
return all available API versions and querying /v2.0 for example is a
similar result with just the v2 endpoint. So you can hard pin a version
by using the versioned URL.
I spoke to somebody the other day about the discovery process in
services. The long term goal should be that the service catalog contains
unversioned endpoints and that all clients should do discovery. For
keystone the review has been underway for a while now:
https://review.openstack.org/#/c/38414/ the basics of this should be
able to be moved into OSLO for other projects if required.
Did you guys create your own 'home document' language? or did you base
it on some existing format? Is it documented somewhere? IIRC, there's
a thread where part of this was discussed, it was related to horizon.
I'm curious to know what you guys did and if you knew about
JSON-Home[0] when you started working on this.
We used json-home for Marconi v1 and we'd want the client to work in a
'follow your nose' way. Since, I'd prefer OpenStack modules to use the
same language for this, I'm curious to know why - if so - you
created your own spec, what are the benefits and if it's documented
somewhere.
Cheers,
FF
[0] http://tools.ietf.org/html/draft-nottingham-json-home-02
--
@flaper87
Flavio Percoco
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
--
-Dolph
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] Multidomain User Ids
On Sun, Nov 24, 2013 at 9:39 PM, Adam Young ayo...@redhat.com wrote: The #1 pain point I hear from people in the field is that they need to consume read only LDAP but have service users in something Keystone specific. We are close to having this, but we have not closed the loop. This was something that was Henry's to drive home to completion. Do we have a plan? Federation depends on this, I think, but this problem stands alone. I'm still thinking through the idea of having keystone natively federate to itself out of the box, where keystone presents itself as an IdP (primarily for service users). It sounds like a simpler architectural solution than having to shuffle around code paths for both federated identities and local identities. Two Solutions: 1 always require domain ID along with the user id for role assignments. From an API perspective, how? (while still allowing for cross-domain role assignments) 2 provide some way to parse from the user ID what domain it is. I think you meant this one the other way around: Determine the domain given the user ID. I was thinking that we could do something along the lines of 2 where we provide domain specific user_id prefix for example, if there is just one ldpa service, and they wanted to prefix anyting out of ldap with ldap@, then an id would be prefix field from LDAP. And would be configured on a per domain basis. THis would be optional. The weakness is that itbe Log N to determine which Domain a user_id came from. A better approach would be to use a divider, like '@' and then prefix would be the key for a hashtable lookup. Since it is optional, domains could still be stored in SQL and user_ids could be uuids. One problem is if someone comes by later an must use email address as the userid, the @ would mess them up. So The default divider should be something URL safe but no likely to be part of a userid. I realize that it might be impossible to match this criterion. For usernames, sure... but I don't know why anyone would care to use email addresses as ID's. Actually, there might be other reasons to forbid @ signs from IDs, as they look like phishing attempts in URLs. Phishing attempts?? They need to be encoded anyway... ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev -- -Dolph ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [keystone][py3] Usage of httpretty
Looks like there's some recent progress here: https://github.com/gabrielfalcao/HTTPretty/pull/124 On Wed, Nov 20, 2013 at 9:30 PM, Morgan Fainberg m...@metacloud.com wrote: I'd be more willing to toss in and help to maintain/fix appropriately on StackForge if that is needed. Though I am very much hoping upstream can be used. Cheers, Morgan Fainberg On Wed, Nov 20, 2013 at 7:21 PM, Chuck Short chuck.sh...@canonical.com wrote: Hi, So maybe if it gets to the point where it gets too be much of a porblem we should just put it on stackforge. Regards chuck On Wed, Nov 20, 2013 at 9:08 PM, Jamie Lennox jamielen...@redhat.com wrote: Chuck, So it is being used to handle stubbing returns from requests and httplib rather than having to having fake handlers in place in our testing code, or stubbing out the request library and continually having to update the arguments being passed to keep the mocks working. From my looking around it is the best library for this sort of job. When i evalutated it for keystoneclient upstream (https://github.com/gabrielfalcao/HTTPretty/ ) was quickly responsive and had CI tests that seemed to be checking python 3 support. I haven't seen as much happening recently as there are pull requests upstream for python 3 fixes that just don't seem to be moving anywhere. The CI for python 3 was also commented out at some point. It also turns out to be a PITA to package correctly. I attempted this for fedora, and i know there was someone attempting the same for gentoo. I have a pull request upstream that would at least get the dependencies under control. I do not want to go back to stubbing the request library, or having a fake client path that is only used in testing. However I have also noticed it is the cause of at least some of our python 3 problems. If there are other libraries out there that can do the same job we should consider them though i am holding some hope for upstream. Jamie On Wed, 2013-11-20 at 14:27 -0800, Morgan Fainberg wrote: Chuck, The reason to use httpretty is that it handles everything at the socket layer, this means if we change out urllib for requests or some other transport to make HTTP requests to we don't need to refactor every one of the mock/mox subouts to match the exact set of parameters to be passed. Httpretty makes managing this significantly easier (hence was the reasoning to move towards it). Though, I'm sure Jamie Lennox can provide more insight into deeper specifics as he did most of the work to convert it. At least the above is my understanding of the reasoning. --Morgan On Wed, Nov 20, 2013 at 2:08 PM, Dolph Mathews dolph.math...@gmail.com wrote: I don't have a great answer -- do any projects depend on it other than python-keystoneclient? I'm happy to see it removed -- I see the immediate benefit but it's obviously not significant relative to python 3 support. BTW, this exact issue is being tracked here- https://bugs.launchpad.net/python-keystoneclient/+bug/1249165 On Wed, Nov 20, 2013 at 3:28 PM, Chuck Short chuck.sh...@canonical.com wrote: Hi, I was wondering for the reason behind the usage httpretty in python-keystoneclient. It seems to me like a total overkill for a test. It also has some problems with python3 support that is currently blocking python3 porting as well. Regards chuck ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev -- -Dolph ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev -- -Dolph ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] Tool for detecting commonly misspelled words
On Tue, Dec 3, 2013 at 12:46 PM, John Griffith john.griff...@solidfire.comwrote: On Tue, Dec 3, 2013 at 11:38 AM, Russell Bryant rbry...@redhat.com wrote: On 12/03/2013 09:22 AM, Joe Gordon wrote: HI all, Recently I have seen a few patches fixing a few typos. I would like to point out a really nifty tool to detect commonly misspelled words. So next time you want to fix a typo, instead of just fixing a single one you can go ahead and fix a whole bunch. https://github.com/lyda/misspell-check To install it: $ pip install misspellings To use it in your favorite openstack repo: $ git ls-files | grep -v locale | misspellings -f - Sample output: http://paste.openstack.org/show/54354 Are we going to start gating on spellcheck of code and commit messages? :-) NO please (please please please). We have enough grammar reviewers at this point already IMO and I honestly think I might puke if jenkins fails my patch because I didn't put a '.' at the end of my comment line in the code. Spelling and grammar are two totally separate issues, and I think grammar is out of scope here. A non-human gate for basic spelling would be fantastic- faster feedback for the author and fewer wasted cycles by reviewers. I'd much rather see us focus on things like... I dunno... maybe having the code actually work? Effectively communicating the intent and thinking behind the code is just as important as the code itself :) -- Russell Bryant ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev -- -Dolph ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [openstack-tc] Incubation Request for Barbican
On Mon, Dec 2, 2013 at 11:55 AM, Russell Bryant rbry...@redhat.com wrote: On 12/02/2013 12:46 PM, Monty Taylor wrote: On 12/02/2013 11:53 AM, Russell Bryant wrote: * Scope ** Project must have a clear and defined scope This is missing ** Project should not inadvertently duplicate functionality present in other OpenStack projects. If they do, they should have a clear plan and timeframe to prevent long-term scope duplication. ** Project should leverage existing functionality in other OpenStack projects as much as possible I'm going to hold off on diving into this too far until the scope is clarified. I'm not. *snip* Ok, I can't help it now. The list looks reasonable right now. Barbican should put migrating to oslo.messaging on the Icehouse roadmap though. *snip* Yeahhh ... I looked and even though rpc and notifier are imported, they do not appear to be used at all. http://git.openstack.org/cgit/stackforge/barbican/tree/tools/pip-requires It looks like the only item here not in the global requirements is Celery, which is licensed under a 3-clause BSD license. I'd like to address the use of Celery. WTF Barbican has been around for 9 months, which means that it does not predate the work that has become oslo.messaging. It doesn't even try. It uses a completely different thing. The use of celery needs to be replaced with oslo. Full stop. I do not believe it makes any sense to spend further time considering a project that's divergent on such a core piece. Which is a shame - because I think that Barbican is important and fills an important need and I want it to be in. BUT - We don't get to end-run around OpenStack project choices by making a new project on the side and then submitting it for incubation. It's going to be a pile of suck to fix this I'm sure, and I'm sure that it's going to delay getting actually important stuff done - but we deal with too much crazy as it is to pull in a non-oslo messaging and event substrata. Yeah, I'm afraid I agree with Monty here. I didn't really address this because I was trying to just do a first pass and not go too far into the tech bits. I think such a big divergence is going to be a hard sell for a number of reasons. It's a significant dependency that I don't think is justified. Further, it won't work in all of the same environments that OpenStack works in today. You can't use Celery with all of the same messaging transports as oslo.messaging (or the older rpc lib). One example is Qpid. I feel like I'm trying to read past the rant :) so I'd like to stop an ask for clarification on the exact argument being made. Is the *only* reason that celery should not be used in openstack because it is incapable of being deployed against amqp 1.0 brokers (i.e. qpid). I'm really trying to understand if the actual objection is over the use (or not) of oslo (which seems like something the TC should express an opinion on, if that's the case), but rather about limiting OpenStack's deployment options. -- Russell Bryant ___ OpenStack-TC mailing list openstack...@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-tc -- -Dolph ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] tenant or project
On Wed, Nov 27, 2013 at 8:12 AM, Steven Hardy sha...@redhat.com wrote: On Tue, Nov 26, 2013 at 10:17:56PM +1030, Christopher Yeoh wrote: On Mon, Nov 25, 2013 at 7:50 PM, Flavio Percoco fla...@redhat.com wrote: On 24/11/13 12:47 -0500, Doug Hellmann wrote: On Sun, Nov 24, 2013 at 12:08 AM, Morgan Fainberg m...@metacloud.com wrote: In all honesty it doesn't matter which term we go with. As long as we are consistent and define the meaning. I think we can argue intuitive vs non-intuitive in this case unto the ground. I prefer project to tenant, but beyond being a bit of an overloaded term, I really don't think anyone will really notice one way or another as long as everything is using the same terminology. We could call it grouping-of-openstack-things if we wanted to (though I might have to pull some hair out if we go to that terminology). However, with all that in mind, we have made the choice to move toward project (horizon, keystone, OSC, keystoneclient) and have some momentum behind that push (plus newer projects already use the project nomenclature). Making a change back to tenant might prove a worse UX than moving everything else in line (nova I think is the one real major hurdle to get converted over, and deprecation of keystone v2 API). FWIW, ceilometer also uses project in our API (although some of our docs use the terms interchangeably). And, FWIW, Marconi uses project as well. Well project seems to be the way everyone is heading long term. So we'll do this for the Nova V3 API. As others have mentioned, I think the most important this is that we all end up using the same terminology (though with the long life of APIs we're stuck with the both for a few years at least). So, Heat has some work to do as we're still using tenant in various places. However, I've been thinking, why do the APIs requests have to contain the project ID at all? Isn't that something we derive from the token in auth_token (setting X-Project-Id, which we use to set the project in the request context)? +1 Maybe I'm missing something obvious, but at the moment, when you create a heat stack, you specify the tenant ID three times, once in the path, once in the request body, and again in the context. I'm wondering why, and if we can kill the first two? Unless they have different reasons for being, stick with the one in the request context. Users shouldn't have to care about multi-tenancy once they've obtained a scoped token, it should just happen. Clearly this is different for keystone, where the top level of request granularity is Domain not Project, but for all other services, every request is scoped to a Project is it not? Steve ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev -- -Dolph ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [all project] Treating recently seen recheck bugs as critical across the board
On Tue, Nov 26, 2013 at 5:23 AM, Thierry Carrez thie...@openstack.orgwrote: Dolph Mathews wrote: On Mon, Nov 25, 2013 at 8:12 PM, Robert Collins robe...@robertcollins.net mailto:robe...@robertcollins.net wrote: So my proposal is that we make it part of the base hygiene for a project that any recheck bugs being seen (either by elastic-recheck or manual inspection) be considered critical and prioritised above feature work. I agree with the notion here (that fixing transient failures is critically high priority work for the community) -- but marking the bug as critical priority is just a subjective abuse of the priority field. A non-critical bug is not necessarily non-critical work. The critical status should be reserved for issues that are actually non-shippable, catastrophically breaking issues. It's a classic bugtracking dilemma where the Importance field is both used to describe bug impact and priority... while they don't always match. ++ That said, the impact of those bugs, considering potential development activity breakage, *is* quite critical (they all are timebombs which will create future gate fails if not handled at top priority). I generally agree, but I don't think it's fair to say that the impact of a transient is universally a single priority, either. Some transient issues occur more frequently and therefore have higher impact. So I think marking them Critical + tagging them is not that much of an abuse, if we start including the gate impact in our bug Impact assessments. That said, I'm also fine with High+Tag, as long as it triggers the appropriate fast response everywhere. I'm fine with starting them at High, and elevating to Critical as appropriate. Is the idea here to automatically apply a tag + priority as a result of recheck/reverify bug X ? (as long as existing priority isn't overwritten!) -- Thierry Carrez (ttx) ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev -- -Dolph ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Keystone][Marconi][Oslo] Discoverable home document for APIs (Was: Re: [Nova][Glance] Support of v1 and v2 glance APIs in Nova)
On Tue, Nov 26, 2013 at 2:47 AM, Flavio Percoco fla...@redhat.com wrote: On 25/11/13 16:50 -0600, Dolph Mathews wrote: On Mon, Nov 25, 2013 at 2:41 AM, Flavio Percoco fla...@redhat.com wrote: On 25/11/13 09:28 +1000, Jamie Lennox wrote: So the way we have this in keystone at least is that querying GET / will return all available API versions and querying /v2.0 for example is a similar result with just the v2 endpoint. So you can hard pin a version by using the versioned URL. I spoke to somebody the other day about the discovery process in services. The long term goal should be that the service catalog contains unversioned endpoints and that all clients should do discovery. For keystone the review has been underway for a while now: https://review.openstack.org/#/c/38414/ the basics of this should be able to be moved into OSLO for other projects if required. Did you guys create your own 'home document' language? or did you base it on some existing format? Is it documented somewhere? IIRC, there's a thread where part of this was discussed, it was related to horizon. I'm curious to know what you guys did and if you knew about JSON-Home[0] when you started working on this. It looks like our multiple choice response might predate Nottingham's proposal, but not by much. In keystone, it's been stable since I joined the project, midway through the diablo cycle (summer 2011). I don't know any more history than that, but I've CC'd Jorge Williams, who probably knows. I really like Nottingham's approach of adding relational links from the base endpoint, I've been thinking about doing the same for keystone for quite a while. As crazy as it sounds, have you guys considered migrating to Nottingham's approach? It only sounds crazy because I have no idea how to migrate an unversioned endpoint :) Skimming through his proposal, it looks like it might actually be compatible with ours to include side by side? If so, we could support both for a couple releases before moving on. Does this proposal have much traction outside the OpenStack community? (and how much traction does it have within the community already?) We picked this approach because we didn't want to invent it ourselves and this happens to have a well defined RFC as well. If there's something Nottingham's proposal lacks of, I think we could provide some feedback and help making it better. We used json-home for Marconi v1 and we'd want the client to work in a 'follow your nose' way. Since, I'd prefer OpenStack modules to use the same language for this, I'm curious to know why - if so - you created your own spec, what are the benefits and if it's documented somewhere. Then why didn't Marconi follow the lead of one of the other projects? ;) LOOOL, I knew you were going to say that. I think I knew about you guys having something similar but at some point I most have forgotten about it. That being said, the main rationals were: 1) Using something documented and known upstream made more sense and it also helps getting more contributions from the community. 2) We already knew it, which falls back in point 1. Hey, you set yourself up! Seriously though, I welcome the innovation. I completely agree though - standardized version discovery across the ecosystem would be fantastic. All that being said, I don't think it would be very hard to migrate Marconi to something common if we agree that json-home is not good enough for OpenStack. Nonetheless, it'd be a shame not to provide feedback to Mark Nottingham about it. So far, his approach has been good enough for us - but, you know, Marconi is still way too small. Is keystone's home schema spec documented somewhere? It's actually documented as part of the v2.0 API for keystone, although I really believe it should be it's own API specification (e.g. Nottingham's approach): https://github.com/openstack/identity-api/blob/master/openstack-identity-api/v2.0/src/docbkx/wadl/identity-admin.wadl#L185 https://github.com/openstack/identity-api/blob/master/openstack-identity-api/v2.0/src/docbkx/xsd/version.xsd#L35 We use the same basic structure to serve the multiple choice response and versioned endpoints: GET / GET /v2.0/ GET /v3/ However, the multiple choice response contains the details for both versions. Cheers, FF -- @flaper87 Flavio Percoco ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev -- -Dolph ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Keystone][Marconi][Oslo] Discoverable home document for APIs (Was: Re: [Nova][Glance] Support of v1 and v2 glance APIs in Nova)
On Mon, Nov 25, 2013 at 2:41 AM, Flavio Percoco fla...@redhat.com wrote: On 25/11/13 09:28 +1000, Jamie Lennox wrote: So the way we have this in keystone at least is that querying GET / will return all available API versions and querying /v2.0 for example is a similar result with just the v2 endpoint. So you can hard pin a version by using the versioned URL. I spoke to somebody the other day about the discovery process in services. The long term goal should be that the service catalog contains unversioned endpoints and that all clients should do discovery. For keystone the review has been underway for a while now: https://review.openstack.org/#/c/38414/ the basics of this should be able to be moved into OSLO for other projects if required. Did you guys create your own 'home document' language? or did you base it on some existing format? Is it documented somewhere? IIRC, there's a thread where part of this was discussed, it was related to horizon. I'm curious to know what you guys did and if you knew about JSON-Home[0] when you started working on this. It looks like our multiple choice response might predate Nottingham's proposal, but not by much. In keystone, it's been stable since I joined the project, midway through the diablo cycle (summer 2011). I don't know any more history than that, but I've CC'd Jorge Williams, who probably knows. I really like Nottingham's approach of adding relational links from the base endpoint, I've been thinking about doing the same for keystone for quite a while. We used json-home for Marconi v1 and we'd want the client to work in a 'follow your nose' way. Since, I'd prefer OpenStack modules to use the same language for this, I'm curious to know why - if so - you created your own spec, what are the benefits and if it's documented somewhere. Then why didn't Marconi follow the lead of one of the other projects? ;) I completely agree though - standardized version discovery across the ecosystem would be fantastic. Cheers, FF [0] http://tools.ietf.org/html/draft-nottingham-json-home-02 -- @flaper87 Flavio Percoco ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev -- -Dolph ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [all project] Treating recently seen recheck bugs as critical across the board
On Mon, Nov 25, 2013 at 8:12 PM, Robert Collins robe...@robertcollins.netwrote: This has been mentioned in other threads, but I thought I'd call it out and make it an explicit topic. We have over 100 recheck bugs open on http://status.openstack.org/rechecks/ - there is quite a bit of variation in how frequently they are seen :(. In a way thats good, but stuff that have been open for months and not seen are likely noise (in /rechecks). The rest - the ones we see happening are noise in the gate. The lower we can drive the spurious failure rate, the less repetitive analysing a failure will be, and the more obvious new ones will be - it forms a virtuous circle. However, many of these bugs - a random check of the first 5 listed found /none/ that had been triaged - are no prioritised for fixing. So my proposal is that we make it part of the base hygiene for a project that any recheck bugs being seen (either by elastic-recheck or manual inspection) be considered critical and prioritised above feature work. I agree with the notion here (that fixing transient failures is critically high priority work for the community) -- but marking the bug as critical priority is just a subjective abuse of the priority field. A non-critical bug is not necessarily non-critical work. The critical status should be reserved for issues that are actually non-shippable, catastrophically breaking issues. Thoughts? -Rob -- Robert Collins rbtcoll...@hp.com Distinguished Technologist HP Converged Cloud ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev -- -Dolph ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] tenant or project
+1 for using the term project across all services. Projects provide multi-tenant isolation for resources across the cloud. Part of the reason we prefer projects in keystone is that domains conceptually provide multi-tenant isolation within keystone itself, so the overloaded tenant terminology gets really confusing. It's been a slow process to kill the term tenant in keystone, here's the current status: - the v2.0 API is finally being deprecated altogether, which should be the last mentions of tenant - the v3 API speaks in terms of projects - keystoneclient already supports projects from a library perspective (including auth_token) - keystoneclient's CLI is deprecated in favor of openstackclient's CLI, which supports the project terminology if you pass the --identity-api-version=3 flag On Sat, Nov 23, 2013 at 7:48 AM, Anne Gentle annegen...@justwriteclick.comwrote: Project is what Keystone chose to standardize on for their v3 API. Lots of other APIs are affected as you can imagine. Here's a thread http://openstack.markmail.org/thread/wyce6kvkfqexcpuu +1, everything mentioned in that email has now happened :) Anne On Sat, Nov 23, 2013 at 7:07 AM, Nick Chase nch...@mirantis.com wrote: From a purely documentation and explanatory standpoint I vote for project, if we're going to standardize on one or the other. On Nov 23, 2013 7:13 AM, Christopher Yeoh cbky...@gmail.com wrote: Hi, So in the past we've used both tenant and project to refer to the same thing and I think its been a source of confusion for people new to OpenStack. In the Nova code we use both, but at least for the API we've been trying to consistently present to the client tenant (which is the majority usage) rather than project. And then Russell pointed out in https://review.openstack.org/#/c/57612/ that the Keystone uses project in the Keystone V3 API rather than tenant. http://api.openstack.org/api-ref-identity.html#identity-v3 I think that we should be consistent across the openstack projects. From a very quick look at the core openstack projects I think that they mostly use tenant at the moment rather than project. Does this change in Keystone nomenclature signify that we all should be moving to use project rather than tenant in the future (its not too late to do a big a search and replace for the Nova V3 API). And is the plan for Keystone python client to also change to project rather than tenant? Chris ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev -- Anne Gentle annegen...@justwriteclick.com ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev -- -Dolph ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] tenant or project
On Sat, Nov 23, 2013 at 2:27 PM, Caitlin Bestler caitlin.best...@nexenta.com wrote: On November 23, 2013 4:09:49 AM Christopher Yeoh cbky...@gmail.com wrote: Hi, So in the past we've used both tenant and project to refer to the same thing and I think its been a source of confusion for people new to OpenStack. In the Nova code we use both, but at least for the API we've been trying to consistently present to the client tenant (which is the majority usage) rather than project. And then Russell pointed out in https://review.openstack.org/#/c/57612/ that the Keystone uses project in the Keystone V3 API rather than tenant. http://api.openstack.org/api-ref-identity.html#identity-v3 I think that we should be consistent across the openstack projects. From a very quick look at the core openstack projects I think that they mostly use tenant at the moment rather than project. Does this change in Keystone nomenclature signify that we all should be moving to use project rather than tenant in the future (its not too late to do a big a search and replace for the Nova V3 API). And is the plan for Keystone python client to also change to project rather than tenant? The advantage of Tenant over project is that it is far more intuitively obvious that resources and users belong to a single tenant than it is that they belong to a single project. You're making a couple assertions here I'd like to discuss individually... 1) it is more intuitive for resources to belong to a single tenant than it is to belong to a single project Do you have any justification for this? I personally don't find this more intuitive at all, but it's certainly subject. I frequently create new projects to play around with compute instances, etc, and delete the project when I'm done. The term project fits that really well. If we're using the term tenant in this example, it sounds like a I have to provide new billing information every time I want to goof around, which is not appealing at all. - it is more intuitive for user to belong to a single tenant than it is to belong to a single project First of all, I absolutely despise the phrasing users belong to tenants because it's both vague and wrong. Projects do not own or belong to users. Users do not own or belong to projects. One identity can hold explicit authorization in one or more projects via role assignments, group membership, etc. It's a many-to-many relationship. The distinction between tenant and project here is entirely inconsequential because the statement being made is fundamentally flawed to begin with. Companies that are the customers of data centers are likely to have many projects, and want their employees to have access to multiple projects. ++ I think we are better off with a label that creates a clear expectation of one-bill-payer equals one-tenant. This seems to contradict your previous statement entirely. Alternatively, you could have one-bill-payer equals one-domain, where each domain contains multiple cloud projects with distinct sets of cloud resources all being aggregated together for billing purposes. As a customer of a data center, I might be running production, staging, continuous build, and development environments all as separate tenants/projects. Why would you want my billing information 100 times a month? All the data is far simpler with that rule. What data is simpler in what way with what rule? ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev -- -Dolph ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] How to stage client major releases in Gerrit?
On Fri, Nov 22, 2013 at 3:31 AM, Thierry Carrez thie...@openstack.orgwrote: Robert Collins wrote: I don't understand why branches would be needed here *if* the breaking changes don't impact any supported release of OpenStack. Right -- the trick is what does supported mean in that case. When the client libraries were first established as separate deliverables, they came up with a blanket statement that the latest version could always be used with *any* version of openstack you may have. The idea being, if some public cloud was still stuck in pre-diablo times, you could still use the same library to address both this public cloud and the one which was 2 weeks behind Havana HEAD. I'm curious about the historical circumstance of this requirement -- were the services supported master, minus two releases at the time? We're not testing more than two stable releases against the latest clients in CI today, so I find it odd that we still bother with this claim, without demonstrating that it's true. -- Thierry Carrez (ttx) ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev -- -Dolph ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] How to best make User Experience a priority in every project
On Wed, Nov 20, 2013 at 9:09 AM, Thierry Carrez thie...@openstack.orgwrote: Hi everyone, How should we proceed to make sure UX (user experience) is properly taken into account into OpenStack development ? Historically it was hard for UX sessions (especially the ones that affect multiple projects, like CLI / API experience) to get session time at our design summits. This visibility issue prompted the recent request by UX-minded folks to make UX an official OpenStack program. However, as was apparent in the Technical Committee meeting discussion about it yesterday, most of us are not convinced that establishing and blessing a separate team is the most efficient way to give UX the attention it deserves. Ideally, UX-minded folks would get active *within* existing project teams rather than form some sort of counter-power as a separate team. In the same way we want scalability and security mindset to be present in every project, we want UX to be present in every project. It's more of an advocacy group than a program imho. So my recommendation would be to encourage UX folks to get involved within projects and during project-specific weekly meetings to efficiently drive better UX there, as a direct project contributor. If all the UX-minded folks need a forum to coordinate, I think [UX] ML threads and, maybe, a UX weekly meeting would be an interesting first step. ++ UX is an issue at nearly every layer. OpenStack has a huge variety of interfaces, all of which deserve consistent, top tier UX attention and community-wide HIG's-- CLIs, client libraries / language bindings, HTTP APIs, web UIs, messaging and even pluggable driver interfaces. Each type of interface generally caters to a different audience, each with slightly different expectations. There would still be an issue with UX session space at the Design Summit... but that's a well known issue that affects more than just UX: the way our design summits were historically organized (around programs only) made it difficult to discuss cross-project and cross-program issues. To address that, the plan is to carve cross-project space into the next design summit, even if that means a little less topical sessions for everyone else. I'd be happy to contribute a design session to focus on improving UX across the community, and I would certainly attend! Thoughts ? -- Thierry Carrez (ttx) ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev -- -Dolph ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Climate] How we agree to determine that an user has admin rights ?
On Wed, Nov 20, 2013 at 10:24 AM, Yuriy Taraday yorik@gmail.com wrote: On Wed, Nov 20, 2013 at 3:21 PM, Sylvain Bauza sylvain.ba...@bull.netwrote: Yes indeed, that's something coming into my mind. Looking at Nova, I found a context_is_admin policy in policy.json allowing you to say which role is admin or not [1] and is matched in policy.py [2], which itself is called when creating a context [3]. I'm OK copying that, any objections to it ? I would suggest not to copy this stuff from Nova. There's a lot of legacy there and it's based on old openstack.common.policy version. We should rely on openstack.common.policy alone, no need to add more checks here. [1] https://github.com/openstack/nova/blob/master/etc/nova/policy.json#L2 This rule is here just to support [2] https://github.com/openstack/nova/blob/master/nova/policy.py#L116 this, which is used only [3] https://github.com/openstack/nova/blob/master/nova/context.py#L102 here. This is not what I would call a consistent usage of policies. If we need to check access rights to some method, we should use an appropriate decorator or helper method and let it check appropriate policy rule that would contain rule:admin_required, just like in Keystone: https://github.com/openstack/keystone/blob/master/etc/policy.json. ++ Define actual policy rules with a suggested policy.json file, but do NOT hardcode a definition of admin. Allow the deployer to define more granular policy. oslo.policy makes this pretty easy. If you're looking at keystone, be sure to look at how we protect v3 controller methods (which ask the policy engine, does the requestor have authorization to perform this operation?), NOT how we protect v2 controller methods (which ask the policy engine, does the requestor have a magical pre-defined role? regardless of what operation is actually being performed). context.is_admin should not be checked directly from code, only through policy rules. It should be set only if we need to elevate privileges from code. That should be the meaning of it. is_admin is a short sighted and not at all granular -- it needs to die, so avoid imitating it. -- Kind regards, Yuriy. ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev -- -Dolph ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Climate] How we agree to determine that an user has admin rights ?
On Wed, Nov 20, 2013 at 10:52 AM, Yuriy Taraday yorik@gmail.com wrote: Hello, Dolph. On Wed, Nov 20, 2013 at 8:42 PM, Dolph Mathews dolph.math...@gmail.comwrote: On Wed, Nov 20, 2013 at 10:24 AM, Yuriy Taraday yorik@gmail.comwrote: context.is_admin should not be checked directly from code, only through policy rules. It should be set only if we need to elevate privileges from code. That should be the meaning of it. is_admin is a short sighted and not at all granular -- it needs to die, so avoid imitating it. I suggest keeping it in case we need to elevate privileges from code. Can you expand on this point? It sounds like you want to ignore the deployer-specified authorization configuration... In this case we can't rely on roles so just one flag should work fine. As I said before, we should avoid setting or reading is_admin directly from code. It should be set only in context.elevated and read only by admin_required policy rule. Does this sound reasonable? -- Kind regards, Yuriy. ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev -- -Dolph ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [keystone][py3] Usage of httpretty
I don't have a great answer -- do any projects depend on it other than python-keystoneclient? I'm happy to see it removed -- I see the immediate benefit but it's obviously not significant relative to python 3 support. BTW, this exact issue is being tracked here- https://bugs.launchpad.net/python-keystoneclient/+bug/1249165 On Wed, Nov 20, 2013 at 3:28 PM, Chuck Short chuck.sh...@canonical.comwrote: Hi, I was wondering for the reason behind the usage httpretty in python-keystoneclient. It seems to me like a total overkill for a test. It also has some problems with python3 support that is currently blocking python3 porting as well. Regards chuck ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev -- -Dolph ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] Search Project - summit follow up
On Wed, Nov 20, 2013 at 1:06 PM, Dmitri Zimin(e) | StackStorm d...@stackstorm.com wrote: Thanks Terry for highlighting this: Yes, tenant isolation is the must. It's not reflected in the prototype - it queries Solr directly; but the proper implementation will go through the query API service, where ACL will be applied. UX folks are welcome to comment on expected queries. I think the key benefit of cross-resource index over querying DBs is that it saves the clients from implementing complex queries case by case, leaving flexibility to the user. I question the need for this service, as this service **should** very much be dependent on the clients for this functionality. Expecting to query backends directly must be a misunderstanding somewhere... Start with a specification for filtering across all services and advocate for it on both existing and new APIs. -- Dmitri. On Wed, Nov 20, 2013 at 2:27 AM, Thierry Carrez thie...@openstack.orgwrote: Dmitri Zimin(e) | StackStorm wrote: Hi Stackers, The project Search is a service providing fast full-text search for resources across OpenStack services. [...] At first glance this looks slightly scary from a security / tenant isolation perspective. Most search results would be extremely user-specific (and leaking data from one user to another would be catastrophic), so the benefits of indexing (vs. querying DB) would be very limited ? -- Thierry Carrez (ttx) ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev -- -Dolph ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] Propose project story wiki idea
Hmm, I was sort of thinking along the same lines after writing my post-summit summary for keystone: https://gist.github.com/dolph/7366031 Granted this is the first time I've written such a document, I could see this evolving into a regularly updated document on the long term direction that keystone-core is in agreement on. Weekly updates is a bit demanding (our direction isn't necessarily tweaked on a weekly basis and actual progress is already tracked via wishlist bugs and blueprints), but I'd be interested in participating in a formalized cross-project approach to communicating this information. On Wed, Nov 20, 2013 at 5:17 AM, Thierry Carrez thie...@openstack.orgwrote: Boris Pavlovic wrote: The idea of this proposal is that every OpenStack project should have story wiki page. It means to publish every week one short message that contains most interesting updates for the last week, and high level road map for future week. So reading this for 10-15 minutes you can see what changed in project, and get better understanding of high level road map of the project. [...] I like the idea, can be very short updates, I don't think it should be automated (and it doesn't have to be every week if there is nothing to say). Ideally we would have a single forum for all of those, rather than have to fish for each appropriate wiki page. If everyone posted to planet.o.o that would be a start... Some other aggregator or site (like the one Flavio suggested) could also be used. -- Thierry Carrez (ttx) ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev -- -Dolph ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova][heat][[keystone] RFC: introducing request identification
Related BP: Create a unified request identifier https://blueprints.launchpad.net/nova/+spec/cross-service-request-id On Tue, Nov 19, 2013 at 5:04 AM, haruka tanizawa harube...@gmail.comwrote: Hi stackers!! I'd like to ask for your opinions about my idea of identifying request. Challenges == We have no way to know the final result of an API request. Indeed we can continuously get the status of allocated resources, but this is just resource status, not request status. It doesn't matter so much for manual operations. But it does for automated clients like heat. We need request-oriented-status and it should be disclosed to clients. Literally, we need to address two items for it. 1. how to identify request from clients 2. how to disclose status of request to clients Note that this email includes only 1 for initiating the discussion. Also, bp:instance-tasks-api[0] should include both two items above. Proposal: introducing request identification = I'd like to introduce request identification, which is disclosed to clients. There are two characteristics: - request identification is unique ID for each request so that we can identify tell a specific request from others. - request identification is available for clients so that we can enable any after-request-operations like check, retry[1] or cancel[2]. (of course we need to add more logic for check/retry/cancel, but I'm pretty sure that indentifying request is the starting point for these features) In my understandings, main objections should be 'who should generate and maintain such IDs?'. How to implement request identification = There are several options at least. (See also recent discussion at openstack-dev[3]) 1. Enable user to provide his/her own request identification within API request. This should be the simplest way. But providing id from outside looks hard to be accepted. For example, Idempotency for OpenStack API[4]. Or instance-tasks-api enable to user to put his/her own request identification. 2. Use correlation_id in oslo as request identification. correlation_id looks similar concept of request indentification, but correlation_id in nova was already rejected[5]. 3. Enable keystone to generate request identification (we can call it 'request-token', for example). Before sending actual API request to nova-api, client sends a request to keystone to get 'request-token'. -2 Then client sends an actual API request with 'request-token'. Nova-api will consult it to keystone whether it was really generated. Sounds like a auth-token generated by keystone, differences are: [lifecycle] auth-token is used for multiple API requests before it expires, 'request-token' is used for only single API request. [reusing] if the same 'request-token' was specified twice or more times, nova-api simply returns 20x (works like client token in AWS[6]). Keystone needs to maintain 'request-tokens' until they expire. For backward compatibility, actual API request without 'request-token' should work as before. We can consider several options for uniqueness of 'request-token': uuid, any string with uniqueness per tennant, etc. IMO, since adding new implementation to Keystone is a little bit hard work, so implement of 1 is reasonable for me, just idea. Any comments will be appreciated. Sincerely, Haruka Tanizawa [0] https://blueprints.launchpad.net/nova/+spec/instance-tasks-api [1] https://wiki.openstack.org/wiki/Support-retry-with-idempotency [2] https://blueprints.launchpad.net/nova/+spec/cancel-swap-volume [3] http://www.mail-archive.com/openstack-dev@lists.openstack.org/msg09023.html [4] https://blueprints.launchpad.net/nova/+spec/idempotentcy-client-token [5] https://review.openstack.org/#/c/29480/ [6] http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/Run_Instance_Idempotency.html ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev -- -Dolph ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Keystone] Blob in keystone v3 certificate API
It sounds like you're looking for barbican :) https://github.com/stackforge/barbican On Thu, Nov 14, 2013 at 8:55 PM, Nachi Ueno na...@ntti3.com wrote: Hi Keystone guys I'm going to use keystone credentials API to store SSL-VPN certificate. However I have a concern about blob attribute. Since it is really free format. We can't provider validation on the data. Of course, we can write some helper validation function, but users can break it... Also we can't ensure the backward compatibilities with such free format API definitions. (1) IMO, we should not use free format attribute such as blob or arbitrary key,value pairs. (2) Should we use this API as a storage for certificate used in any openstack services? Since it is hard to provider validation on such API, I'm start thinking to have vpn certificate API in neutron. Best Nachi ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev -- -Dolph ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [style] () vs \ continuations
On Wed, Nov 13, 2013 at 6:46 PM, Robert Collins robe...@robertcollins.netwrote: Hi so - in http://docs.openstack.org/developer/hacking/ it has as bullet point 4: Long lines should be wrapped in parentheses in preference to using a backslash for line continuation. I'm seeing in some reviews a request for () over \ even when \ is significantly clearer. I'd like us to avoid meaningless reviewer churn here: can we either: - go with PEP8 which also prefers () but allows \ when it is better - and reviewers need to exercise judgement when asking for one or other - make it a hard requirement that flake8 detects +1 for the non-human approach. My strong recommendation is to go with PEP8 and exercising of judgement. The case that made me raise this is this: folder_exists, file_exists, file_size_in_kb, disk_extents = \ self._path_file_exists(ds_browser, folder_path, file_name) Wrapping that in brackets gets this; folder_exists, file_exists, file_size_in_kb, disk_extents = ( self._path_file_exists(ds_browser, folder_path, file_name)) The root of the problem is that it's a terribly named method with a terrible return value... fix the underlying problem. Which is IMO harder to read - double brackets, but no function call, and no tuple: it's more ambiguous than \. from https://review.openstack.org/#/c/48544/15/nova/virt/vmwareapi/vmops.py Cheers, Rob -- Robert Collins rbtcoll...@hp.com Distinguished Technologist HP Converged Cloud ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev -- -Dolph ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] Using AD for keystone authentication only
You can assign roles to users in keystoneclient ($ keystone help user-role-add) -- the assignment would be persisted in SQL. openstackclient supports assignments to groups as well if you switch to --identity-api-version=3 On Wed, Nov 13, 2013 at 3:08 PM, Avi L aviost...@gmail.com wrote: Oh ok so in this case how does the Active Directory user gets a id , and how do you map the user to a role? Is there any example you can point me to? On Wed, Nov 13, 2013 at 11:24 AM, Dolph Mathews dolph.math...@gmail.comwrote: Yes, that's the preferred approach in Havana: Users and Groups via LDAP, and everything else via SQL. On Wednesday, November 13, 2013, Avi L wrote: Hi, I understand that the LDAP provider in keystone can be used for authenticating a user (i.e validate username and password) , and it also authorize it against roles and tenant. However this requires AD schema modification. Is it possible to use AD only for authentication and then use keystone's native database for roles and tenant lookup? The advantage is that then we don't need to touch the enterprise AD installation. Thanks Al -- -Dolph ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev -- -Dolph ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [heat][keystone] APIs, roles, request scope and admin-ness
On Sat, Nov 2, 2013 at 11:06 AM, Steven Hardy sha...@redhat.com wrote: Hi all, Looking to start a wider discussion, prompted by: https://review.openstack.org/#/c/54651/ https://blueprints.launchpad.net/heat/+spec/management-api https://etherpad.openstack.org/p/heat-management-api Summary - it has been proposed to add a management API to Heat, similar in concept to the admin/public API topology used in keystone. I'm concerned that this may not be a pattern we want to propagate throughout OpenStack, and that for most services, we should have one API to access data, with the scope of the data returned/accessible defined by the roles held by the user (ie making proper use of the RBAC facilities afforded to us via keystone). Agree with the concern; Identity API v3 abandons this topology in favor of more granular access controls (policy.json) on a single API. From an HTTP perspective, API responses should vary according to the token used to access the API. Literally, Vary: X-Auth-Token in HTTP headers. In the current PoC patch, a users admin-ness is derived from the fact that they are accessing a specific endpoint, and that policy did not deny them access to that endpoint. I think this is wrong, and we should use keystone roles to decide the scope of the request. ++ (although use of the word scope here is dangerous, as I think you mean something different from the usual usage?) The proposal seems to consider tenants as the top-level of abstraction, with the next level up being a global service provider admin, but this does not consider the keystone v3 concept of domains [1] v3 also allows domain-level roles to be inherited to all projects owned by that domain, so in effect-- it does (keystone just takes care of it). , or that you may wish to provide some of these admin-ish features to domain-admin users (who will adminster data accross multiple tenants, just like has been proposed), via the public-facing API. It seems like we need a way of scoping the request (via data in the context), based on a heirarchy of admin-ness, like: 1. Normal user I assume normal user has some non-admin role on a project/tenant. 2. Tenant Admin (has admin role in a tenant) 3. Domain Admin (has admin role in all tenants in the domain) As mentioned above, keystone provides a solution to this already that other projects don't need to be aware of. 4. Service Admin (has admin role everywhere, like admin_token for keystone) admin_token is a role-free, identity-free hack. With v3, it's only necessary for bootstrapping keystone if you're not backing to an existing identity store, and can be removed after that. The current is_admin flag which is being used in the PoC patch won't allow this granularity of administrative boundaries to be represented, and splitting admin actions into a separate API will prevent us providing tenant and domain level admin functionality to customers in a public cloud environment. admin should not be a binary thing -- in the real world it's much more blurry. Users have a finite set of roles/attributes, some of which can be delegated, and those roles/attributes grant the user different sets of capabilities. It has been mentioned that in keystone, if you have admin in one tenant, you are admin everywhere, which is a pattern I think we should not follow Good! We're working towards eliminating that, but it's been a long, slow road. Deprecating v2 is one next step in that direction. Building a more powerful policy engine is another. Considering identity management as out of scope keystone folks, what are your thoughts in terms of roadmap to make role assignment (at the request level) scoped to tenants rather than globally applied? That's how all role assignments behave today, except for the magical admin role in keystone where the scope is completely ignored. Because keystone doesn't manage resources that can are owned by tenants/projects like the bulk of OpenStack does (identity management especially). E.g what data can we add to move from X-Roles in auth_token, to expressing roles in multiple tenants and domains? Tokens can only be scoped to a single project or domain, so that's your mapping. All X-Roles apply to the X-Project or X-Domain in context. I don't think we have a good roadmap to support a single authenticated request with multi-project authorization. The best solution I have is to pass an unscoped token that can be rescoped to two or more projects as needed. Trust-based tokens are explicitly scoped already. Basically, I'm very concerned that we discuss this, get a clear roadmap which will work with future keystone admin/role models, and is not a short-term hack which we won't want to maintain long-term. What are peoples thoughts on this? [1]: https://wiki.openstack.org/wiki/Domains ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org
Re: [openstack-dev] sqlalchemy-migrate needs a new release
On Thu, Nov 14, 2013 at 2:55 PM, Matt Riedemann mrie...@linux.vnet.ibm.comwrote: On 11/14/2013 2:43 PM, David Ripton wrote: On 11/11/2013 03:35 PM, David Ripton wrote: I'll volunteer to do this release. I'll wait 24 hours from the timestamp of this email for input first. So, if anyone has opinions about the timing of this release, please speak up. (In particular, I'd like to do a release *before* Matt Riedermann's DB2 support patch https://review.openstack.org/#/c/55572/ lands, just in case it breaks anything. Of course we could do another release shortly after it gets in, to make folks who use DB2 happy.) Update: There's now a 0.8 tag in Git but that release failed to reach PyPI, so please ignore it. Thanks fungi and mordred for helping debug what went wrong. https://review.openstack.org/#/c/56449/ (a one-liner) should fix the problem. Once it gets approved, I will attempt to push 0.8.1. Any particular reason to go with 0.8 rather than 0.7.3 as a bug fix release? As an outside observer, I had assumed that the version number was (at least in part) indicating compatibility with the corresponding release of sqlalchemy itself (current stable is 0.8.x). -- Thanks, Matt Riedemann ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev -- -Dolph ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [heat][keystone] APIs, roles, request scope and admin-ness
On Thu, Nov 14, 2013 at 11:43 AM, Steven Hardy sha...@redhat.com wrote: On Thu, Nov 14, 2013 at 10:20:02AM -0600, Dolph Mathews wrote: On Sat, Nov 2, 2013 at 11:06 AM, Steven Hardy sha...@redhat.com wrote: Hi all, Looking to start a wider discussion, prompted by: https://review.openstack.org/#/c/54651/ https://blueprints.launchpad.net/heat/+spec/management-api https://etherpad.openstack.org/p/heat-management-api Summary - it has been proposed to add a management API to Heat, similar in concept to the admin/public API topology used in keystone. I'm concerned that this may not be a pattern we want to propagate throughout OpenStack, and that for most services, we should have one API to access data, with the scope of the data returned/accessible defined by the roles held by the user (ie making proper use of the RBAC facilities afforded to us via keystone). Agree with the concern; Identity API v3 abandons this topology in favor of more granular access controls (policy.json) on a single API. From an HTTP perspective, API responses should vary according to the token used to access the API. Literally, Vary: X-Auth-Token in HTTP headers. In the current PoC patch, a users admin-ness is derived from the fact that they are accessing a specific endpoint, and that policy did not deny them access to that endpoint. I think this is wrong, and we should use keystone roles to decide the scope of the request. ++ (although use of the word scope here is dangerous, as I think you mean something different from the usual usage?) I was using scope to say the context of the request can affect what data is returned, ie the filters we apply when processing it. The proposal seems to consider tenants as the top-level of abstraction, with the next level up being a global service provider admin, but this does not consider the keystone v3 concept of domains [1] v3 also allows domain-level roles to be inherited to all projects owned by that domain, so in effect-- it does (keystone just takes care of it). Ok, thanks, that's useful info , or that you may wish to provide some of these admin-ish features to domain-admin users (who will adminster data accross multiple tenants, just like has been proposed), via the public-facing API. It seems like we need a way of scoping the request (via data in the context), based on a heirarchy of admin-ness, like: 1. Normal user I assume normal user has some non-admin role on a project/tenant. Yep, that's my assumption, just not a role associated with admin-ness. snip E.g what data can we add to move from X-Roles in auth_token, to expressing roles in multiple tenants and domains? Tokens can only be scoped to a single project or domain, so that's your mapping. All X-Roles apply to the X-Project or X-Domain in context. I don't think we have a good roadmap to support a single authenticated request with multi-project authorization. The best solution I have is to pass an unscoped token that can be rescoped to two or more projects as needed. Trust-based tokens are explicitly scoped already. So this is a piece of the puzzle I was missing until now, combined with the fact that Domain scoped tokens do not imply authorization with all projects within that domain. Thanks for the IRC conversation which cleared that up! Based on this revised understanding, it sounds like, for now at least, some of the global management api requirements may be best served via some client tools which make multiple API calls to get the required information, on behalf of a user who has the necessary roles to access all the projects. (we discussed this a bit in IRC, but I wanted to bring this back to the list as well) I'm going to use projects users sort of interchangeably here depending, so beware... it's a bumpy ride :) The underlying use case, so far as I can see, is actually a tenant-less request, rather than a multi-tenant request. This is exactly the use case we have in keystone for things like managing the service catalog -- we're not managing the service catalog for a specific tenant (although, I suppose we could), we're managing it for the instance of keystone itself (and ultimately, all users of that cloud). The community has previously discussed global roles (roles applicable to all tenants) and expressed opposition -- but I think this concept and use case are very distinct. Hopefully I can explain... The modern parity for global roles is domain-based role assignments that are inherited to all projects owned by that domain. Which is to say, one role assignment makes a role available in a large collection of project scopes. That's available in keystone as of Havana, and *could* be used by heat here, but requires looping through all projects with a bunch of tokens to get all the data as shardy
Re: [openstack-dev] sqlalchemy-migrate needs a new release
On Thursday, November 14, 2013, David Ripton wrote: On 11/14/2013 03:55 PM, Matt Riedemann wrote: On 11/14/2013 2:43 PM, David Ripton wrote: On 11/11/2013 03:35 PM, David Ripton wrote: I'll volunteer to do this release. I'll wait 24 hours from the timestamp of this email for input first. So, if anyone has opinions about the timing of this release, please speak up. (In particular, I'd like to do a release *before* Matt Riedermann's DB2 support patch https://review.openstack.org/#/c/55572/ lands, just in case it breaks anything. Of course we could do another release shortly after it gets in, to make folks who use DB2 happy.) Update: There's now a 0.8 tag in Git but that release failed to reach PyPI, so please ignore it. Thanks fungi and mordred for helping debug what went wrong. https://review.openstack.org/#/c/56449/ (a one-liner) should fix the problem. Once it gets approved, I will attempt to push 0.8.1. Any particular reason to go with 0.8 rather than 0.7.3 as a bug fix release? New maintainers, 2 years since a release, SQLAlchemy 0.8 compatibility (we hope). Even in projects that use strict semantic versioning, 0.x just means not ready yet. There are no stability guarantees until 1.0. http://semver.org/ , point 4 If you have users, you should be making stability guarantees, or at least strongly communicating the lack thereof. -- David Ripton Red Hat drip...@redhat.com ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev -- -Dolph ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [keystone] design summit outcomes
I guarantee there's a few things I'm forgetting, but this is my collection of things we discussed at the summit and determined to be good things to pursue during the icehouse timeframe. The contents represent a high level mix of etherpad conclusions and hallway meetings. https://gist.github.com/dolph/7366031 Corrections and amendments appreciated - thanks! -Dolph ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [ALL] Removing generate_uuid() from uuidutils
On Wed, Nov 13, 2013 at 9:47 AM, John Griffith john.griff...@solidfire.comwrote: On Wed, Nov 13, 2013 at 7:21 AM, Andrew Laski andrew.la...@rackspace.com wrote: On 11/13/13 at 05:48am, Gary Kotton wrote: I recall a few cycles ago having str(uuid.uuid4()) replaced by generate_uuid(). There was actually a helper function in neutron (back when it was called quantum) and it was replaced. So now we are going back… I am not in favor of this change. I'm also not really in favor of it. Though it is a trivial method having it in oslo implies that this is what uuids should look like across OpenStack projects. And I'm in favor of consistency for uuids across the projects because the same parsers and checkers can then be used for input validation or log parsing. Parsers? UUID's should be treated as opaque strings once they're generated. From: Zhongyue Luo zhongyue@intel.commailto: zhongyue@intel.com Reply-To: OpenStack Development Mailing List (not for usage questions) openstack-dev@lists.openstack.orgmailto: openstack-dev@lists.openstack.org Date: Wednesday, November 13, 2013 8:07 AM To: OpenStack Development Mailing List openstack-dev@lists.openstack.orgmailto: openstack-dev@lists.openstack.org Subject: [openstack-dev] [ALL] Removing generate_uuid() from uuidutils Hi all, We had a discussion of the modules that are incubated in Oslo. https://etherpad.openstack.org/p/icehouse-oslo-status https://urldefense.proofpoint.com/v1/url?u=https://etherpad.openstack.org/p/icehouse-oslo-statusk=oIvRg1%2BdGAgOoM1BIlLLqw%3D%3D%0Ar=eH0pxTUZo8NPZyF6hgoMQu%2BfDtysg45MkPhCZFxPEq8%3D%0Am=3ns0o3FRyS2%2Fg%2FTFIH7waZX1o%2FHdXvrJ%2FnH9XMCRy08%3D%0As=63eaa20d8c94217d86793a24379b4391179fbfa1fb2c961fb37a5512dbdff69a One of the conclusions we came to was to deprecate/remove uuidutils in this cycle. The first step into this change should be to remove generate_uuid() from uuidutils. The reason is that 1) generating the UUID string seems trivial enough to not need a function and 2) string representation of uuid4 is not what we want in all projects. There's room for long term improvement such as decreasing string length, increasing entropy, linearly distributed output, etc. I agree that the current implementation is useless/trivial, but the work to build upon it should happen in oslo to benefit all projects. To address this, a patch is now on gerrit. https://review.openstack.org/#/c/56152/ https://urldefense.proofpoint.com/v1/url?u=https://review.openstack.org/%23/c/56152/k=oIvRg1%2BdGAgOoM1BIlLLqw%3D%3D%0Ar=eH0pxTUZo8NPZyF6hgoMQu%2BfDtysg45MkPhCZFxPEq8%3D%0Am=3ns0o3FRyS2%2Fg%2FTFIH7waZX1o%2FHdXvrJ%2FnH9XMCRy08%3D%0As=adb860d11d1ad02718e306b9408c603daa00970685a208db375a9ec011f13978 Each project should directly use the standard uuid module or implement its own helper function to generate uuids if this patch gets in. Any thoughts on this change? Thanks. -- Intel SSG/STO/DCST/CIT 880 Zixing Road, Zizhu Science Park, Minhang District, 200241, Shanghai, China +862161166500 ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev Trivial or not, people use it and frankly I don't see any value at all in removing it. As far as the some projects want a different format of UUID that doesn't make a lot of sense to me but if that's what somebody wants they should write their own method. I strongly agree with others with respect to the comments around code-churn. I see little value in this. ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev -- -Dolph ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [keystone] design summit outcomes
On Wed, Nov 13, 2013 at 11:00 AM, David Chadwick d.w.chadw...@kent.ac.ukwrote: Hi Dolph I have one comment concerning Refactoring: role assignment tables in SQL backend should be unified into a SQL table which lacks referential integrity I am not quite sure what this means, but I did suggest creating an attribute definition table that would include the definitions of all Keystone authz attributes such as role, domain, project as well as identity attributes such as location, group, email address etc. Definitions would include such things as: delegatable or not, for keystone authz or not (see below). Once this table has been defined, then all user role assignments can be collapsed into one table (no need for separate role, domain tables etc) with the assigned attribute pointing to the entry in the definition table. Was this what your bullet point was referring to, or was it something different? I intended to leave the specific implementation details out of this document (they're already captured in the relevant etherpad), but yes -- that would be an improvement on the current table that fits the one liner in the gist. The additional features (such as delegatable) would require a subsequent discussion / change. Here is a strawman proposal Table name = Attribute id: the unique primary key Attribute name: (user friendly name e.g. role, domain, etc.) Attribute Ref: (global id of attribute such as OID or URL, as used by SAML and LDAP) Type: (authz or identity) Delegatable: [Yes|No] Values: [string|integer|Boolean] Now when you assign a role, domain, location, email address or any other attribute to a user a single assignment table can be used such as: Table name: Attribute Assignment id: the unique primary key of this assignment UserID: id of user this attribute is assigned to AttributeID: id of attribute from above table Value: the value of the assigned attribute you dont need to change the existing APIs and procedure calls, as they can be re-written to access the new tables. regards David On 13/11/2013 16:04, Dolph Mathews wrote: I guarantee there's a few things I'm forgetting, but this is my collection of things we discussed at the summit and determined to be good things to pursue during the icehouse timeframe. The contents represent a high level mix of etherpad conclusions and hallway meetings. https://gist.github.com/dolph/7366031 Corrections and amendments appreciated - thanks! -Dolph ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev -- -Dolph ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [PTL] Proposed Icehouse release schedule
On Wed, Nov 13, 2013 at 7:58 AM, Russell Bryant rbry...@redhat.com wrote: On 11/13/2013 08:15 AM, Thierry Carrez wrote: Two options are possible for that off week: * Week of April 21 - this one is just after release, and some people still have a lot to do during that week. On the plus side it's conveniently placed next to the Easter weekend. * Week of April 28 - that's the middle week, which sounds a bit weird... but for me that would be the less active week, so I have a slight preference for it. What would be your preference, if any ? I'm especially interested in opinions from people who have a hard time taking some time off (PTLs, infra and release management people). I think my preference is the second week. Easter makes the first week tempting, but as you point out, realistically there is still going to be some amount of looking out for and potentially dealing with release aftermath. Conversely, there's likely to be less immediate feedback about the release since it occurs just before Easter weekend. (I don't have a preference between the two weeks... yet) The second week everyone really should be able to relax. -- Russell Bryant ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev -- -Dolph ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] Using AD for keystone authentication only
Yes, that's the preferred approach in Havana: Users and Groups via LDAP, and everything else via SQL. On Wednesday, November 13, 2013, Avi L wrote: Hi, I understand that the LDAP provider in keystone can be used for authenticating a user (i.e validate username and password) , and it also authorize it against roles and tenant. However this requires AD schema modification. Is it possible to use AD only for authentication and then use keystone's native database for roles and tenant lookup? The advantage is that then we don't need to touch the enterprise AD installation. Thanks Al -- -Dolph ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] Horizon PTL candidacy
On Fri, Nov 8, 2013 at 2:38 AM, Matthias Runge mru...@redhat.com wrote: Those are my primary targets I'd like to see addressed in Horizon during the cycle. Another thing I'd like to see addressed is the lack of listening to a notification service. That's probably an integration point with Marconi, and it might be possible, this won't make it until Icehouse. This bit caught me off guard - what's the use case here? Is there a link to a blueprint? Thanks! Matthias ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev -- -Dolph ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] RFC: reverse the default Gerrit sort order
On Sun, Nov 10, 2013 at 3:06 PM, Monty Taylor mord...@inaugust.com wrote: https://review.openstack.org/#/mine/important/ Shows me old changes at the top of the reviewable section. Do you use that view at all? That shows me all my own reviews merged abandoned first, which are just noise when I'm trying to review other people's patches. ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev -- -Dolph ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [horizon] User registrations
So, there's a bunch of use case questions here where I suspect there are no correct answers (so preferences will vary per deployment). The first ones that come to mind- Are the users accessing this web form trusted or untrusted? Do they need to be verified, somehow? Are they going to be billed for their resource consumption? After registration, should they own their own domain in keystone? Or be assigned their own project in an existing domain? Or simply be added to an existing group with limited authorization? On Sun, Nov 10, 2013 at 6:26 PM, Paul Belanger paul.belan...@polybeacon.com wrote: Greeting, In a previous thread I talked about building an application atop of horizon and keystone. So far things are working out pretty well. One thing I have been trying to figure out is how to move forward with user registration for the horizon application. A few moons ago, IIRC, horizon actually use django-registration however the move to Keystone removed that functionality. For me, I'd like to expose some functionality within my web application allow users to register vs having an admin provisioning accounts. So, I'm curious if there is anything interest in having such a module back in horizon but leveraging keystone this time around. I'm actually curious to hear how people see this working since this is the next thing I need to deal with. -- Paul Belanger | PolyBeacon, Inc. Jabber: paul.belan...@polybeacon.com | IRC: pabelanger (Freenode) Github: https://github.com/pabelanger | Twitter: https://twitter.com/pabelanger ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev -- -Dolph ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [horizon / keystone] Marker could not be found?
On Thu, Oct 31, 2013 at 8:38 AM, Sebastian Porombka
porom...@uni-paderborn.de wrote:
Hello Folks.
I have a problem after grizzly-havana migration where i’m unable to
rescue myself.
When I open the Admin - Resource-Usage View i get no results – only a
red error box with the message *Error: *Unable to retrieve tenant list.“.
Horizon log:
[Thu Oct 31 11:39:44 2013] [error] Creating a new keystoneclient
connection to http://$controller:35357/v2.0.
[Thu Oct 31 11:39:44 2013] [error] REQ: curl -i -X GET
http://$controller:35357/v2.0/tenants?marker=tenant_markerlimit=21
-H User-Agent: python-keystoneclient -H Forwarded:
for=131.234.5.178;by=python-keystoneclient -H X-Auth-Token: 82[…]f46
[Thu Oct 31 11:39:44 2013] [error] REQ: curl -i -X GET
http://$controller:35357/v2.0/tenants?marker=tenant_markerlimit=21
-H User-Agent: python-keystoneclient -H Forwarded:
for=131.234.5.178;by=python-keystoneclient -H X-Auth-Token: 82[…]46
[Thu Oct 31 11:39:44 2013] [error] INFO:urllib3.connectionpool:Starting
new HTTP connection (1): $controller
[Thu Oct 31 11:39:44 2013] [error] DEBUG:urllib3.connectionpool:GET
/v2.0/tenants?marker=tenant_markerlimit=21 HTTP/1.1 400 88
[Thu Oct 31 11:39:44 2013] [error] RESP: [400]
CaseInsensitiveDict({'date': 'Thu, 31 Oct 2013 11:39:47 GMT', 'vary':
'X-Auth-Token', 'content-length': '88', 'content-type': 'application/json'})
[Thu Oct 31 11:39:44 2013] [error] RESP BODY: {error: {message:
Marker could not be found, code: 400, title: Bad Request}}
[Thu Oct 31 11:39:44 2013] [error]
[Thu Oct 31 11:39:44 2013] [error] RESP: [400]
CaseInsensitiveDict({'date': 'Thu, 31 Oct 2013 11:39:47 GMT', 'vary':
'X-Auth-Token', 'content-length': '88', 'content-type': 'application/json'})
[Thu Oct 31 11:39:44 2013] [error] RESP BODY: {error: {message:
Marker could not be found, code: 400, title: Bad Request}}
[Thu Oct 31 11:39:44 2013] [error]
[Thu Oct 31 11:39:44 2013] [error] Request returned failure status: 400
[Thu Oct 31 11:39:44 2013] [error] Request returned failure status: 400
[Thu Oct 31 11:39:44 2013] [error] Recoverable error: Marker could not be
found (HTTP 400)
Keystone Log:
2013-10-31 12:39:47.352 17187 DEBUG routes.middleware [-] Matched GET
/tenants __call__ /usr/lib/python2.7/dist-packages/routes/middleware.py:100
2013-10-31 12:39:47.352 17187 DEBUG routes.middleware [-] Route path:
'{path_info:.*}', defaults: {'controller':
keystone.contrib.ec2.routers.Ec2Extension object at 0x4156f10} __call__
/usr/lib/python2.7/dist-packages/routes/middleware.py:102
2013-10-31 12:39:47.352 17187 DEBUG routes.middleware [-] Match dict:
{'controller': keystone.contrib.ec2.routers.Ec2Extension object at
0x4156f10, 'path_info': '/tenants'} __call__
/usr/lib/python2.7/dist-packages/routes/middleware.py:103
2013-10-31 12:39:47.353 17187 DEBUG routes.middleware [-] Matched GET
/tenants __call__ /usr/lib/python2.7/dist-packages/routes/middleware.py:100
2013-10-31 12:39:47.353 17187 DEBUG routes.middleware [-] Route path:
'{path_info:.*}', defaults: {'controller':
keystone.contrib.s3.core.S3Extension object at 0x4156cd0} __call__
/usr/lib/python2.7/dist-packages/routes/middleware.py:102
2013-10-31 12:39:47.353 17187 DEBUG routes.middleware [-] Match dict:
{'controller': keystone.contrib.s3.core.S3Extension object at 0x4156cd0,
'path_info': '/tenants'} __call__
/usr/lib/python2.7/dist-packages/routes/middleware.py:103
2013-10-31 12:39:47.354 17187 DEBUG routes.middleware [-] Matched GET
/tenants __call__ /usr/lib/python2.7/dist-packages/routes/middleware.py:100
2013-10-31 12:39:47.354 17187 DEBUG routes.middleware [-] Route path:
'{path_info:.*}', defaults: {'controller':
keystone.contrib.admin_crud.core.CrudExtension object at 0x41517d0}
__call__ /usr/lib/python2.7/dist-packages/routes/middleware.py:102
2013-10-31 12:39:47.355 17187 DEBUG routes.middleware [-] Match dict:
{'controller': keystone.contrib.admin_crud.core.CrudExtension object at
0x41517d0, 'path_info': '/tenants'} __call__
/usr/lib/python2.7/dist-packages/routes/middleware.py:103
2013-10-31 12:39:47.355 17187 DEBUG routes.middleware [-] Matched GET
/tenants __call__ /usr/lib/python2.7/dist-packages/routes/middleware.py:100
2013-10-31 12:39:47.355 17187 DEBUG routes.middleware [-] Route path:
'{path_info:.*}', defaults: {'controller':
keystone.common.wsgi.ComposingRouter object at 0x4151e50} __call__
/usr/lib/python2.7/dist-packages/routes/middleware.py:102
2013-10-31 12:39:47.356 17187 DEBUG routes.middleware [-] Match dict:
{'controller': keystone.common.wsgi.ComposingRouter object at 0x4151e50,
'path_info': '/tenants'} __call__
/usr/lib/python2.7/dist-packages/routes/middleware.py:103
2013-10-31 12:39:47.356 17187 DEBUG routes.middleware [-] Matched GET
/tenants __call__ /usr/lib/python2.7/dist-packages/routes/middleware.py:100
2013-10-31 12:39:47.357 17187 DEBUG routes.middleware [-] Route path:
'/tenants', defaults: {'action':
Re: [openstack-dev] distibuted caching system in front of mysql server for openstack transactions
On Mon, Oct 28, 2013 at 5:46 PM, Qing He qing...@radisys.com wrote: In my hard drive-less use case, I need an in-core-db/cache that can be in the same db cluster with real db (with hard drive) with the same sql api so that the current openstack code do not need to be changed, instead, just a pluggin with some configurations. This is pretty much the original use case that dogpile.cache grew out of, see: http://docs.sqlalchemy.org/en/rel_0_9/orm/examples.html#dogpile-caching -Original Message- From: Morgan Fainberg [mailto:m...@metacloud.com] Sent: Monday, October 28, 2013 10:22 AM To: OpenStack Development Mailing List (not for usage questions) Subject: Re: [openstack-dev] distibuted caching system in front of mysql server for openstack transactions In light of what Dolph said with regards to Keystone, we are using dogpile.cache to implement memoization in front of our driver calls. It it has the ability to cache directly as well, but it has been effective (so far) for our use-case. That being said, I am unsure if caching in front of MySQL is really what we want. I believe that we should be caching after processing work (hence memoization mechanism) instead of at the SQL layer. This also means we can be measured in what we cache (oh hey, it makes no sense to cache X because it needs to be real time or there isn't a performance issue with that query / call, but Y does a ton of processing and is an expensive join/temptable query). In my experience, unless the whole application is designed with caching in mind, caching something as broad as MySQL calls (or any SQL store) is likely going to net exactly what Shawn Hartsock stated, adding a second performance issue. --Morgan On Mon, Oct 28, 2013 at 10:05 AM, Shawn Hartsock hartso...@vmware.com wrote: I once heard a quote.. I had a performance problem, so I added caching. now I have two performance problems. this. 1,000 times this. Just to float this thought ... make sure it's considered... I've seen a *lot* of people misuse caching when what the really want is memoization. * http://stackoverflow.com/questions/1988804/what-is-memoization-and-how -can-i-use-it-in-python * http://stackoverflow.com/questions/10879137/how-can-i-memoize-a-class- instantiation-in-python ... I'm not sure what you're trying to do. So YMMV, TTFN, BBQ. # Shawn Hartsock - Original Message - From: Clint Byrum cl...@fewbar.com To: openstack-dev openstack-dev@lists.openstack.org Sent: Monday, October 28, 2013 12:12:49 PM Subject: Re: [openstack-dev] distibuted caching system in front of mysql server for openstack transactions Excerpts from Dolph Mathews's message of 2013-10-28 08:40:19 -0700: It's not specific to mysql (or sql at all), but keystone is using dogpile.cache around driver calls to a similar effect. http://dogpilecache.readthedocs.org/en/latest/ It can persist to memcache, redis, etc. I once heard a quote.. I had a performance problem, so I added caching. now I have two performance problems. Caching is unbelievably awesome in the jobs it can do well. When the problem is straight forward and the requirements are few, it is just the right thing to relieve engineering pressure to make an application more scalable. However, IMO, more than narrow, well defined cache usage is a sign that the application needs some reworking to scale. I like the principle of let's use dogpile so we don't have to reinvent multi-level caching. However, let's make sure we look at each performance issue individually, rather than just throwing them all in a cache box and waving the memcache wand. https://github.com/openstack/keystone/blob/master/keystone/common/c ache/core.py On Fri, Oct 25, 2013 at 6:53 PM, Qing He qing...@radisys.com wrote: All, Has anyone looked at the options of putting a distributed caching system in front of mysql server to improve performance? This should be similar to Oracle Coherence, or VMware VFabric SQLFire. ** ** Thanks, ** ** Qing ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] Keystone Concurrency Races in SQL Assignment Backend
On Wed, Oct 30, 2013 at 5:08 PM, Peter Feiner pe...@gridcentric.ca wrote: Hi Brant, In addition to the race you've fixed in https://review.openstack.org/#/c/50767/, it looks like there are quite a few more races in the SQL backend of keystone.assignment. I filed a bug to this effect: https://bugs.launchpad.net/keystone/+bug/1246489. The general problem is that transactions are used somewhat indiscriminately. The fix (i.e., using transactions judiciously) is straightforward and should be mostly independent of your ongoing oslo.db sessions port in https://review.openstack.org/#/c/49460/. So, unless you already have something in the works, I'll get started on that tomorrow. I'm eager to fix these races so https://review.openstack.org/#/c/42967/ can reliably pass tempest :-) Thanks for pursuing this :) Blocked by one Brant's dependent patches at the moment, which LGTM: https://review.openstack.org/#/c/51481/ Peter ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev -- -Dolph ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] RFC: Filtering boring commit subjects from ChangeLog
On Mon, Oct 28, 2013 at 3:18 AM, Mark McLoughlin mar...@redhat.com wrote: On Sun, 2013-10-27 at 21:50 -0400, Monty Taylor wrote: Hey all! We're adding a little bit of code to pbr to make the auto-generated ChangeLog files a bit more useful. Currently, they are just the git changelog, which is kinda useless. So we wrote this: https://review.openstack.org/#/c/52367/ which produces output like this: http://paste.openstack.org/show/4 # on a tag and http://paste.openstack.org/show/50001 # not on a tag It underscores the need for commit subjects to communicate something, which is a good thing. With that there, it makes changes like: * Updated from global requirements and * Update my mailmap Kinda lame in the changelog. So we were thinking - what if we recognized one or more subject tags to skip things going into the ChangeLog file. My first thought was: NIT: # use this for tiny commits that are not really worth changelogging and AUTO: # use for commits generated by our machines, such as the global requirements sync or the Translations sync. What do people think? Should we bother? Are those good? Would something else be better? It's sort of an opt-in feature, so adding it SHOULDN'T bork too many people. So long as it isn't so SHOUTY it could work out nicely :) +1, I like the approach to nit / auto so long as it isn't strict. For example, ''.join(c for c in summary.split()[0] if c.isalnum()).upper() in (NIT, AUTO) would support NIT:, (nit) ..., Nit- ..., etc. I'd also be happy with a footer, since a lot of nit / auto commits don't have bugs / bp's to reference... but I don't know what that would look like as a footer? Getting these ChangeLogs published is goodness - the more the more people see their one-line message around the place, the more people they'll make an effort to write a decent one. Cheers, Mark. ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev -- -Dolph ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] distibuted caching system in front of mysql server for openstack transactions
It's not specific to mysql (or sql at all), but keystone is using dogpile.cache around driver calls to a similar effect. http://dogpilecache.readthedocs.org/en/latest/ It can persist to memcache, redis, etc. https://github.com/openstack/keystone/blob/master/keystone/common/cache/core.py On Fri, Oct 25, 2013 at 6:53 PM, Qing He qing...@radisys.com wrote: All, Has anyone looked at the options of putting a distributed caching system in front of mysql server to improve performance? This should be similar to Oracle Coherence, or VMware VFabric SQLFire. ** ** Thanks, ** ** Qing ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev -- -Dolph ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] Remove vim modelines?
On Thu, Oct 24, 2013 at 1:48 PM, Robert Collins robe...@robertcollins.netwrote: *) They help casual contributors *more* than long time core contributors : and those are the folk that are most likely to give up and walk away. Keeping barriers to entry low is an important part of making OpenStack development accessible to new participants. This is an interesting point. My reasoning for removing them was that I've never seen *anyone* working to maintain them, or to add them to files where they're missing. However, I suspect that the users benefiting from them simply aren't deeply enough involved with the project to notice or care about the inconsistency? I'm all for low barriers of entry, so if there's any evidence that this is true, I'd want to make them more prolific. -Dolph ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] Remove vim modelines?
On Fri, Oct 25, 2013 at 2:43 PM, Robert Collins robe...@robertcollins.netwrote: On 26 October 2013 08:40, Dolph Mathews dolph.math...@gmail.com wrote: On Thu, Oct 24, 2013 at 1:48 PM, Robert Collins robe...@robertcollins.net wrote: *) They help casual contributors *more* than long time core contributors : and those are the folk that are most likely to give up and walk away. Keeping barriers to entry low is an important part of making OpenStack development accessible to new participants. This is an interesting point. My reasoning for removing them was that I've never seen *anyone* working to maintain them, or to add them to files where they're missing. However, I suspect that the users benefiting from them simply aren't deeply enough involved with the project to notice or care about the inconsistency? Thats my hypothesis too. I'm all for low barriers of entry, so if there's any evidence that this is true, I'd want to make them more prolific. I'm not sure how to gather evidence for this, either for or against ;(. If we removed them, we might see an uptick in whitespace PEP8 violations. Alternatively, compare the number of historical whitespace violations in files with modelines vs those without. Collecting the data for either of those sound like more work than just adding the modelines to files where they are missing. -Rob -- Robert Collins rbtcoll...@hp.com Distinguished Technologist HP Converged Cloud ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev -- -Dolph ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [keystone] updating password user_crud vs credentials
On Wed, Oct 23, 2013 at 8:14 AM, Chmouel Boudjnah chmo...@enovance.comwrote:
Hello,
If i understand correctly (and I may be wrong) we are moving away from
user_crud to use /credentials for updating password including ec2. The
credentials facility was implemented in this blueprint :
https://blueprints.launchpad.net/keystone/+spec/extract-credentials-id
and documented here :
http://docs.openstack.org/api/openstack-identity-service/2.0/content/POST_updateUserCredential_v2.0_users__userId__OS-KSADM_credentials__credential-type__.html
I may be low on my grep-fu today but I can't seem to find anything
implementing something like :
POST /v2.0/users/{userId}/OSKSADM/credentials/password
The v3 version of this call is in progress:
https://blueprints.launchpad.net/keystone/+spec/v3-user-update-own-password
but only implemented for OS-EC2
So my question is, user_crud seems to be way to update password currently
(by /OS-KSADM/password path) is it something that would need to be added in
the future to /credentials/password ?
That's sort of being tackled here, with slightly different terminology:
https://blueprints.launchpad.net/keystone/+spec/access-key-authentication
Regular passwords are currently backed to the identity driver, but
there's no reason why they couldn't be managed via /v3/credentials.
Cheers,
Chmouel.
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
--
-Dolph
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] RFC - Icehouse logging harmonization
On Wed, Oct 23, 2013 at 1:20 PM, Sean Dague s...@dague.net wrote: One of the efforts that we're working on from the QA team is tooling that ensures we aren't stack tracing into our test logs during normal tempest runs. Random stack traces are scary to cloud admins consuming OpenStack logs, and exceptions in the logs should really be exceptional events (and indicative of a failing system), not something that we do by default. Our intent is to gate code on clean logs (no stacktraces) eventually (i.e. if you try to land a patch that causes stack traces in OpenStack, that becomes a failing condition), and we've got an incremental white list based approach that should let us make forward progress on that. But on that thread - http://lists.openstack.org/**pipermail/openstack-dev/2013-** October/017012.htmlhttp://lists.openstack.org/pipermail/openstack-dev/2013-October/017012.htmlwe exposed another issue... across projects, OpenStack is very inconsistent with logging. First... baseline, these are the logging levels that we have in OpenStack today (with numeric values, higher = worse): CRITICAL = 50 FATAL = CRITICAL ERROR = 40 WARNING = 30 WARN = WARNING AUDIT = 21 # invented for oslo-logging INFO = 20 DEBUG = 10 NOTSET = 0 We also have TRACE, which isn't a level per say, it happens at another level. However TRACE is typically an ERROR in the way we use it. Some examples of oddities in the current system (all from a single devstack/tempest run): Example 1: == n-conductor log in tempest/devstack - http://logs.openstack.org/70/** 52870/3/check/check-tempest-**devstack-vm-full/f46b756/logs/** screen-n-cond.txt.gzhttp://logs.openstack.org/70/52870/3/check/check-tempest-devstack-vm-full/f46b756/logs/screen-n-cond.txt.gz Total log lines: 84076 Total non DEBUG lines: 61 Question: do we need more than 1 level of DEBUG? 3 orders of magnitude information change between INFO - DEBUG seems too steep a cliff. Example 2: == ceilometer-collector - http://logs.openstack.org/70/** 52870/3/check/check-tempest-**devstack-vm-full/f46b756/logs/** screen-ceilometer-collector.**txt.gzhttp://logs.openstack.org/70/52870/3/check/check-tempest-devstack-vm-full/f46b756/logs/screen-ceilometer-collector.txt.gz AUDIT log level being used as DEBUG level (even though it's higher than INFO). 2013-10-23 12:24:20.093 26234 AUDIT ceilometer.pipeline [-] Flush pipeline meter_pipeline 2013-10-23 12:24:20.093 26234 AUDIT ceilometer.pipeline [-] Flush pipeline cpu_pipeline 2013-10-23 12:24:20.094 26234 AUDIT ceilometer.pipeline [-] Flush pipeline meter_pipeline 2013-10-23 12:24:20.094 26234 AUDIT ceilometer.pipeline [-] Flush pipeline cpu_pipeline (this is every second, for most seconds, for the entire run) Example 3: === cinder-api - http://logs.openstack.org/70/** 52870/3/check/check-tempest-**devstack-vm-full/f46b756/logs/** screen-c-api.txt.gz?level=**ERRORhttp://logs.openstack.org/70/52870/3/check/check-tempest-devstack-vm-full/f46b756/logs/screen-c-api.txt.gz?level=ERROR ERROR level being used for 404s of volumes Example 4: === glance-api - http://logs.openstack.org/70/**52870/3/check/check-tempest-** devstack-vm-full/f46b756/logs/**screen-g-api.txt.gzhttp://logs.openstack.org/70/52870/3/check/check-tempest-devstack-vm-full/f46b756/logs/screen-g-api.txt.gz 2013-10-23 12:23:27.436 23731 ERROR glance.store.sheepdog [-] Error in store configuration: Unexpected error while running command.Command: collieExit code: 127Stdout: ''Stderr: '/bin/sh: 1: collie: not found\n' 2013-10-23 12:23:27.436 23731 WARNING glance.store.base [-] Failed to configure store correctly: Store sheepdog could not be configured correctly. Reason: Error in store configuration: Unexpected error while running command.Command: collieExit code: 127Stdout: ''Stderr: '/bin/sh: 1: collie: not found\n' Disabling add method. part of every single Tempest / Devstack run, even though we aren't trying to configure sheepdog in the gate I think we can, and should do better, and started trying to brain dump into this etherpad - https://etherpad.openstack.**org/p/icehouse-logging-* *harmonizationhttps://etherpad.openstack.org/p/icehouse-logging-harmonization(examples included). This is one of those topics that I think our current 6 track summit model doesn't make easy address, as we really need general concensus across any project that's using oslo-logging, so I believe mailing list is the better option, at least for now. Goals - Short Term === As much feedback as possible from both core projects and openstack deployers about the kinds of things that they believe we should be logging, and the kinds of levels they think those things should land at. Deprecation warnings! Based on the approach we're taking in the patch below, we'll be able to notate how imminently a feature is facing
