Re: [openstack-dev] [keystone] Signing off

[David Stanek]

On 30-May-2018 08:45, Henry Nash wrote:
> Hi
>  
> It is with a somewhat heavy heart that I have decided that it is time to hang
> up my keystone core status. Having been involved since the closing stages of
> Folsom, I've had a good run! When I look at how far keystone has come since 
> the
> v2 days, it is remarkable - and we should all feel a sense of pride in that.
>  
> Thanks to all the hard work, commitment, humour and support from all the
> keystone folks over the years - I am sure we will continue to interact and 
> meet
> among the many other open source projects that many of us are becoming 
> involved
> with. Ad astra!
>  

Hey Henry!

It's good to hear from you! You were always fun to work with and I got a
lot out of our chats about crazy, enterprisey things. I guess the world
with have to wait for another episode of "Stanek & Nash".

-- 
david stanek
web: https://dstanek.com
twitter: https://twitter.com/dstanek

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [keystone] Adding foreign keys between subsystems

[David Stanek]

On 12-Apr 15:25, Rodrigo Duarte wrote:
> On Wed, Apr 12, 2017 at 2:47 PM, David Stanek <dsta...@dstanek.com> wrote:
> 
> > On 12-Apr 14:30, Rodrigo Duarte wrote:
> > > Just to illustrate the discussion, we have a bug fix that currently tries
> > > to drop a FK between the federation and identity subsystems [1].
> > >
> > > [1] https://review.openstack.org/#/c/445505/
> >
> > I think this highlights one of my problems with the current architecture.
> > I see that
> > you've removed the FK and added delete logic to do what the data layer
> > would be doing
> > for you. I didn't see any added get_user() checks to make sure the user_id
> > being used
> > in creates/updates is valid. Are we already checking that somewhere else
> > or is this
> > introducing a new bug?
> >
> 
> The review [1] is dropping the idp_id and idp_id + protocol_id FKs, not the
> user_id one.
> 

Right, misremembered. Just run s/user_id/idp_id/ and s/get_user/get_idp/ and 
you'll
have the same issues.

-- 
david stanek
web: https://dstanek.com
twitter: https://twitter.com/dstanek

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [keystone] Adding foreign keys between subsystems

[David Stanek]

On 12-Apr 14:30, Rodrigo Duarte wrote:
> Just to illustrate the discussion, we have a bug fix that currently tries
> to drop a FK between the federation and identity subsystems [1].
> 
> [1] https://review.openstack.org/#/c/445505/

I think this highlights one of my problems with the current architecture. I see 
that
you've removed the FK and added delete logic to do what the data layer would be 
doing
for you. I didn't see any added get_user() checks to make sure the user_id 
being used
in creates/updates is valid. Are we already checking that somewhere else or is 
this
introducing a new bug?

-- 
david stanek
web: https://dstanek.com
twitter: https://twitter.com/dstanek

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [keystone] Adding foreign keys between subsystems

[David Stanek]

[tl;dr I want to remove the artificial restriction of not allowing FKs between
subsystems and I want to stop FK enforcement in code.]

The keystone code architecture is pretty simple. The data and functionality are
divided up into subsystems. Each subsystem can be configured to use a different
backend datastore. Of course, there are always exceptions to the rule like how
the federation and identity subsystems are highly coupled in the data model.

On the surface this flexible model sounds good, but there are some interesting
consequences. First, you can't tell from looking at the data model that there
actually is a lot of coupling between the subsystems. So instead of looking at
our sqlalchemy models to see relationships, we must look throughout the code
for where a particular primary key is used and look for enforcement. (Hopefully
we enforce it in all of the right places.) Additionally, a DBA/data architect/
whenever can't see the relationship at all by looking at the database. 

Second, this has polluted our code and causes erroneous API errors. We have 
added
lots of get_*() calls in our code that checks for the existence of IDs in
another subsystem. In some cases we probably don't do the check and thus would
allow bad data to be stored. The check often causes 404s instead of 400s when
bad data is provided.

So I'd like us to be more deliberate in defining which parts of the data model
are truly independent and a separate backend datastore would make sense. For
instance, we know we want to support LDAP for identity (although this still puts
identity info into a SQL database) and catalog is very isolated from the rest of
the data model.

As a side effect this means that if deployers wished to use a separate backend
for something they would need to also implement it for the other highly coupled
subsystems. They would also have to provide any FK enforcement that their own
datastore does not provide.

Thoughts?

-- 
david stanek
web: https://dstanek.com
twitter: https://twitter.com/dstanek

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [keystone]PKI token VS Fernet token

[David Stanek]

On 15-Feb 18:16, 王玺源 wrote:
> Hello everyone,
>   PKI/PKIZ token has been removed from keystone in Ocata. But recently our
> production team did some test about PKI and Fernet token (With Keystone
> Mitaka). They found that in large-scale production environment, Fernet
> token's performance is not as good as PKI. Here is the test data:
> 
> https://docs.google.com/document/d/12cL9bq9EARjZw9IS3YxVmYsGfdauM25NzZcdzPE0fvY/edit?usp=sharing

This is nice to see. Thanks.


> 
> From the data, we can see that:
> 1. In large-scale concurrency test, PKI is much faster than Fernet.
> 2. PKI token revoke can't immediately make the token invalid. So it has the
> revoke issue.  https://wiki.openstack.org/wiki/OSSN/OSSN-0062
> 
> But in our production team's opinion, the revoke issue is a small problem,
> and can be avoided by some periphery ways. (More detail solution could be
> explained by them in the follow email).
> They think that the performance issue is the most important thing. Maybe
> you can see that in some production environment, performance is the first
> thing to be considered.

I'd like to hear solutions to this if you have already come up with
them. This issue, however, isn't the only one that led us to remove PKI
tokens.

> 
> So here I'd like to ask you, especially the keystone experts:
> 1. Is there any chance to bring PKI/PKIZ back to Keystone?

I would guess that, at least in the immediate future, we would not want
to put it back into keystone until someone can fix the issues. Also
ideally running the token provider in production.


> 2. Has Fernet token improved the performance during these releases? Or any
> road map so that we can make sure Fernet is better than PKI in all side.
> Otherwise, I don't think that remove PKI in Ocata is the right way. Or
> even, we can keep the PKI token in Keystone for more one or two cycles,
> then remove it once Fernet is stable enough.
> 3. Since I'll be in Atalanta next week, if it is possible, I'd like to
> bring this topic to Keystone PTG. can I?

Sure. We have a pretty packed calendar, but I'm sure you could steal a
few minutes somewhere.


> 
> It is a real production problem and I really need your feedback.
> 

Have you tried playing with the crypt_strength[1]? If the slowness is
the crypto (which it was in the past) then you can tune it a little bit.
Another option might be to keep the same token flow and find a faster
method for hashing a token.

1. 
http://git.openstack.org/cgit/openstack/keystone/tree/etc/keystone.conf.sample#n67


-- 
david stanek
web: https://dstanek.com
twitter: https://twitter.com/dstanek

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [keystone] webob 1.7

[David Stanek]

On 18-Jan 08:59, Chuck Short wrote:
> Hi
> 
> We have been expericing problems with newer versions of webob (webob 1.7).
> Reading the changelog, it seems that the upstream developers have
> introduced some backwards incompatibility with previous versions of webob
> that seems to be hitting keystone and possibly other projects as well
> (nova/glance in particular). For keystone this bug has been reported in bug
> #1657452. I would just like to get more developer's eyes on this particular
> issue and possibly get a fix. I suspect its starting to hit other distros
> as well or already have hit.
> 

I've confirmed that this is an issue. I'll work on a fix. We can take 
further discussion to the bug tracker.

--
david stanek
web: https://www.dstanek.com
twitter: https://twitter.com/dstanek

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [keystone] Custom ProjectID upon creation

[David Stanek]

On 05-Dec 15:14, Lance Bragstad wrote:
> I put myself in Boris' camp on this one. This can open up the opportunity
> for negative user-experience, purely based on where I authenticate and
> which token I happen to authenticate with. A token would no longer be
> something I can assume to be properly validated against any node in my
> deployment. Now, when I receive a 401 Unauthorized, is it because the token
> is actually invalid, did I use the wrong endpoint, or did I use a token
> with the wrong scope for the endpoint I wanted to interact with?
> 

I agree. I think having different behavior for tokens based on scope
will not only lead to bad user experiences, but will lead to baking in
those rules into the client. Someone will propose this as soon as they
get confused by the token 401ing unexpectedly. 

-- 
david stanek
web: http://www.dstanek.com
blog: http://www.traceback.org

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [keystone] new core reviewer (rderose)

[David Stanek]

On Thu, Sep 01 at 10:44 -0400, Steve Martinelli wrote:
> 
> Thanks for all your hard work Ron, we sincerely appreciate it.
> 

Contrats! Well deserved for sure!

-- 
David Stanek
web: http://dstanek.com
blog: http://traceback.org

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [keystone][nova][neutron][all] Rolling upgrades: database triggers and oslo.versionedobjects

[David Stanek]

On Wed, Aug 31 at 17:18 -0500, Monty Taylor wrote:
> 
> Nova and Neutron have an approach for this. It may or may not be ideal -
> but it exists right now. While it can be satisfying to discount the
> existing approach and write a new one, I do not believe that is in the
> best interests of OpenStack as a whole. To diverge in _keystone_ - which
> is one of the few projects that must exist in every OpenStack install -
> when there exists an approach in the two other most commonly deployed
> projects - is such a terrible example of the problems inherent in
> Conway's Law that it makes me want to push up a proposal to dissolve all
> of the individual project teams and merge all of the repos into a single
> repo.

That's a bit overly dramatic. I think having some innovation is a good
thing. Specifically in this case where our needs appear to be a little
simpler than those of nova.

> 
> Make the oslo libraries Nova and Neutron are using better. Work with the
> Nova and Neutron teams on a consolidated approach. We need to be driving
> more towards an OpenStack that behaves as if it wasn't written by
> warring factions of developers who barely communicate.

I believe we tried to keep with the same extract/migrate/contract
patterns. Sure our implementation differs, but I don't see operators
caring about that as long as it works.

> 
> Even if the idea was one I thought was good technically, the above would
> still trump that. Work with Nova and Neutron. Be more similar.
> 
> PLEASE
> 
> BUT - I also don't think it's a good technical solution. That isn't
> because triggers don't work in MySQL (they do) - but because we've spent
> the last six years explicitly NOT writing raw SQL. We've chosen an
> abstraction layer (SQLAlchemy) which does its job well.
> 
> IF this were going to be accompanied by a corresponding shift in
> approach to not support any backends by MySQL and to start writing our
> database interactions directly in SQL in ALL of our projects - I could
> MAYBE be convinced. Even then I think doing it in triggers is the wrong
> place to put logic.
> 
> "Database triggers are obviously a new challenge for developers to
> write, honestly challenging to debug (being side effects), and are made
> even more difficult by having to hand write triggers for MySQL,
> PostgreSQL, and SQLite independently (SQLAlchemy offers no assistance in
> this case)"
> 
> If you look at:
> 
> https://review.openstack.org/#/c/355618/40/keystone/common/sql/expand_repo/versions/002_add_key_hash_and_encrypted_blob_to_credential.py
> 
> You will see the three different SQL dialects this. Not only that, but
> some of the more esoteric corners of those backends. We can barely get
> _indexes_ right in our database layers ... now we think we're going to
> get triggers right? Consistently? And handle things like Galera?
> 
> The other option is app level, which is what nova and neutron are doing.
> It's a good option, because it puts the logic in python, which is a
> thing we have 2500 developers fairly well versed in. It's also scalable,
> as the things executing whatever the logic is are themselves a scale-out
> set of servers. Finally, it's a known and accepted pattern in large
> scale MySQL shops ... Roll out a new version of the app code which
> understands both the old and the new schema version, then roll out a
> no-downtime additive schema change to the database, then have the app
> layer process and handle on the fly transformation if needed.

I've done both types of migrations in the past, but with one imporant
exception. We could roll out our application on Tuesday and then the
cleanup on Thursday. We didn't carry baggage for 6 months to a year. My
fear with keystone is that we'd slow development even more by adding
more cruft and cruft on top of cuft.

> 
> SO ...
> 
> Just do what Nova and Neutron are doing - and if it's not good enough,
> fix it. Having some projects use triggers and other projects not use
> triggers is one of the more epically crazypants things I've heard around
> here ... and I lived through the twisted/eventlet argument.

-- 
David Stanek
web: http://dstanek.com
blog: http://traceback.org

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [keystone][nova][neutron][all] Rolling upgrades: database triggers and oslo.versionedobjects

[David Stanek]

On Thu, Aug 25 at 13:13 -0400, Steve Martinelli wrote:
> The keystone team is pursuing a trigger-based approach to support rolling,
> zero-downtime upgrades. The proposed operator experience is documented here:
> 
>   http://docs.openstack.org/developer/keystone/upgrading.html
> 

I wanted to mention a few things. One of the reasons I suggested this
approach for keystone is that I've had success in the past using a
combination of triggers and code to do live, online migrations. Many
times using completely different schemas.

In keystone we are just talking about some simple data transformations
between columns and things like that. The triggers themselves shouldn't
get too complicated. If there are cases where triggers won't work, then
we won't force them. (A current example of this is encrypting
credentials.)

The online migrations are not required. Operators can still go the old
route and db_sync while others help test out the cutting edge features.

The triggers are not there during the entire lifecycle of the
application. The expand phase adds them and the contract removes them.

-- 
David Stanek
web: http://dstanek.com
blog: http://traceback.org

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Keystone] Multi-factor Auth with Keystone and TOTP

[David Stanek]

On Mon, Jul 18, 2016 at 9:13 AM, Adrian Turjak  wrote:
> We need an MFA solution, and this doesn't seem like too terrible an option.


One thing to note here is that the credentials for TOTP stored in the
keystone credentials backend are not encrypted. So a breach of your
database could expose those to an attacker. This is a review[1] to fix
this issue that is close to merging.

1. https://review.openstack.org/#/c/317169/

-- 
David
blog: http://www.traceback.org
twitter: http://twitter.com/dstanek
www: http://dstanek.com

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [security] [horizon] Security implications of exposing a keystone token to a JS client

[David Stanek]

On 07/01 at 19:41, Fox, Kevin M wrote:
> Hi David,
> 
> How do you feel about the approach here:
> https://review.openstack.org/#/c/311189/
> 
> Its lets the existing angular js module:
> horizon.app.core.openstack-service-api.keystone
> 
> access the current token via getCurrentUserSession().token
> 

Hey Kevin,

It's hard to tell without a lot of the context. From what I can tell the
token is pulled down as part of the data of an API request. As long as
that's not cached I think you are OK.

-- 
David Stanek
web: http://dstanek.com
blog: http://traceback.org

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] Syntribos Error : AttributeError: 'tuple' object has no attribute 'headers'

[David Stanek]

On 07/02 at 15:52, OpenStack Mailing List Archive wrote:
> Link: https://openstack.nimeyo.com/89478/?show=89478#q89478
> From: run2obtain <run2obt...@gmail.com>
> 
> 
> Hi... I tried to use OpenStack Syntribos today for security testing against my
> devstack kilo cloud. I followed installation and configuration instructions
> provided at the openstack syntribos repo .Unfortunately, I received some 
> errors
> after running the command : syntribos keystone.config .opencafe/templates/
> keystone/roles_get.txt . The errors are File "/usr/local/lib/python2.7/
> dist-packages/syntribos/extensions/identity/client.py", line 146, in
> get_token_v3 return r.headers["X-Subject-Token"] AttributeError: 'tuple' 
> object
> has no attribute 'headers'. ' I have not been successful at discovering what
> could be wrong or how to resolve this issue, even after googling. Does anyone
> have a hint as to how to resolve this issue. Many thanks for your anticipated
> response.
> 

A quick look at the code[1] show that the HTTP client used by the identity
client actually returns a tuple instead of a response object. The tuple
contains two items: a response object and a signal handler object.

This is the first I've heard of this project, so I was very disappointed
to not find any docs for it.

1. 
https://github.com/openstack/syntribos/blob/master/syntribos/clients/http/base_http_client.py#L143

-- 
David Stanek
web: http://dstanek.com
blog: http://traceback.org

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [security] [horizon] Security implications of exposing a keystone token to a JS client

[David Stanek]

On 06/29 at 21:10, Timur Sufiev wrote:
> Hello, vigilant folks of OpenStack Security team!
> 
> The commit(s) I'd like you to take a look at introduces a new Horizon
> feature, Create (Glance) Image using CORS (AKA Cross-Origin Resource
> Sharing) [1].
> 
> The main idea is to bypass Horizon web-server when uploading large local
> image and to send it directly to Glance server, thus saving network
> bandwidth and disk space on the controller node where Horizon web-server is
> deployed. However there is one possible security trade-off I had to make so
> that Glance service would allow me to upload an image - I'm passing the
> Keystone token to the Horizon JS runtime [2], and then pass it to Glance
> service [3] or [4] (different links here correspond to different versions
> of new Create Image - Django and Angular). This trade-off made Horizon
> community somewhat hesitant if we should push these changes forward, but
> nobody yet voiced a viable alternative, so here I'm writing this letter to
> you.
> 
> The usual Horizon workflow for working with Keystone tokens is the
> following: retrieve scoped token and put it into web-server session, which
> is itself not exposed to browser (unless SESSION_STORAGE signed_cookies
> backend was chosen, but even in that case session contents are encrypted in
> some way), but is kept on web-server and referenced using the session key
> which is kept in browser cookies - so one may say that in existing setup
> keystone token never leaks to browser.
> 
> On the other hand, in some not so far (I hope) future, when more logic is
> moved to client-side UI (i.e. browser), the issue of browser authenticating
> to some OpenStack services directly would become more widespread, it just
> happened that this work on Create Image in Horizon is pioneering this area
> (AFAIK). So, what do you think of possible security implications of this
> setup?
> 
> Just for the reference, three patches mentioned in [1-3] implement most of
> the logic of new Create Image feature.
> 
> [1]
> https://blueprints.launchpad.net/horizon/+spec/horizon-glance-large-image-upload
> [2]
> https://review.openstack.org/#/c/317365/15/openstack_dashboard/api/glance.py@215
> [3]
> https://review.openstack.org/#/c/230434/37/horizon/static/horizon/js/horizon.modals.js@212
> [4]
> https://review.openstack.org/#/c/317456/16/openstack_dashboard/static/app/core/openstack-service-api/glance.service.js@151

Since tokens are bearer tokens any leak could possibly lead to a
security issue. I don't see allowing the JS application to have access
to the token as being a terrible thing.

We just need to make sure we do it as safely as we can in order to
prevent the token from lingering around after the web session has
completed. For example, putting the token in redirect URLs may cause
it to end up in browser history, putting it in the source of page
that could be cached may write it to disk, etc, etc.

-- 
David Stanek
web: http://dstanek.com
blog: http://traceback.org

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [nova] Placement API WSGI code -- let's just use flask

[David Stanek]

On Tue, Jun 21, 2016 at 08:00:50AM -0400, Sean Dague wrote:
> 
> Keystone - custom WSGI with Routes / Paste
>

Keystone is moving toward Flask. I have an experimental patch that
moves us in that direction. I'm in the process to rebasing and
fixing it up since it's wildly out of date.

-- David

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [keystone] orchestration and db_sync

[David Stanek]

On Fri, May 27, 2016 at 12:08 PM, Ryan Hallisey  wrote:

Theses changes do not all happen at the same times for an OpenStack
installation.

> - Create the service's users and add a password into the databse

Should only happen once during installation.

> - Sync the service with the database

Should happen during installation and for every upgrade.

> - Start the service
>
> I was wondering if for some services they could be aware of whether or not 
> they need
> to sync with the database at startup.  Or maybe the service runs a db_sync 
> every time
> is starts?  I figured I would start a thread about this because Keystone has 
> some
> flexibility when running N+1 in a cluster of N. If Keystone could have that
> that ability maybe Keystone could db_sync each time it starts without harming 
> the
> cluster?

This isn't something I would want to see for a few reasons. The most
important one is that I think the decision to run db_sync needs to be
explicit. An operator should run it when they are ready (maybe they
need to shut something down, ensure up-to-date backups, etc.).

Another issue is database modification permissions. The user running
the application, as well as the DB user the application uses,
shouldn't have access to DML for security reasons. Little Bobby
Tables' mom found this out the hard way[1].

1. https://xkcd.com/327/

-- 
David
blog: http://www.traceback.org
twitter: http://twitter.com/dstanek
www: http://dstanek.com

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] summit tools

[David Stanek]

On Wed, Apr 20, 2016 at 12:14 PM Neil Jerram 
wrote:

> A couple of questions about our Austin-related planning tools...
>
> - Can one's calendar at
>
> https://www.openstack.org/summit/austin-2016/summit-schedule/#day=2016-04-25
> be exported as .ics, or otherwise integrated into a wider calendaring
> system?
>

I put together a tool[1] that can do this. There is some manual work
involved because you have to save the print view of your calendar from your
browser.

It was a quick and dirty hackso patches accepted and encouraged.

1. https://gist.github.com/dstanek/83d536af9461eb26d7a19ff73a7391c2

-- David
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [python-keystoneclient] Return request-id to caller

[David Stanek]

On Wed, Apr 13, 2016 at 3:26 AM koshiya maho 
wrote:

>
> My request to all keystone cores to give their suggestions about the same.
>
>
I'll test this a little and see if I can see how it breaks.

Overall I'm not really a fan of this design. It's just a hack to add
attributes where they don't belong. Long term I think this will be hard to
maintain.

-- David
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [tc][ptl][keystone] Proposal to split authentication part out of Keystone to separated project

[David Stanek]

On Wed, Apr 6, 2016 at 3:26 PM Boris Pavlovic 
wrote:

>
> 2) This will reduce scope of Keystone, which means 2 things
> 2.1) Smaller code base that has less issues and is simpler for testing
> 2.2) Keystone team would be able to concentrate more on fixing
> perf/scalability issues of authorization, which is crucial at the moment
> for large clouds.
>

I'm not sure that this is entirely true. If we truly just split up the
project, meaning we don't remove functionality, then we'd have the same
number of bugs and work. It would just be split across two projects.

I think the current momentum to get out of the authn business is still our
best bet. As Steve mentioned this is ongoing work.

-- David
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [magnum] High Availability

[David Stanek]

On Fri, Mar 18, 2016 at 4:03 PM Douglas Mendizábal <
douglas.mendiza...@rackspace.com> wrote:

> [snip]
> >
> > Regarding the Keystone solution, I'd like to hear the Keystone team's
> feadback on that.  It definitely sounds to me like you're trying to put a
> square peg in a round hole.
> >
>
>
I believe that using Keystone for this is a mistake. As mentioned in the
blueprint, Keystone is not encrypting the data so magnum would be on the
hook to do it. So that means that if security is a requirement you'd have
to duplicate more than just code. magnum would start having a larger
security burden. Since we have a system designed to securely store data I
think that's the best place for data that needs to be secure.
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all][infra] eventlet 0.18.1 not on PyPi anymore

[David Stanek]

On Wed, Feb 17, 2016 at 6:58 PM Sean Dague  wrote:

>
> Question. Are we only tripping this up in unit tests because the tests
> are doing things we'd never really do in real life?
>

I think that some of the issues have been real. Keystone had issues with
0.18.0 because it dropped methods from subprocess after patching it. They
called it out in their release notes as an incompatible change.

Version 0.18.2 broke the Keystone tests for Python 34. The poll() method
seemed to be in some sort of deadlock and the tests timed out. This may
have been a testing problem, but for whatever reason Python27 didn't have
the issue.

I want to say that we've seen other issues after 0.17.4, but I can't find
definitive evidence.

-- David
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [trove][neutron][cinder][swift][ceilometer][nova][keystone][sahara][glance][neutron-lbaas][imm] stylistic changes to code, how do we handle them?

[David Stanek]

On Tue, Jan 12, 2016 at 8:51 AM, Amrith Kumar  wrote:

> I've tagged this message with the projects impacted by a series of change
> sets:
>
> [trove] https://review.openstack.org/#/c/266220/
> [neutron] https://review.openstack.org/#/c/266156/1
> [cinder] https://review.openstack.org/#/c/266099/2
> [swift] https://review.openstack.org/#/c/266185/1
> [ceilometer] https://review.openstack.org/#/c/266211/1
> [nova] https://review.openstack.org/#/c/266143/2
> [keystone] https://review.openstack.org/#/c/266203/2
> [sahara] https://review.openstack.org/#/c/266230/1
> [glance] https://review.openstack.org/#/c/266192/1
> [neutron-lbaas] https://review.openstack.org/#/c/266181/1
>
> I would like the guidance of the developer community in figuring out how
> to proceed with this change, and changes like this.
>
> The change, in essence changes a construct of the kind:
>
> if var > 0:
> ... something ...
>
> To
>
> if var:
> ... something ...
>
> In a couple of cases, it also changes messages from (for example, in Trove)
>
> "Limit value must be > 0" to
> "Limit value must be greater than zero"
>
> My question to the ML is this, should stylistic changes of this kind be
> handled in a consistent way across all projects, maybe with a hacking rule
> and some discussion on the ML first? After all, if this change is
> worthwhile, it is worth ensuring that this construct that we are seeking to
> eliminate, does not reenter the code base.
>

The code above is not stylistic in nature.  It actually changes the
semantics of the check. This should not be done blindly across the board.

In the first case it looks for a value to be greater than a constant. In
the second case it looks for a value to have a truthy value.


> For what it is worth, I agree with Vitaly Grindev [sahara, in review
> https://review.openstack.org/#/c/266230/1]. I think the code before the
> change was more intuitive and readable.
>

Only if the code is correct. In that review I saw you changing "len(x) > 0"
to "len(x)" and that's fine. But you also changed a check looking at the
status code from a conductor call to no longer allow negative return
values. I have no idea if it returns negative numbers, but in this case I
would guess the "> 0" had a purpose.

-- 
David
blog: http://www.traceback.org
twitter: http://twitter.com/dstanek
www: http://dstanek.com
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [trove][neutron][cinder][swift][ceilometer][nova][keystone][sahara][glance][neutron-lbaas][imm] stylistic changes to code, how do we handle them?

[David Stanek]

On Tue, Jan 12, 2016 at 9:13 AM, Amrith Kumar  wrote:

> Chris, Ihar,
>
> I assumed that this was stylistic based on the fact that in the places
> where I was seeing it, it seemed to be the case that the LHS was
> intuitively positive (a length, for example). I did not exhaustively verify
> this but yes, you are correct, the construct
>
> If var
>
> is the same as  (I believe)
>
> if var is not None


Those two statements are also not the same. 'if var' will be True is 'var'
is *only* True values. 'if var is not None' will be True *everything*
except None, so 0, '', [], etc. will not pass the first test, but will pass
the second.



-- 
David
blog: http://www.traceback.org
twitter: http://twitter.com/dstanek
www: http://dstanek.com
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] re-introducing twisted to global-requirements

[David Stanek]

On Thu, Jan 7, 2016 at 3:01 PM, Jim Rollenhagen 
wrote:

> We'd be using this for functional tests, not unit, where we can't really
> inject mocks. The idea is that we could run a full functional suite
> against either mimic or a full ironic environment, just by changing a
> test setting.
>

I'm assuming that this would be for client functional tests because it
wouldn't make sense for testing a server. How is the interface created? It
seems like it would be possible for the mocked API to not match the actual
API.


-- 
David
blog: http://www.traceback.org
twitter: http://twitter.com/dstanek
www: http://dstanek.com
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [keystone][all] Move from active distrusting model to trusting model

[David Stanek]

On Mon, Nov 23, 2015 at 6:06 PM David Chadwick 
wrote:
[snip]

> >
> > This is just a vote for distrusting the community. If you think there's
> > "power" in being able to merge things, and that organizations will abuse
> > this power, then you vote for distrust.
>
> No, rather for the abuse of power by organisations, which is rather
> different :-) We have multiple examples of these all the time.
>

I don't know if it's fair to say that voting for the current, strict rules
is the same as voting for distrust. I also don't know if it's fair to say
that we have multiple examples of organizations abusing power without
clarifying if you mean in our community.

As I said earlier I can see these rules as being protective. Similar to how
organizations have lots of rules about getting gifts from suppliers,
customers, etc. and a variety of other 'conflict of interest' situations.
Much of that is useful to avoid the appearance of impropriety. Simple rules
can keep you out of some bad situations.

Is that valuable at all within our community? Is that valuable to protect
our community from outside forces?

-- David
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [keystone][all] Move from active distrusting model to trusting model

[David Stanek]

On Mon, Nov 23, 2015 at 11:52 AM Dmitry Tantsur  wrote:

> On 11/23/2015 05:42 PM, Morgan Fainberg wrote:
> > Hi everyone,
> >
>
[snip,snip]

> >
> > This type of policy is an actively distrustful policy.
>

I don't see it quite like that. I don't think the policy is there because
I'm not trusted to make good decisions, I see it more as protecton from the
appearance that I am doing it.

I trust all of the Keystone cores and would have no issue lifting the
policy. The question is if that opens us up to scrutiny (or anything else)
from outside our development group. I don't know the history behind the
policy, so I can't answer that question.

-- David
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] service catalog: TNG

[David Stanek]

On Fri, Oct 9, 2015 at 1:28 PM, Jonathan D. Proulx 
wrote:

> On Fri, Oct 09, 2015 at 01:01:20PM -0400, Shamail wrote:
> :> On Oct 9, 2015, at 12:28 PM, Monty Taylor  wrote:
> :>
> :>> On 10/09/2015 11:21 AM, Shamail wrote:
> :>>
> :>>
> :>>> On Oct 9, 2015, at 10:39 AM, Sean Dague  wrote:
> :>>>
> :>>> It looks like some great conversation got going on the service catalog
> :>>> standardization spec / discussion at the last cross project meeting.
> :>>> Sorry I wasn't there to participate.
> :>> Apologize if this is a question that has already been address but why
> can't we just leverage something like consul.io?
> :>
> :> It's a good question and there have actually been some discussions
> about leveraging it on the backend. However, even if we did, we'd still
> need keystone to provide the multi-tenancy view on the subject. consul
> wasn't designed (quite correctly I think) to be a user-facing service for
> 50k users.
> :>
> :> I think it would be an excellent backend.
> :Thanks, that makes sense.  I agree that it might be a good backend but
> not the overall solution... I was bringing it up to ensure we consider
> existing options (where possible) and spend cycles on the unsolved bits.
>
> As an operator I'd be happy to use SRV records to define endpoints,
> though multiple regions could make that messy.
>
> would we make subdomins per region or include region name in the
> service name?
>
> _compute-regionone._tcp.example.com
>-vs-
> _compute._tcp.regionone.example.com
>
> Also not all operators can controll their DNS to this level so it
> couldn't be the only option.
>
> Or are you talking about using an internal DNS implementation private
> to the OpenStack Deployment?  I'm actually a bit less happy with that
> idea.
>

I was able to put together an implementation[1] of DNS-SD loosely based on
RFC-6763[2]. It'd really a proof of concept, but we've talked so much about
it that I decided to get something working. Although if this seems like a
viable option then there's still much work to be done.

I'd love feedback.

1. https://gist.github.com/dstanek/093f851fdea8ebfd893d
2. https://tools.ietf.org/html/rfc6763

-- 
David
blog: http://www.traceback.org
twitter: http://twitter.com/dstanek
www: http://dstanek.com
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] Apache2 vs uWSGI vs ...

[David Stanek]

On Fri, Sep 25, 2015 at 8:25 AM Adam Heczko  wrote:

> Are we discussing mod_wsgi and Keystone or OpenStack as a general?
> If Keystone specific use case, then probably Apache provides broadest
> choice of tested external authenticators.
> I'm not against uwsgi at all, but to be honest expectation that nginx
> could substitute Apache in terms of authentication providers is simply
> unrealistic.
>

uwsgi isn't a replacement for Apache. It's a replacement for mod_wsgi. It
just so happens that it does let you use user web servers if that's what
your usecase dictates.

As a Keystone developer I don't want to tell deployers that they have to
use Apache. It should be their choice. Since Apache is the most common web
server in our community I think we should continue to provide example
configurations and guidance for it.


-- David
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [releases][requirements][keystone]something incompatible with our requirements

[David Stanek]

On Fri, Sep 18, 2015 at 3:32 PM, Robert Collins 
wrote:

> I know this is terrible timing with the release and all, but
> constraints updates are failing. This is the first evidence - and it
> doesn't look like a race to me:
>
> http://logs.openstack.org/57/221157/10/check/gate-tempest-dsvm-full/18eb440/logs/devstacklog.txt.gz#_2015-09-18_13_51_46_902
>
> https://review.openstack.org/#/c/221157/ is the updated review to
> bring it all together. I'm worried that the incompatibility is going
> to impact distributors and/or may even be from one of our own recent
> library releases.
>

This looks like the issue I just ran into. The newest os-client-config
depends on keystoneauth1 and this breaks openstackclient since it registers
its plugins under the keystoneclient entrypoint and not the keystoneauth1
entry point.

-- 
David
blog: http://www.traceback.org
twitter: http://twitter.com/dstanek
www: http://dstanek.com
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] Apache2 vs uWSGI vs ...

[David Stanek]

I thoughts below mention Keystone, but in reality I would apply the same
logic to any OpenStack service.


On Fri, Sep 18, 2015 at 10:32 AM, Boris Bobrov  wrote:

> There are 2 dimensions this discussion should happen in: web server and
> application server. Now we use apache2 as web server and mod_wsgi as app
> server.
>

This is exactly true and Keystone should be deployable in an WSGI compliant
application server. If it's not I would consider it a bug.



>
> I don't have a specific opinion on the app server (mod_wsgi vs uwsgi) and I
> don't really care.
>
> Regarding apache2 vs nginx. I don't see any reasons for the switch.
> Apache2 is
> well known to deployers and sysadmins. It is very rich for modules. I
> wonder
> if there are customer-written modules.
>

Keystone doesn't use or require Apache. We recommend that it is deployed
using Apache, but there is no requirement to do that if you don't need to
use any Apache modules. For example, at least one of my devstack nodes
happily runs Keystone's manage_all.

 [snip]



> There are things in keystone that work under apache. They are not tested.
> They
> were written to work under apache because it's the simplest and the most
> standard way to do. Making them work in nginx means forcing developers
> write
> some code. You're ready to do that?
>

This should only be true for optional features and currently require Apache
modules.



>
> > May be someone does not need something that apache supports and nginx not
> > and needs nginx features which apache does not support. Let's let our
> users
> > decide what they want.
> >
> > And the first step should be simple here - support for uwsgi.
>
> Why uwsgi? Why not gunicorn? Cherrypy? Twisted?
>

uwsgi and gunicorn should both work fine, as should any WSGI application
server. CherryPy and Twister are more framework than application server, so
I would not expect them to work.


-- 
David
blog: http://www.traceback.org
twitter: http://twitter.com/dstanek
www: http://dstanek.com
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [keystone] PTL Candidacy

[David Stanek]

Hello everyone,

I'd like to announce my candidacy for the Keystone PTL for the Mitaka cycle.
The project has had great leadership the entire time I have been involved
and
I want to keep up the tradition.

I've been working on Keystone for the past 2 years and I'd like to step up
and take more of a leadership role. I have a long track record of technical
leadership that can be directly applied to Keystone. This includes
everything
from project management to application architecture and many things in
between.

My thoughts on the direction of the project really boil down to landing the
features we have already committed to, taking our experimental features and
making them stable, improving general project stability and expanding our
community.

Goals for this cycle:

* Land the features

  Several features that are very beneficial to the community are currently
  in development. We need to get them stable and shipped for Mitaka. This
  includes, but is not limited to federation, reseller, and centralized
  policy. There is also additional work that needs to be done with
  federation to take it from experimental to stable, like helping other
  projects adapt to the new requirements and making it a first class
OpenStack
  feature.

* Fix the bugs

  We have 293 bugs (at the time of this writing) that need some attention.
  Many have patches that need some work or just need reviews. Others need
  to be investigated and fixed. I'd like to cut this number in half. To do
  that I'd like to get people to focus more on them through activities such
  as bug reviews and bug days (more on that below).

* Expand out testing

  Over the last two cycles we have made some significant advances in our
  testing practices. We need to expand on this cultural shift and even
expand
  the focus on testing.

  The full test runtime needs to be cut by at least 50% to improve developer
  workflow. Near immediate test feedback is important for not breaking flow
  when writing code. This can be accomplished by refactoring test code to
  reduce unnecessary setup and focus on the code being tested.

* Expand our community

  Being PTL isn't about me making all of the decisions or calling all of the
  shots. It's about facilitating the design and development of Keystone by
  working with the community. Through mentoring we can get more developers
  ready to be core to speed up our review pace. We need to work together to
  find ways to give more people the ability to contribute upstream. I do
  believe it's possible to make our thriving community even better.

Thank you for voting for me as your PTL for the Mitaka release cycle.


-- David Stanek
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [keystone] PTL non-candidacy

[David Stanek]

On Thu, Sep 10, 2015 at 5:40 PM, Morgan Fainberg 
wrote:

> While I will be changing my focus to spend more time on the general needs
> of OpenStack and working on the Public Cloud story, I am confident in those
> who can, and will, step up to the challenges of leading development of
> Keystone and the associated projects. I may be working across more
> projects, but you can be assured I will be continuing to work hard to see
> the initiatives I helped start through. I wish the best of luck to the next
> PTL.


It's been an honor and a privilege to work with you. You've done a great
job and I'm sorry to see you go. Fortunately you're not going too far! Good
luck with your future OpenStack adventures!


-- 
David
blog: http://www.traceback.org
twitter: http://twitter.com/dstanek
www: http://dstanek.com
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [keystone] creating new users with invalid mail addresses possible

[David Stanek]

On Fri, Sep 11, 2015 at 8:26 AM, Christian Berendt 
wrote:

> At the moment it is possible to create new users with invalid mail
> addresses. I pasted the output of my test at
> http://paste.openstack.org/show/456642/. (the listing of invalid mail
> addresses is available at
> http://codefool.tumblr.com/post/15288874550/list-of-valid-and-invalid-email-addresses
> ).
>
> Is it intended that addresses are not be validated?
>
> Does it makes sense to validate addresses (e.g. with
> https://github.com/mailgun/flanker)?
>

I don't know the complete history of this (I'm sure others can chime in
later), but since Keystone doesn't use the email address for anything it
was never really considered a first class attribute. It is just something
we accept and return through the API. It doesn't even have its own column
in the database.

I don't like this for a variety of reasons and we do have a bug[1] for
fixing this. Last Thursday several of us were discussing making a database
column for the email address as part of the fix for that bug.

1. https://bugs.launchpad.net/keystone/+bug/1218682

-- 
David
blog: http://www.traceback.org
twitter: http://twitter.com/dstanek
www: http://dstanek.com
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] FFE Request for completion of data driven assignment testing in Keystone

[David Stanek]

On Thu, Sep 3, 2015 at 3:44 PM Henry Nash  wrote:

>
> I would like to request an FFE for the remaining two patches that are
> already in review (https://review.openstack.org/#/c/153897/ and
> https://review.openstack.org/#/c/154485/).  These contain only test code
> and no functional changes, and increase our test coverage - as well as
> enable other items to be re-use the list_role_assignment backend method.
>

Do we need a FFE for changes to tests?

-- David
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Keystone][Fernet] HA SQL backend for Fernet keys

[David Stanek]

On Sat, Aug 1, 2015 at 8:03 PM, Boris Bobrov bbob...@mirantis.com wrote:

 On Sat, Aug 1, 2015 at 3:41 PM, Clint Byrum cl...@fewbar.com wrote:

  This too is overly complex and will cause failures. If you replace key 0,

  you will stop validating tokens that were encrypted with the old key 0.



 No. Key 0 is replaced after rotation.



 Also, come on, does http://paste.openstack.org/show/406674/ look overly
 complex? (it should be launched from Fuel master node).


I'm reading this on a small phone, so I may have it wrong, but the script
appears to be broken.

It will ssh to node-1 and rotate. In the simplest case this takes key 0 and
moves it to the next highest key number. Then a new key 0 is generated.

Later there is a loop that will again ssh into node-1 and run the rotation
script. If there is a limit set on the number of keys and you are at that
limit a key will be deleted. This extra rotation on node-1 means that it's
possible that it has a different set of keys than are on node-2 and node-3.

What's the issue with just a simple rsync of the directory?

-- 
David
blog: http://www.traceback.org
twitter: http://twitter.com/dstanek
www: http://dstanek.com
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Keystone][Fernet] HA SQL backend for Fernet keys

[David Stanek]

On Mon, Aug 3, 2015 at 7:14 AM, Davanum Srinivas dava...@gmail.com wrote:

 agree. Native HA solution was already ruled out in several email
 threads by keystone cores already (if i remember right). This is a
 devops issue and should be handled as such was the feedback.


I'm sure you are right. I'm not sure why we would want to add that much
complexity into Keystone.


-- 
David
blog: http://www.traceback.org
twitter: http://twitter.com/dstanek
www: http://dstanek.com
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [keystone][reseller] New way to get a project scoped token by name

[David Stanek]

On Thu, Jun 4, 2015 at 10:10 AM Rodrigo Duarte rodrigodso...@gmail.com
wrote:


 Also, if we are going to use a delimiter, we need to update the way
 projects names are returned in the GET v3/projects API to include the
 hierarchy so the user (or client) knows how to request a token using the
 project name.


This comment made me think of something (maybe crazy)...

I'm hoping that we are not expecting a client to understand how to create
the hierarchical representation of the fully qualified project name and
instead feed it to them in the JSON responses.

If this is the case we can probably control the delimiter on the server
side without the client even knowing that there is a delimiter. We could
have the first character of this new property actually contain the
delimiter used.

  .A.B.C - would mean that . is the delimiter
  - vs -
  #A#B#C - would mean that # is the delimiter

The challenge left is to figure out which delimiter to use.

-- David
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [nova][security] Enable user password complexity verification

[David Stanek]

On Wed, Jun 3, 2015 at 6:04 AM liusheng liusheng1...@126.com wrote:

  Thanks for this topic, also, I think it is similar situation when talking
 about keystone users, not only the instances's password.


In the past we've talked about having more advanced password management
features in Keystone (complexity checks, rotation, etc). The end result is
that we are not adding them because we would like to get away from managing
users in Keystone that way. Instead we are pushing for users to integrate
Keystone with more fully featured identity products.



 在 2015/6/3 17:48, 郑振宇 写道:

 Hi All,

  The current OpenStack does not provide user password complexity
 verification option.


   When performing actions such as create instances, evacuate instances,
 rebuild instances, rescue instances and update instances' admin password.
 The complexity of user provided admin password has not been verified. This
 can cause security problems.

  One solution will be adding a configuration option:
 using_complex_admin_password = True, if this option is set in configure
 file by administrator, then Nova will perform password complexity checks,
 the check standards can be set to following the IT industry general
 standard, if the provided admin password is not complex enough, an
 exception will be throw. If this option is not set in configure file, then
 the complexity check will be skipped.

  When the user dose not provide admin password, generate_password() in
 utils.py is used to generate an admin password. Generate_password() now
 uses two password symbol groups: default and easier, the default symbol
 group contains numbers, upper case letters and small case letters. the
 easier symbol group contains only numbers and upper case letters. The
 generated password is not complex enough and can also cause security
 problems.

  One possible solution is to add a new symbol group:
 STRONGER_PASSWORD_SYMBOLS which contains numbers, upper case letters, lower
 case letters and also special characters such as `~!@#$%^*()-_=+ and
 space. Then adding a new option in configuration file:
 generate_strong_password = True, when this option is set, nova will
 generate password using STRONGER_PASSWORD_SYMBOLS symbol group and with
 longer password length. If this option is not set, the password will be
 generated using the default symbol group and default length.

  AWS allows the selection of password policy to configure which kind of
 password complexity is used in the cloud. Please see:

 http://docs.aws.amazon.com/IAM/latest/UserGuide/Using_ManagingPasswordPolicies.html

  And about the standard of complexity, Microsoft also have an advise
 about it, please see:
 https://technet.microsoft.com/en-us/library/hh994562%28v=ws.10%29.aspx

  Thanks,
 BR,
 Zhenyu Zheng


 __
 OpenStack Development Mailing List (not for usage questions)
 Unsubscribe: 
 openstack-dev-requ...@lists.openstack.org?subject:unsubscribehttp://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


  __
 OpenStack Development Mailing List (not for usage questions)
 Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all][code quality] Voting coverage job (-1 if coverage get worse after patch)

[David Stanek]

On Sat, Apr 18, 2015 at 9:30 PM, Boris Pavlovic bo...@pavlovic.me wrote:

 Code coverage is one of the very important metric of overall code quality
 especially in case of Python. It's quite important to ensure that code is
 covered fully with well written unit tests.

 One of the nice thing is coverage job.


I really like the idea of adding the coverage job everywhere so that
developers can view the results be using a link in Gerrit. I think this
would make it easier for many.

I don't like the idea of checking that coverage is increased. There are
many issues with that. The two biggest one for me are:

1. It will either lead to people doing things to game the system or overuse
of the #no-coverage-check  tag you mentioned.

2. It really doesn't tell you too much. A core developer should really be
looking at the tested use cases to see if they are all there. Line coverage
and even branch coverage won't tell you that.


-- 
David
blog: http://www.traceback.org
twitter: http://twitter.com/dstanek
www: http://dstanek.com
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Keystone] SQLite support (migrations, work-arounds, and more), is it worth it?

[David Stanek]

On Mon, Apr 6, 2015 at 2:41 PM, Mike Bayer mba...@redhat.com wrote:



 On 4/6/15 12:49 PM, David Stanek wrote:

 Exactly. This is the direction I have been going. Functional tests are
 written using the public APIs using the client.

 I would also add that I don't like that the Keystone unit tests are so
 database heavy. I would not want MySQL or ant RDBMS to be a requirement for
 running the tests.

 is that because you'd prefer that the unit tests were more isolated, or
 just that an external service is being used?


There are two types of testing that I'm interested in really promoting in
Keystone. I want to see functional tests that only use the published APIs
to trigger and observe behavior. I also want to see unit tests that are
small, fast and that test the business logic of the system.

I started working[1] on functional testing in the Kilo cycle. The spec[2]
talks about standing up Keystone in different configurations (MySQL, LDAP,
Federated, etc.) and then running a set of tests against that instance.
This will allow us the test any backend we want and confirm it works the
way we expect Keystone to work.

On the other hand I want the unit tests to be smaller and faster. The job
of the unit tests should be to test small isolated bits of code and help
guide/evaluate the code design. Except to test the SQL backends I don't see
a need for using a database and even then I'd like to get by without it.


 I've done some work with extensive mocking of SQL databases; specifically
 mocking at the ORM level.  It is nice when you get it to run, but it's also
 a much bigger job to write fine-grained mocks like that, the mocks can be
 brittle in relation to the code they're targeting, and you also need to
 come up with some solution to actually functional test your database access
 code.

 I tend to think that having a mysqld service running is the lesser of two
 evils and you get a lot more code coverage by going all the way out to the
 DB.



The simplicity of the Keystone design is actually very good for dealing
with this (controller - manager - backend). Since backends are where the
actually query work happens we can easily provide a fake for each subsystem
that we want to test. This would allow us the test the business rules in
the manager without a database. The reason that is good is that we'd
probably be better off requiring LDAP (if anything) for some of the
subsystems since that is what they are typically configured to use. I would
rather just leave that level of detail to the functional tests.

1.
https://review.openstack.org/#/q/status:open+project:openstack/keystone+branch:master+topic:functonal-testing,n,z
2. https://review.openstack.org/#/c/153300/l


-- 
David
blog: http://www.traceback.org
twitter: http://twitter.com/dstanek
www: http://dstanek.com
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Keystone] SQLite support (migrations, work-arounds, and more), is it worth it?

[David Stanek]

Exactly. This is the direction I have been going. Functional tests are
written using the public APIs using the client.

I would also add that I don't like that the Keystone unit tests are so
database heavy. I would not want MySQL or ant RDBMS to be a requirement for
running the tests.

On Mon, Apr 6, 2015, 12:42 Morgan Fainberg morgan.fainb...@gmail.com
wrote:



  On Apr 6, 2015, at 09:20, Mike Bayer mba...@redhat.com wrote:
 
 
 
  On 4/6/15 12:06 PM, Clint Byrum wrote:
  Excerpts from Boris Bobrov's message of 2015-04-03 18:29:08 -0700:
  On Saturday 04 April 2015 03:55:59 Morgan Fainberg wrote:
  I am looking forward to the Liberty cycle and seeing the special
 casing we
  do for SQLite in our migrations (and elsewhere). My inclination is
 that we
  should (similar to the deprecation of eventlet) deprecate support for
  SQLite in Keystone. In Liberty we will have a full functional test
 suite
  that can (and will) be used to validate everything against much more
 real
  environments instead of in-process “eventlet-like”
 test-keystone-services;
  the “Restful test cases” will no longer be part of the standard unit
 tests
  (as they are functional testing). With this change I’m inclined to say
  SQLite (being the non-production usable DB) what it is we should look
 at
  dropping migration support for SQLite and the custom work-arounds.
 
  Most deployers and developers (as far as I know) use devstack and
 MySQL or
  Postgres to really suss out DB interactions.
 
  I am looking for feedback from the community on the general stance for
  SQLite, and more specifically the benefit (if any) of supporting it in
  Keystone.
  +1. Drop it and clean up tons of code used for support of sqlite only.
 
  Doing tests with mysql is as easy, as with sqlite (mysqladmin drop -f;
  mysqladmin create for reset), and using it by default will finally
 make
  people test their code on real rdbmses.
  Please please please be careful with that and make sure the database
  name is _always_ random in tests... or even better, write a fixture to
  spin up a mysqld inside a private tempdir. That would be a really cool
  thing for oslo.db to provide actually.
 
  I'm just thinking some poor sap runs the test suite with the wrong
  .my.cnf in the wrong place and poof there went keystone's db.
  The standard approach here is that tests based on the oslo.db approach
 by default connect using a username openstack_citest on localhost, which is
 then used to create databases of random names. If you base your database
 tests on oslo.db, you get that right now.   This username/host/etc is also
 configurable through environment variables.  I don't see how my.cnf is
 involved in this nor how it would impact someone's keystone database,
 unless we're talking about tests that happen to load down and/or crash the
 whole database server.
 
  spinning up a whole mysqld seems really heavy-handed and unnecessary.
 Not to mention the tests run on other backends as well such as Postgresql.
 

 The reasons outlined by both Clint and Mike are why we won't be attempting
 this until we are happy with our functional test suite. But once we are
 happy dropping SQLite is on the table. The way I see it the functional
 tests should be performed against a real keystone server, even if it is one
 spun up for testing specifically.

 Per test db creation / other such stuff will be part of that discussion.
 __
 OpenStack Development Mailing List (not for usage questions)
 Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [keystone][fernet] Fernet tokens sync

[David Stanek]

On Fri, Mar 27, 2015 at 10:14 AM, Boris Bobrov bbob...@mirantis.com wrote:

 As you know, keystone introduced non-persistent tokens in kilo -- Fernet
 tokens. These tokens use Fernet keys, that are rotated from time to time. A
 great description of key rotation and replication can be found on [0] and
 [1]
 (thanks, lbragstad). In HA setup there are multiple nodes with Keystone and
 that requires key replication. How do we do that with new Fernet tokens?

 Please keep in mind that the solution should be HA -- there should not be
 any
 master server, pushing keys to slave servers, because master server
 might go
 down.


In my test environment I was using ansible to sync the keys across multiple
nodes. Keystone should probably provide some guidance around this process,
but I don't think it should deal with the actual syncing. I think that's
better left to an installation's existing configuration management tools.


-- 
David
blog: http://www.traceback.org
twitter: http://twitter.com/dstanek
www: http://dstanek.com
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Keystone]ON DELETE RESTRICT VS ON DELETE CASCADE

[David Stanek]

On Sun, Mar 8, 2015 at 10:28 PM, Chen, Wei D wei.d.c...@intel.com wrote:

 +1,



 I am fan of checking the constraints in the controller level instead of
 relying on FK constraints itself, thanks.


The Keystone controllers shouldn't do any business logic. This should be in
the managers. The controllers should do nothing more that take web stuff
and convert it for use by the managers.


-- 
David
blog: http://www.traceback.org
twitter: http://twitter.com/dstanek
www: http://dstanek.com
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Keystone]ON DELETE RESTRICT VS ON DELETE CASCADE

[David Stanek]

On Sun, Mar 8, 2015 at 1:37 PM, Mike Bayer mba...@redhat.com wrote:

 can you elaborate on your reasoning that FK constraints should be used less
 overall?  or do you just mean that the client side should be mirroring the
 same
 rules that would be enforced by the FKs?


I don't think he means that we will use them less.  Our SQL backends are
full of them.  What Keystone can't do is rely on them because not all
implementations of our backends support FKs.


-- 
David
blog: http://www.traceback.org
twitter: http://twitter.com/dstanek
www: http://dstanek.com
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [keystone] Output on stderr

[David Stanek]

On Wed, Mar 4, 2015 at 6:50 AM, Abhishek Talwar/HYD/TCS 
abhishek.tal...@tcs.com wrote:

 While working on a bug for keystoneclient I have replaced sys.exit with
 return. However, the code reviewers want that the output should be on
 stderr(as sys.exit does). So how can we get the output on stderr.


The print function allows you to specify a file:

 from __future__ import print_function
 import sys
 print('something to', file=sys.stderr)

The __future__ import is needed for Python 2.6/2.7 because print was
changed to a function in Python 3.


-- 
David
blog: http://www.traceback.org
twitter: http://twitter.com/dstanek
www: http://dstanek.com
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Keystone] Proposing Marek Denis for the Keystone Core Team

[David Stanek]

+1

On Tue, Feb 10, 2015 at 12:51 PM, Morgan Fainberg morgan.fainb...@gmail.com
 wrote:

 Hi everyone!

 I wanted to propose Marek Denis (marekd on IRC) as a new member of the
 Keystone Core team. Marek has been instrumental in the implementation of
 Federated Identity. His work on Keystone and first hand knowledge of the
 issues with extremely large OpenStack deployments has been a significant
 asset to the development team. Not only is Marek a strong developer working
 on key features being introduced to Keystone but has continued to set a
 high bar for any code being introduced / proposed against Keystone. I know
 that the entire team really values Marek’s opinion on what is going in to
 Keystone.

 Please respond with a +1 or -1 for adding Marek to the Keystone core team.
 This poll will remain open until Feb 13.

 --
 Morgan Fainberg

 __
 OpenStack Development Mailing List (not for usage questions)
 Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev




-- 
David
blog: http://www.traceback.org
twitter: http://twitter.com/dstanek
www: http://dstanek.com
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Keystone] Nominating Brad Topol for Keystone-Spec core

[David Stanek]

+1

On Sun, Jan 18, 2015 at 2:11 PM, Morgan Fainberg morgan.fainb...@gmail.com
wrote:

 Hello all,

 I would like to nominate Brad Topol for Keystone Spec core (core reviewer
 for Keystone specifications and API-Specification only:
 https://git.openstack.org/cgit/openstack/keystone-specs ). Brad has been
 a consistent voice advocating for well defined specifications, use of
 existing standards/technology, and ensuring the UX of all projects under
 the Keystone umbrella continue to improve. Brad brings to the table a
 significant amount of insight to the needs of the many types and sizes of
 OpenStack deployments, especially what real-world customers are demanding
 when integrating with the services. Brad is a core contributor on pycadf
 (also under the Keystone umbrella) and has consistently contributed code
 and reviews to the Keystone projects since the Grizzly release.

 Please vote with +1/-1 on adding Brad to as core to the Keystone Spec
 repo. Voting will remain open until Friday Jan 23.

 Cheers,
 Morgan Fainberg


 __
 OpenStack Development Mailing List (not for usage questions)
 Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev




-- 
David
blog: http://www.traceback.org
twitter: http://twitter.com/dstanek
www: http://dstanek.com
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Glance] IRC logging

[David Stanek]

It's also important to remember that IRC channels are typically not private
and are likely already logged by dozens of people anyway.

On Tue, Jan 6, 2015 at 1:22 PM, Christopher Aedo ca...@mirantis.com wrote:

 On Tue, Jan 6, 2015 at 2:49 AM, Flavio Percoco fla...@redhat.com wrote:
  Fully agree... I don't see how enable logging should be a limitation
  for freedom of thought. We've used it in Zaqar since day 0 and it's
  bee of great help for all of us.
 
  The logging does not remove the need of meetings where decisions and
  more relevant/important topics are discussed.

 Wanted to second this as well.  I'm strongly in favor of logging -
 looking through backlogs of chats on other channels has been very
 helpful to me in the past, and it sure to help others in the future.
 I don't think there is danger of anyone pointing to a logged IRC
 conversation in this context as some statement of record.

 -Christopher

 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev




-- 
David
blog: http://www.traceback.org
twitter: http://twitter.com/dstanek
www: http://dstanek.com
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Keystone] Alternative federation mapping

[David Stanek]

On Tue, Nov 4, 2014 at 10:30 AM, John Dennis jden...@redhat.com wrote:

 O.K. group assignment is the final goal in Keystone. I suppose the
 relevant question then is the functionality in the current Keystone
 mapper sufficiently rich such that you can present to it an arbitrary
 set of values and yield a group assessment? It's been a while since I
 looked at the mapper, things might have changed, but it seemed to me it
 had a lot of baked in assumptions about the data (assertion) it would
 receive. As long as those assumptions held true all is good.


There are probably a few other assumptions, but the main one is that the
mapper expects the incoming data to be a dictionary where the value is a
string. If there are multiple values we expect them to be delimited with a
semicolon in the string.


-- 
David
blog: http://www.traceback.org
twitter: http://twitter.com/dstanek
www: http://dstanek.com
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Keystone] external AuthN Identity Backend

[David Stanek]

On Thu, Oct 16, 2014 at 2:54 PM, Dave Walker em...@daviey.com wrote:

 Hi Steve,

 Thanks for your response.  I am talking generally about the external
 auth support.  One use case is Kerberos, but for the sake of argument
 this could quite easily be Apache Basic auth.  The point is, we have
 current support for entrusting AuthN outside of Keystone.

 What I was trying to outline is that it seems that the current design
 of external auth is that keystone is not in the auth pipeline as we
 trust auth at the edge.  However, we then do additional auth within
 keystone.

 With external auth and SQL, we drop the user provided username and
 password on the floor and use what was provided in REMOTE_USER (set by
 the webserver).

 Therefore the check as it currently stands in SQL is basically 'is
 this username in the database'.  The LDAP plugin does Authentication
 via username and password, which is clearly not sufficient for
 external auth.  The LDAP plugin could be made to check in a similar
 manner to SQL 'is this a valid user' - but this would seem to be a
 duplicate check, as we already did this at the edge.

 If the webserver granted access to keystone, the user has already been
 checked to see if they are a valid user.  However, your response seems
 to suggest that current external auth should be formally deprecated?


I may be missing something, but can you use the external auth method with
the LDAP backend?

-- 
David
blog: http://www.traceback.org
twitter: http://twitter.com/dstanek
www: http://dstanek.com
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Nova] [All] API standards working group

[David Stanek]

On Wed, Sep 24, 2014 at 11:42 AM, Dean Troyer dtro...@gmail.com wrote:

 On Tue, Sep 23, 2014 at 5:18 PM, Jay Pipes jaypi...@gmail.com wrote:

 Yes, I'd be willing to head up the working group... or at least
 participate in it.


 I'll bring an API consumer's perspective.



I would love to participate too. I have an interest in RESTful API design
and the surrounding architecture.

-- 
David
blog: http://www.traceback.org
twitter: http://twitter.com/dstanek
www: http://dstanek.com
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Ceilometer] Question on decorators in Ceilometer pecan framework

[David Stanek]

It looks like maybe WSME or Pecan is inspecting the method signature. Have you 
tried to change the order of the decorators?


On Aug 8, 2014, at 9:16, Pendergrass, Eric eric.pendergr...@hp.com wrote:

 Wrong link again, this is embarrassing L 
 https://review.openstack.org/#/c/112137/3
  
 From: Pendergrass, Eric 
 Sent: Friday, August 08, 2014 7:15 AM
 To: openstack-dev@lists.openstack.org
 Subject: RE: [Ceilometer] Question on decorators in Ceilometer pecan framework
  
 Sorry, wrong BP review link below.  Here is the correct one: 
 https://review.openstack.org/#/c/112127/3.  Please disregard the wiki link.
  
 From: Pendergrass, Eric 
 Sent: Friday, August 08, 2014 6:50 AM
 To: openstack-dev@lists.openstack.org
 Cc: Giannetti, Fabio
 Subject: [Ceilometer] Question on decorators in Ceilometer pecan framework
  
 Hi,
  
 We have been struggling to get a decorator working for proposed new RBAC 
 functionality in ceilometer-api.  We’re hitting a problem where GET request 
 query parameters are mucked up by our decorator.  Here’s an example call:
  
 curl -H X-Auth-Token:$TOKEN 
 'http://localhost:8777/v2/meters?q.field=project_idq.value=8c678720fb5b4e3bb18dee222d7d7933'
  
 And here’s the decorator method (we’ve tried changing the kwargs, args, etc. 
 with no luck):
  
 _ENFORCER = None
  
 def protected(controller_class):
  
 global _ENFORCER
 if not _ENFORCER:
 _ENFORCER = policy.Enforcer()
  
 def wrapper(f):
 @functools.wraps(f)
 def inner(self, **kwargs):
 pdb.set_trace()
 self._rbac_context = {}
 if not _ENFORCER.enforce('context_is_admin',
  {},
  {'roles': 
 pecan.request.headers.get('X-Roles', ).split(,)}):
 self._rbac_context['project_id'] = 
 pecan.request.headers.get('X-Project-Id')
 self._rbac_context['user_id'] = 
 pecan.request.headers.get('X-User-Id')
 return f(self, **kwargs)
 return inner
 return wrapper
  
 tried this too:
  
 _ENFORCER = None
  
 def protected(*args):
  
 controller_class = 'meter'
 global _ENFORCER
 if not _ENFORCER:
 _ENFORCER = policy.Enforcer()
  
 def wrapper(f, *args):
 def inner(self, *args):
 pdb.set_trace()
 #self._rbac_context = {}
 #if not _ENFORCER.enforce('context_is_admin',
 # {},
 # {'roles': 
 pecan.request.headers.get('X-Roles', ).split(,)}):
 #self._rbac_context['project_id'] = 
 pecan.request.headers.get('X-Project-Id')
 #self._rbac_context['user_id'] = 
 pecan.request.headers.get('X-User-Id')
 #return f(*args)
 f(self, *args)
 return inner
 return wrapper
  
 and here’s how it’s used:
  
 class MetersController(rest.RestController):
 Works on meters.
  
 _rbac_context = {}
 @pecan.expose()
 def _lookup(self, meter_name, *remainder):
 return MeterController(meter_name), remainder
  
 @wsme_pecan.wsexpose([Meter], [Query])
 @rbac_validate.protected('meters')
 def get_all(self, q=None):
 Return all known meters, based on the data recorded so far.
  
 :param q: Filter rules for the meters to be returned.
 
 q = q or [] …
  
  
 but we get errors similar to below where the arg parser cannot find the query 
 parameter because the decorator doesn’t take a q argument as 
 MetersController.get_all does. 
  
 Is there any way to get a decorator to work within the v2 API code and wsme 
 framework or should we consider another approach?  Decorators would really 
 simplify the RBAC idea we’re working on, which is mostly code-implemented 
 save for this fairly major problem.
  
 I have a WIP registered BP on this 
 athttps://blueprints.launchpad.net/ceilometer/+spec/ready-ceilometer-rbac-keystone-v3.
  
 If I can provide more details I’ll be happy to.
  
 Thanks
 Eric
  
   /usr/local/bin/ceilometer-api(10)module()
 - sys.exit(api())
   /opt/stack/ceilometer/ceilometer/cli.py(96)api()
 - srv.serve_forever()
   /usr/lib/python2.7/SocketServer.py(227)serve_forever()
 - self._handle_request_noblock()
   /usr/lib/python2.7/SocketServer.py(284)_handle_request_noblock()
 - self.process_request(request, client_address)
   /usr/lib/python2.7/SocketServer.py(310)process_request()
 - self.finish_request(request, client_address)
   /usr/lib/python2.7/SocketServer.py(323)finish_request()
 - self.RequestHandlerClass(request, client_address, self)
   /usr/lib/python2.7/SocketServer.py(638)__init__()
 - self.handle()
   /usr/lib/python2.7/wsgiref/simple_server.py(124)handle()
 - handler.run(self.server.get_app())
   /usr/lib/python2.7/wsgiref/handlers.py(85)run()
 - self.result = application(self.environ, self.start_response)
   
 

Re: [openstack-dev] [Barbican][OSSG][Keystone] Mid-Cycle Meetup

[David Stanek]

On Thu, May 22, 2014 at 10:48 AM, Jarret Raim jarret.r...@rackspace.comwrote:


 This should make travel a bit easier for everyone as people won't need


Hey Jarret,

I'm going to be at the Keystone meetup for sure, but I'm also thinking
about going to the Barbican meetup too.

-- 
David
blog: http://www.traceback.org
twitter: http://twitter.com/dstanek
www: http://dstanek.com
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [oslo] Proposal: add local hacking for oslo-incubator

[David Stanek]

On Mon, May 5, 2014 at 5:28 PM, Doug Hellmann
doug.hellm...@dreamhost.comwrote:


  The assert ones do seem to fit the best practices as I understand them,
 but
  I suspect there's going to be quite a bit of work to get projects
 compliant.

 I've seen some work being done on that already, but I don't know how
 strongly we care about those specific rules as an overall project.


I created the Keystone blueprint[1] to automate the things we already check
for in reviews. My motivation was to make it faster for contributors to
contribute because they would get feedback before getting a bunch of -1s in
Gerrit. I also wanted to free up core dev resources so that we can focus on
more important parts of reviews.

I'd be happy to start putting some of these in hacking, but I don't know
which rules would be acceptable to all projects. Maybe there is a way to
make optional checks that can be enabled in the tox.ini

1.
https://blueprints.launchpad.net/keystone/+spec/more-code-style-automation

-- 
David
blog: http://www.traceback.org
twitter: http://twitter.com/dstanek
www: http://dstanek.com
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] Hierarchicical Multitenancy Discussion

[David Stanek]

That's why I love this site:
http://www.timeanddate.com/worldclock/fixedtime.html?iso=20140130T2100


On Thu, Jan 30, 2014 at 1:46 PM, Vishvananda Ishaya
vishvana...@gmail.comwrote:

 Thanks Soren, you are correct! Yay Timezones

 Vish

 On Jan 30, 2014, at 10:39 AM, Soren Hansen so...@linux2go.dk wrote:

 2100 UTC is 1 PM Pacific. :-)
 Den 29/01/2014 17.01 skrev Vishvananda Ishaya vishvana...@gmail.com:

 I apologize for the confusion. The Wiki time of 2100 UTC is the correct
 time (Noon Pacific time). We can move tne next meeting to a different
 day/time that is more convienient for Europe.

 Vish


 On Jan 29, 2014, at 1:56 AM, Florent Flament 
 florent.flament-...@cloudwatt.com wrote:

  Hi Vishvananda,
 
  I would be interested in such a working group.
  Can you please confirm the meeting hour for this Friday ?
  I've seen 1600 UTC in your email and 2100 UTC in the wiki (
 https://wiki.openstack.org/wiki/Meetings#Hierarchical_Multitenancy_Meeting). 
 As I'm in Europe I'd prefer 1600 UTC.
 
  Florent Flament
 
  - Original Message -
  From: Vishvananda Ishaya vishvana...@gmail.com
  To: OpenStack Development Mailing List (not for usage questions) 
 openstack-dev@lists.openstack.org
  Sent: Tuesday, January 28, 2014 7:35:15 PM
  Subject: [openstack-dev] Hierarchicical Multitenancy Discussion
 
  Hi Everyone,
 
  I apologize for the obtuse title, but there isn't a better succinct
 term to describe what is needed. OpenStack has no support for multiple
 owners of objects. This means that a variety of private cloud use cases are
 simply not supported. Specifically, objects in the system can only be
 managed on the tenant level or globally.
 
  The key use case here is to delegate administration rights for a group
 of tenants to a specific user/role. There is something in Keystone called a
 domain which supports part of this functionality, but without support
 from all of the projects, this concept is pretty useless.
 
  In IRC today I had a brief discussion about how we could address this.
 I have put some details and a straw man up here:
 
  https://wiki.openstack.org/wiki/HierarchicalMultitenancy
 
  I would like to discuss this strawman and organize a group of people to
 get actual work done by having an irc meeting this Friday at 1600UTC. I
 know this time is probably a bit tough for Europe, so if we decide we need
 a regular meeting to discuss progress then we can vote on a better time for
 this meeting.
 
 
 https://wiki.openstack.org/wiki/Meetings#Hierarchical_Multitenancy_Meeting
 
  Please note that this is going to be an active team that produces code.
 We will *NOT* spend a lot of time debating approaches, and instead focus on
 making something that works and learning as we go. The output of this team
 will be a MultiTenant devstack install that actually works, so that we can
 ensure the features we are adding to each project work together.
 
  Vish
 
  ___
  OpenStack-dev mailing list
  OpenStack-dev@lists.openstack.org
  http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
 
  ___
  OpenStack-dev mailing list
  OpenStack-dev@lists.openstack.org
  http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev




-- 
David
blog: http://www.traceback.org
twitter: http://twitter.com/dstanek
www: http://dstanek.com
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [qa][keystone] Keystoneclient tests to tempest

[David Stanek]

On Sun, Dec 8, 2013 at 3:37 PM, Matt Riedemann
mrie...@linux.vnet.ibm.comwrote:



 On Sunday, December 08, 2013 11:26:07 AM, Brant Knudson wrote:


 We'd like to get the keystoneclient tests out of keystone. They're
 serving a useful purpose of catching problems with non-backwards
 compatible changes in keystoneclient so we still want them run.
 Problem is they're running at the wrong time -- only on changes to
 keystone and not changes to keystoneclient.

 The tests need to be run:

 When keystoneclient changes
  - run the tests against the change

 When the tests change
  - run the change against the current keystoneclient and also old clients

 When keystone changes
  - run the tests against the change with current client

 So here's what I think we need to do to get keystone client tests out
 of keystone:

  1) Figure out where to put the tests - is it tempest or something else?
  2) Write up a test and put it there
  3) Have a job that when there's a change in the tests it runs against
 current client lib
  4) Expand the job to also run against old clients
 - or is there 1 job per version?
 - what versions? (keystone does master, essex-3, and 0.1.1)
 - e.g. tox -e master,essex-3,0.1.1
 - suggest start with these versions and then consider what to use
 in future
  5) Now we can start adding tests
  6) Have a job that when there's a change in keystoneclient it runs
 these tests against the change
  7) When there's a change in keystone, run these tests against the change
  8) Copy the keystoneclient tests from keystone to the new location --
 will require some changes
  9) Remove the tests from keystone \o/
 10) Move tests back to keystone where makes sense -- use webtest like
 v3 tests

 I created an etherpad with this same info so it's easier to discuss:
 https://etherpad.openstack.org/p/KeystoneTestsToTempest

 - Brant


 I'll ask the super obvious question, why not move the keystoneclient tests
 to keystoneclient?


I believe Brant is talking about the tests that use different versions of
the keystone client against the keystone server.


-- 
David
blog: http://www.traceback.org
twitter: http://twitter.com/dstanek
www: http://dstanek.com
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Keystoneclient] [Keystone] [Solum] Last released version of keystoneclient does not work with python33

[David Stanek]

On Wed, Dec 4, 2013 at 6:44 PM, Adrian Otto adrian.o...@rackspace.comwrote:

 Jamie,

 Thanks for the guidance here. I am checking to see if any of our
 developers might take an interest in helping with the upstream work. At the
 very least, it might be nice to have some understanding of how much work
 there is to be done in HTTPretty.


(Dolph correct me if I am wrong, but...)

I don't think that there is much work to be done beyond getting that pull
request merged upstream.  Dolph ran the tests using the code from the pull
request somewhat successfully.  The errors that we saw were just in
keystoneclient code.

-- 
David
blog: http://www.traceback.org
twitter: http://twitter.com/dstanek
www: http://dstanek.com
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [style] () vs \ continuations

[David Stanek]

On Mon, Nov 18, 2013 at 8:23 PM, Vishvananda Ishaya
vishvana...@gmail.comwrote:


 +1 to sticking something in hacking. FWIW I would probably do the following
 to avoid the debate altogether:

 result = self._path_file_exists(ds_browser, folder_path, file_name)
 folder_exists, file_exists, file_size_in_kb, disk_extents = result

 Vish


I'm working on some test code and came up with what I think is a valid use
of a line ending in \.  I'm not escaping the newline from a Python syntax
point of view.  I'm doing in inside of a string literal.  Example:

mismatches_description = textwrap.dedent(\
expected = test xmlns=http://docs.openstack.org/identity/api/v2.0

  success/
/test

actual = test xmlns=http://docs.openstack.org/identity/api/v2.0;
  nope_it_fails/
/test
)

This can be written in several different ways, but I think that none of
them is as clear as the trailing slash.  This is, I think, the clearest
alternative example:

mismatches_description = textwrap.dedent(
expected = test xmlns=http://docs.openstack.org/identity/api/v2.0

  success/
/test

actual = test xmlns=http://docs.openstack.org/identity/api/v2.0;
  nope_it_fails/
/test
).lstrip()

Would this use be forbidden too?

-- 
David
blog: http://www.traceback.org
twitter: http://twitter.com/dstanek
www: http://dstanek.com
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] Bad review patterns

[David Stanek]

On Wed, Nov 6, 2013 at 7:21 PM, Day, Phil philip@hp.com wrote:

 
  Leaving a mark.
  ===
 
  You review a change and see that it is mostly fine, but you feel that
 since you
  did so much work reviewing it, you should at least find
  *something* wrong. So you find some nitpick and -1 the change just so
 that
  they know you reviewed it.
 
  This is quite obvious. Just don't do it. It's OK to spend an hour
 reviewing
  something, and then leaving no comments on it, because it's simply fine,
 or
  because we had to means to test someting (see the first pattern).
 
 

 Another one that comes into this category is adding a -1 which just says
 I agree with
 the other -1's in here.   If you have some additional perspective and can
 expand on
 it then that's fine - otherwise it adds very little and is just review
 count chasing.

 It's an unfortunate consequence of counting and publishing review stats
 that having
 such a measure will inevitable also drive behavour.


I agree that having no comment with a +1 is OK.  If I think the code looks
good I'll +1 it.  If I think the code could be better I'll -1 it and leave
comments.

I don't think that leaving a -1 with a 'what they said' comment is a bad
thing.  As the developer writing the patch it's helpful to know there is a
consensus and it's not just one person requesting a change.

-- 
David
blog: http://www.traceback.org
twitter: http://twitter.com/dstanek
www: http://dstanek.com
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] Remove vim modelines?

[David Stanek]

On Thu, Oct 24, 2013 at 2:48 PM, Robert Collins
robe...@robertcollins.netwrote:


 *) They help casual contributors *more* than long time core
 contributors : and those are the folk that are most likely to give up
 and walk away. Keeping barriers to entry low is an important part of
 making OpenStack development accessible to new participants.


I'm not sure I buy the low barrier to entry in this case.  We could easily
provide a more comprehensive vimscript snippet that experienced vim users
can include into their vimrc.  Not only that, but we can do the same for
any other editor.  If a user is not familiar with vim, I don't think we
can/should do much to teach users how to use their editor.

-- 
David
blog: http://www.traceback.org
twitter: http://twitter.com/dstanek
www: http://dstanek.com
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] RFC - Icehouse logging harmonization

[David Stanek]

On Wed, Oct 23, 2013 at 3:26 PM, Robert Collins
robe...@robertcollins.netwrote:

 On 24 October 2013 08:14, Dolph Mathews dolph.math...@gmail.com wrote:
 
  On Wed, Oct 23, 2013 at 1:20 PM, Sean Dague s...@dague.net wrote:

 
  Deprecation warnings!
 
  Based on the approach we're taking in the patch below, we'll be able to
  notate how imminently a feature is facing deprecation. Right now, they're
  just landing in WARNING, but I think we'll surely a desire to silence,
  prioritize or redirect those messages using different log levels (for
  example, based on how imminently a feature is facing deprecation).
 
  https://review.openstack.org/#/c/50486/

 Huh, I did not see that go by. Python already has built in signalling
 for deprecated features; I think we should be using that. We can of
 course wrap it with a little sugar to make it easy to encode future
 deprecations.


The initial patch used the the warnings modules instead of using the
deprecated logging provided by oslo.  We talked about it IRC and I switched
because there existed a way to do deprecation messaging is olso and it had
a configurable way to turn warnings into exceptions.

I intend to submit a patch to oslo-incubator based off of the above patch.
 I'd love to get some feedback about the needs of the other projects.


-- 
David
blog: http://www.traceback.org
twitter: http://twitter.com/dstanek
www: http://dstanek.com
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [qa][keystone] Adding client library related tests to tempest

[David Stanek]

On Fri, Oct 18, 2013 at 1:48 PM, Sean Dague s...@dague.net wrote:

 On 10/18/2013 12:04 PM, Brant Knudson wrote:


 2) git cloneing the keystoneclient doesn't work well with parallel
 testing (we have a similar problem in our tests with our pristine
 database backup)


 Can you go into the specifics of why?


We use unsafe paths for the test SQLite database and test config files.
 Instead of using something like tempfile we are using hardcoded paths.
 When the setUp method is run in parallel it will stomp on other tests.  I
believe the 'git clone' is the same way.  The clone happens in the setUp so
if you have 2 test methods in that test class one of the cloning operations
will break.

I have a bug filed for the DB/config file issue already.  The cloning issue
may solved by putting it into the setupClass instead of setUp.  I'd have to
try it.

-- 
David
blog: http://www.traceback.org
twitter: http://twitter.com/dstanek
www: http://dstanek.com
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Review] Use of exception for non-exceptional cases

[David Stanek]

On Thu, Jul 11, 2013 at 5:20 AM, Mark McLoughlin mar...@redhat.com wrote:


 But I think what you're saying is missing is the stack trace from the
 underlying exception.

 As I understood it, Python doesn't have a way of chaining exceptions
 like this but e.g. Java does. A little bit more poking right now shows
 up this:

   http://www.python.org/dev/peps/pep-3134/

 i.e. we can't do the right thing until Python 3, where we'd do:

  def download_image(host, port, path):
  try:
  s = socket.create_connection((host, port))
  except socket.error as e:
  raise ImageDownloadFailure(host, port, path, e.strerror) from e

 I haven't read the PEP in detail yet, though.


You can actually do this in Python 2 and keep the original context:

  def download_image(host, port, path):
  try:
  s = socket.create_connection((host, port))
  except socket.error as e:
  raise ImageDownloadFailure, e, sys.exc_info()[-1]

This will keep the original message and stack trace, but change the type.
 You can also change the message if you want my mucking with e's message.
 I've done that to add a string like  (socket.error) at the end of the
exception message so I could see the original type.

If you really, really wanted to use a bare except you could also do
something like:

  try:
  do_something_that_raises_an_exception()
  except:
  exc_value, exc_tb = sys.exc_info()[1:]
  raise MyException, exc_value, exc_tb


-- 
David
blog: http://www.traceback.org
twitter: http://twitter.com/dstanek
www: http://dstanek.com
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev