Re: [openstack-dev] [all] [clients] [keystone] lack of retrying tokens leads to overall OpenStack fragility

2014-09-17 Thread Duncan Thomas
On 16 September 2014 01:28, Nathan Kinder nkin...@redhat.com wrote: The idea would be to leave normal tokens with a smaller validity period (like the current default of an hour), but also allow one-time use tokens to be requested. Cinder backup makes many requests to swift during a backup, one

Re: [openstack-dev] [all] [clients] [keystone] lack of retrying tokens leads to overall OpenStack fragility

2014-09-17 Thread Angus Lees
On Wed, 17 Sep 2014 04:53:28 PM Duncan Thomas wrote: On 16 September 2014 01:28, Nathan Kinder nkin...@redhat.com wrote: The idea would be to leave normal tokens with a smaller validity period (like the current default of an hour), but also allow one-time use tokens to be requested.

Re: [openstack-dev] [all] [clients] [keystone] lack of retrying tokens leads to overall OpenStack fragility

2014-09-16 Thread Adam Young
On 09/15/2014 08:28 PM, Nathan Kinder wrote: On 09/12/2014 12:46 AM, Angus Lees wrote: On Thu, 11 Sep 2014 03:21:52 PM Steven Hardy wrote: On Wed, Sep 10, 2014 at 08:46:45PM -0400, Jamie Lennox wrote: For service to service communication there are two types. 1) using the user's token like

Re: [openstack-dev] [all] [clients] [keystone] lack of retrying tokens leads to overall OpenStack fragility

2014-09-15 Thread Brant Knudson
On Wed, Sep 10, 2014 at 9:14 AM, Sean Dague s...@dague.net wrote: Going through the untriaged Nova bugs, and there are a few on a similar pattern: Nova operation in progress takes a while Crosses keystone token expiration time Timeout thrown Operation fails Terrible 500 error sent

Re: [openstack-dev] [all] [clients] [keystone] lack of retrying tokens leads to overall OpenStack fragility

2014-09-15 Thread Nathan Kinder
On 09/12/2014 12:46 AM, Angus Lees wrote: On Thu, 11 Sep 2014 03:21:52 PM Steven Hardy wrote: On Wed, Sep 10, 2014 at 08:46:45PM -0400, Jamie Lennox wrote: For service to service communication there are two types. 1) using the user's token like nova-cinder. If this token expires there is

Re: [openstack-dev] [all] [clients] [keystone] lack of retrying tokens leads to overall OpenStack fragility

2014-09-12 Thread Angus Lees
On Thu, 11 Sep 2014 03:00:02 PM Duncan Thomas wrote: On 11 September 2014 03:17, Angus Lees g...@inodes.org wrote: (As inspired by eg kerberos) 2. Ensure at some environmental/top layer that the advertised token lifetime exceeds the timeout set on the request, before making the request.

Re: [openstack-dev] [all] [clients] [keystone] lack of retrying tokens leads to overall OpenStack fragility

2014-09-12 Thread Angus Lees
On Thu, 11 Sep 2014 03:21:52 PM Steven Hardy wrote: On Wed, Sep 10, 2014 at 08:46:45PM -0400, Jamie Lennox wrote: For service to service communication there are two types. 1) using the user's token like nova-cinder. If this token expires there is really nothing that nova can do except raise

Re: [openstack-dev] [all] [clients] [keystone] lack of retrying tokens leads to overall OpenStack fragility

2014-09-12 Thread Flavio Percoco
:49 AM Subject: Re: [openstack-dev] [all] [clients] [keystone] lack of retrying tokens leads to overall OpenStack fragility On Wed, Sep 10, 2014 at 10:14:32AM -0400, Sean Dague wrote: Going through the untriaged Nova bugs, and there are a few on a similar pattern: Nova operation in progress

Re: [openstack-dev] [all] [clients] [keystone] lack of retrying tokens leads to overall OpenStack fragility

2014-09-12 Thread Steven Hardy
: Re: [openstack-dev] [all] [clients] [keystone] lack of retrying tokens leads to overall OpenStack fragility On Wed, Sep 10, 2014 at 08:46:45PM -0400, Jamie Lennox wrote: - Original Message - From: Steven Hardy sha...@redhat.com To: OpenStack Development Mailing List

Re: [openstack-dev] [all] [clients] [keystone] lack of retrying tokens leads to overall OpenStack fragility

2014-09-11 Thread Sean Dague
] [clients] [keystone] lack of retrying tokens leads to overall OpenStack fragility On Wed, Sep 10, 2014 at 10:14:32AM -0400, Sean Dague wrote: Going through the untriaged Nova bugs, and there are a few on a similar pattern: Nova operation in progress takes a while Crosses keystone token

Re: [openstack-dev] [all] [clients] [keystone] lack of retrying tokens leads to overall OpenStack fragility

2014-09-11 Thread Sean Dague
On 09/10/2014 11:55 AM, Steven Hardy wrote: On Wed, Sep 10, 2014 at 10:14:32AM -0400, Sean Dague wrote: Going through the untriaged Nova bugs, and there are a few on a similar pattern: Nova operation in progress takes a while Crosses keystone token expiration time Timeout thrown

Re: [openstack-dev] [all] [clients] [keystone] lack of retrying tokens leads to overall OpenStack fragility

2014-09-11 Thread Duncan Thomas
On 11 September 2014 03:17, Angus Lees g...@inodes.org wrote: (As inspired by eg kerberos) 2. Ensure at some environmental/top layer that the advertised token lifetime exceeds the timeout set on the request, before making the request. This implies (since there's no special handling in place)

Re: [openstack-dev] [all] [clients] [keystone] lack of retrying tokens leads to overall OpenStack fragility

2014-09-11 Thread Steven Hardy
: [openstack-dev] [all] [clients] [keystone] lack of retrying tokens leads to overall OpenStack fragility On Wed, Sep 10, 2014 at 10:14:32AM -0400, Sean Dague wrote: Going through the untriaged Nova bugs, and there are a few on a similar pattern: Nova operation in progress

Re: [openstack-dev] [all] [clients] [keystone] lack of retrying tokens leads to overall OpenStack fragility

2014-09-11 Thread Jamie Lennox
- Original Message - From: Sean Dague s...@dague.net To: openstack-dev@lists.openstack.org Sent: Thursday, 11 September, 2014 9:44:43 PM Subject: Re: [openstack-dev] [all] [clients] [keystone] lack of retrying tokens leads to overall OpenStack fragility On 09/10/2014 08:46 PM

Re: [openstack-dev] [all] [clients] [keystone] lack of retrying tokens leads to overall OpenStack fragility

2014-09-11 Thread Jamie Lennox
, September 11, 2014 1:55:49 AM Subject: Re: [openstack-dev] [all] [clients] [keystone] lack of retrying tokens leads to overall OpenStack fragility On Wed, Sep 10, 2014 at 10:14:32AM -0400, Sean Dague wrote: Going through the untriaged Nova bugs, and there are a few on a similar

[openstack-dev] [all] [clients] [keystone] lack of retrying tokens leads to overall OpenStack fragility

2014-09-10 Thread Sean Dague
Going through the untriaged Nova bugs, and there are a few on a similar pattern: Nova operation in progress takes a while Crosses keystone token expiration time Timeout thrown Operation fails Terrible 500 error sent back to user It seems like we should have a standard pattern that on token

Re: [openstack-dev] [all] [clients] [keystone] lack of retrying tokens leads to overall OpenStack fragility

2014-09-10 Thread Endre Karlson
I think at least clients supporting keystone sessions that are configured to use the auth.Password mech supports this since re-auth is done by the session rather then the service client itself. 2014-09-10 16:14 GMT+02:00 Sean Dague s...@dague.net: Going through the untriaged Nova bugs, and

Re: [openstack-dev] [all] [clients] [keystone] lack of retrying tokens leads to overall OpenStack fragility

2014-09-10 Thread Sean Dague
Do we know which versions of the clients do that? -Sean On 09/10/2014 10:22 AM, Endre Karlson wrote: I think at least clients supporting keystone sessions that are configured to use the auth.Password mech supports this since re-auth is done by the session rather then the service

Re: [openstack-dev] [all] [clients] [keystone] lack of retrying tokens leads to overall OpenStack fragility

2014-09-10 Thread Steven Hardy
On Wed, Sep 10, 2014 at 10:14:32AM -0400, Sean Dague wrote: Going through the untriaged Nova bugs, and there are a few on a similar pattern: Nova operation in progress takes a while Crosses keystone token expiration time Timeout thrown Operation fails Terrible 500 error sent back to

Re: [openstack-dev] [all] [clients] [keystone] lack of retrying tokens leads to overall OpenStack fragility

2014-09-10 Thread Jamie Lennox
- Original Message - From: Steven Hardy sha...@redhat.com To: OpenStack Development Mailing List (not for usage questions) openstack-dev@lists.openstack.org Sent: Thursday, September 11, 2014 1:55:49 AM Subject: Re: [openstack-dev] [all] [clients] [keystone] lack of retrying

Re: [openstack-dev] [all] [clients] [keystone] lack of retrying tokens leads to overall OpenStack fragility

2014-09-10 Thread Angus Lees
On Wed, 10 Sep 2014 10:14:32 AM Sean Dague wrote: Going through the untriaged Nova bugs, and there are a few on a similar pattern: Nova operation in progress takes a while Crosses keystone token expiration time Timeout thrown Operation fails Terrible 500 error sent back to user It