Re: [openstack-dev] 2 Minute tokens

2014-10-14 Thread Adam Young

On 10/13/2014 06:21 PM, Preston L. Bannister wrote:
Too-short token expiration times are one of my concerns, in my current 
exercise.


Working on a replacement for Nova backup. Basically creating backups 
jobs, writing the jobs into a queue, with a background worker that 
reads jobs from the queue. Tokens could expire while the jobs are in 
the queue (not too likely). Tokens could expire during the execution 
of a backup (while can be very long running, in some cases).


Had not run into mention of trusts before. Is the intent to cover 
this sort of use-case?
Keystone trusts are User to User delegations appropriate for long 
running tasks.  So, yeah, should work for these use cases.




(Pulled up what I could find on trusts. Need to chew on this a bit, 
as it is not immediately clear if this fits.)








On Wed, Oct 1, 2014 at 6:53 AM, Adam Young ayo...@redhat.com 
mailto:ayo...@redhat.com wrote:


On 10/01/2014 04:14 AM, Steven Hardy wrote:

On Tue, Sep 30, 2014 at 10:44:51AM -0400, Adam Young wrote:

What is keeping us from dropping the (scoped) token
duration to 5 minutes?


If we could keep their lifetime as short as network skew
lets us, we would
be able to:

Get rid of revocation checking.
Get rid of persisted tokens.

OK,  so that assumes we can move back to PKI tokens, but
we're working on
that.

What are the uses that require long lived tokens? Can they
be replaced with
a better mechanism for long term delegation (OAuth or
Keystone trusts) as
Heat has done?

FWIW I think you're misrepresenting Heat's usage of Trusts
here - 2 minute
tokens will break Heat just as much as any other service:

https://bugs.launchpad.net/heat/+bug/1306294


http://lists.openstack.org/pipermail/openstack-dev/2014-September/045585.html


Summary:

- Heat uses the request token to process requests (e.g stack
create), which
   may take an arbitrary amount of time (default timeout one
hour).

- Some use-cases demand timeout of more than one hour
(specifically big
   TripleO deployments), heat breaks in these situations atm,
folks are
   working around it by using long (several hour) token expiry
times.

- Trusts are only used of asynchronous signalling, e.g
Ceilometer signals
   Heat, we switch to a trust scoped token to process the
response to the
   alarm (e.g launch more instances on behalf of the user for
autoscaling)

My understanding, ref notes in that bug, is that using Trusts
while
servicing a request to effectively circumvent token expiry was
not legit
(or at least yukky and to be avoided).  If you think otherwise
then please
let me know, as that would be the simplest way to fix the bug
above (switch
to a trust token while doing the long-running create operation).

Using trusts to circumvent timeout is OK.  There are two issues in
tension here:

1.  A user needs to be able to maintain control of their own data.

2.  We want to limit the attack surface provided by tokens.

Since tokens are currently blanket access to the users data, there
really is no lessening of control by using trusts in a wider
context.  I'd argue that using trusts would actually reduce the
capability for abuse,if coupled with short lived tokens. With long
lived tokens, anyone can reuse the token. With a trust, only the
trustee would be able to create a new token.


Could we start by identifying the set of operations that are
currently timing out due to the one hour token duration and add an
optional trustid on those operations?




Trusts is not really ideal for this use-case anyway, as it
requires the
service to have knowledge of the roles to delegate (or that
the user
provides a pre-created trust), ref bug #1366133.  I suppose we
could just
delegate all the roles we find in the request scope and be
done with it,
given that bug has been wontfixed.

Steve

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
mailto:OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
mailto:OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev




___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org

Re: [openstack-dev] 2 Minute tokens

2014-10-13 Thread Preston L. Bannister
Too-short token expiration times are one of my concerns, in my current
exercise.

Working on a replacement for Nova backup. Basically creating backups jobs,
writing the jobs into a queue, with a background worker that reads jobs
from the queue. Tokens could expire while the jobs are in the queue (not
too likely). Tokens could expire during the execution of a backup (while
can be very long running, in some cases).

Had not run into mention of trusts before. Is the intent to cover this
sort of use-case?

(Pulled up what I could find on trusts. Need to chew on this a bit, as it
is not immediately clear if this fits.)








On Wed, Oct 1, 2014 at 6:53 AM, Adam Young ayo...@redhat.com wrote:

 On 10/01/2014 04:14 AM, Steven Hardy wrote:

 On Tue, Sep 30, 2014 at 10:44:51AM -0400, Adam Young wrote:

 What is keeping us from dropping the (scoped) token duration to 5
 minutes?


 If we could keep their lifetime as short as network skew lets us, we
 would
 be able to:

 Get rid of revocation checking.
 Get rid of persisted tokens.

 OK,  so that assumes we can move back to PKI tokens, but we're working on
 that.

 What are the uses that require long lived tokens?  Can they be replaced
 with
 a better mechanism for long term delegation (OAuth or Keystone trusts) as
 Heat has done?

 FWIW I think you're misrepresenting Heat's usage of Trusts here - 2 minute
 tokens will break Heat just as much as any other service:

 https://bugs.launchpad.net/heat/+bug/1306294

 http://lists.openstack.org/pipermail/openstack-dev/2014-
 September/045585.html


 Summary:

 - Heat uses the request token to process requests (e.g stack create),
 which
may take an arbitrary amount of time (default timeout one hour).

 - Some use-cases demand timeout of more than one hour (specifically big
TripleO deployments), heat breaks in these situations atm, folks are
working around it by using long (several hour) token expiry times.

 - Trusts are only used of asynchronous signalling, e.g Ceilometer signals
Heat, we switch to a trust scoped token to process the response to the
alarm (e.g launch more instances on behalf of the user for autoscaling)

 My understanding, ref notes in that bug, is that using Trusts while
 servicing a request to effectively circumvent token expiry was not legit
 (or at least yukky and to be avoided).  If you think otherwise then please
 let me know, as that would be the simplest way to fix the bug above
 (switch
 to a trust token while doing the long-running create operation).

 Using trusts to circumvent timeout is OK.  There are two issues in tension
 here:

 1.  A user needs to be able to maintain control of their own data.

 2.  We want to limit the attack surface provided by tokens.

 Since tokens are currently blanket access to the users data, there really
 is no lessening of control by using trusts in a wider context.  I'd argue
 that using trusts would actually reduce the capability for abuse,if coupled
 with short lived tokens. With long lived tokens, anyone can reuse the
 token.  With a trust, only the trustee would be able to create a new token.


 Could we start by identifying the set of operations that are currently
 timing out due to the one hour token duration and add an optional trustid
 on those operations?




 Trusts is not really ideal for this use-case anyway, as it requires the
 service to have knowledge of the roles to delegate (or that the user
 provides a pre-created trust), ref bug #1366133.  I suppose we could just
 delegate all the roles we find in the request scope and be done with it,
 given that bug has been wontfixed.

 Steve

 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] 2 Minute tokens

2014-10-01 Thread Chmouel Boudjnah
On Wed, Oct 1, 2014 at 3:47 AM, Adam Young ayo...@redhat.com wrote:

 1.  Identify the roles for the APIs that Cinder is going to be calling on
 swift based on Swifts policy.json


FYI: there is no Swifts policy.json in mainline code, there is one external
middleware available that provides it here
https://github.com/cloudwatt/swiftpolicy.git.

Chmouel
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] 2 Minute tokens

2014-10-01 Thread Steven Hardy
On Tue, Sep 30, 2014 at 10:44:51AM -0400, Adam Young wrote:
 What is keeping us from dropping the (scoped) token duration to 5 minutes?
 
 
 If we could keep their lifetime as short as network skew lets us, we would
 be able to:
 
 Get rid of revocation checking.
 Get rid of persisted tokens.
 
 OK,  so that assumes we can move back to PKI tokens, but we're working on
 that.
 
 What are the uses that require long lived tokens?  Can they be replaced with
 a better mechanism for long term delegation (OAuth or Keystone trusts) as
 Heat has done?

FWIW I think you're misrepresenting Heat's usage of Trusts here - 2 minute
tokens will break Heat just as much as any other service:

https://bugs.launchpad.net/heat/+bug/1306294

http://lists.openstack.org/pipermail/openstack-dev/2014-September/045585.html


Summary:

- Heat uses the request token to process requests (e.g stack create), which
  may take an arbitrary amount of time (default timeout one hour).

- Some use-cases demand timeout of more than one hour (specifically big
  TripleO deployments), heat breaks in these situations atm, folks are
  working around it by using long (several hour) token expiry times.

- Trusts are only used of asynchronous signalling, e.g Ceilometer signals
  Heat, we switch to a trust scoped token to process the response to the
  alarm (e.g launch more instances on behalf of the user for autoscaling)

My understanding, ref notes in that bug, is that using Trusts while
servicing a request to effectively circumvent token expiry was not legit
(or at least yukky and to be avoided).  If you think otherwise then please
let me know, as that would be the simplest way to fix the bug above (switch
to a trust token while doing the long-running create operation).

Trusts is not really ideal for this use-case anyway, as it requires the
service to have knowledge of the roles to delegate (or that the user
provides a pre-created trust), ref bug #1366133.  I suppose we could just
delegate all the roles we find in the request scope and be done with it,
given that bug has been wontfixed.

Steve

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] 2 Minute tokens

2014-10-01 Thread Adam Young

On 10/01/2014 04:14 AM, Steven Hardy wrote:

On Tue, Sep 30, 2014 at 10:44:51AM -0400, Adam Young wrote:

What is keeping us from dropping the (scoped) token duration to 5 minutes?


If we could keep their lifetime as short as network skew lets us, we would
be able to:

Get rid of revocation checking.
Get rid of persisted tokens.

OK,  so that assumes we can move back to PKI tokens, but we're working on
that.

What are the uses that require long lived tokens?  Can they be replaced with
a better mechanism for long term delegation (OAuth or Keystone trusts) as
Heat has done?

FWIW I think you're misrepresenting Heat's usage of Trusts here - 2 minute
tokens will break Heat just as much as any other service:

https://bugs.launchpad.net/heat/+bug/1306294

http://lists.openstack.org/pipermail/openstack-dev/2014-September/045585.html


Summary:

- Heat uses the request token to process requests (e.g stack create), which
   may take an arbitrary amount of time (default timeout one hour).

- Some use-cases demand timeout of more than one hour (specifically big
   TripleO deployments), heat breaks in these situations atm, folks are
   working around it by using long (several hour) token expiry times.

- Trusts are only used of asynchronous signalling, e.g Ceilometer signals
   Heat, we switch to a trust scoped token to process the response to the
   alarm (e.g launch more instances on behalf of the user for autoscaling)

My understanding, ref notes in that bug, is that using Trusts while
servicing a request to effectively circumvent token expiry was not legit
(or at least yukky and to be avoided).  If you think otherwise then please
let me know, as that would be the simplest way to fix the bug above (switch
to a trust token while doing the long-running create operation).
Using trusts to circumvent timeout is OK.  There are two issues in 
tension here:


1.  A user needs to be able to maintain control of their own data.

2.  We want to limit the attack surface provided by tokens.

Since tokens are currently blanket access to the users data, there 
really is no lessening of control by using trusts in a wider context.  
I'd argue that using trusts would actually reduce the capability for 
abuse,if coupled with short lived tokens. With long lived tokens, anyone 
can reuse the token.  With a trust, only the trustee would be able to 
create a new token.



Could we start by identifying the set of operations that are currently 
timing out due to the one hour token duration and add an optional 
trustid on those operations?





Trusts is not really ideal for this use-case anyway, as it requires the
service to have knowledge of the roles to delegate (or that the user
provides a pre-created trust), ref bug #1366133.  I suppose we could just
delegate all the roles we find in the request scope and be done with it,
given that bug has been wontfixed.

Steve

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] 2 Minute tokens

2014-09-30 Thread Louis Taylor
On Tue, Sep 30, 2014 at 10:44:51AM -0400, Adam Young wrote:
 What are the uses that require long lived tokens?

Glance has operations which can take a long time, such as uploading and
downloading large images.


signature.asc
Description: Digital signature
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] 2 Minute tokens

2014-09-30 Thread Jay Pipes

On 09/30/2014 10:44 AM, Adam Young wrote:

What is keeping us from dropping the (scoped) token duration to 5 minutes?

If we could keep their lifetime as short as network skew lets us, we
would be able to:

Get rid of revocation checking.
Get rid of persisted tokens.

OK, so that assumes we can move back to PKI tokens, but we're working
on that.

What are the uses that require long lived tokens?  Can they be replaced
with a better mechanism for long term delegation (OAuth or Keystone
trusts) as Heat has done?


I think you will find that most folks just don't know the intracacies of 
non-UUID tokens in Keystone. I think we'd be open to any options that 
are reliable, well-documented and don't produce 4K in each HTTP request.


Best,
-jay

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] 2 Minute tokens

2014-09-30 Thread Jay Pipes

On 09/30/2014 11:37 AM, Adam Young wrote:

On 09/30/2014 11:06 AM, Louis Taylor wrote:

On Tue, Sep 30, 2014 at 10:44:51AM -0400, Adam Young wrote:

What are the uses that require long lived tokens?

Glance has operations which can take a long time, such as uploading and
downloading large images.

Yes, but the token is only authenticated at the start of the operation.
Does anything need to happen afterwards?


Funny you mention it... :) We were just having this conversation on IRC 
about Nikesh's issues with some Tempest volume tests and a token 
expiration problem.


So, yes, a Glance upload operation makes a series of HTTP calls in the 
course of the upload:


 POST $registry/images -- Creates the queued image record
 ...  upload of chunked body of HTTP request to backend like Swift ..
 PUT $registry/images/IMAGE_ID -- update image status and checksum

So, what seems to be happening here is that the PUT call at the end of 
uploading the snapshot is using the same token that was created in the 
keystone client of the tempest test case during the test classes' 
setUpClass() method, and the test class ends up running for 1 hour, and 
by the time the PUT call is reached, the token has expired.


-jay

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] 2 Minute tokens

2014-09-30 Thread Sean Dague
On 09/30/2014 11:58 AM, Jay Pipes wrote:
 On 09/30/2014 11:37 AM, Adam Young wrote:
 On 09/30/2014 11:06 AM, Louis Taylor wrote:
 On Tue, Sep 30, 2014 at 10:44:51AM -0400, Adam Young wrote:
 What are the uses that require long lived tokens?
 Glance has operations which can take a long time, such as uploading and
 downloading large images.
 Yes, but the token is only authenticated at the start of the operation.
 Does anything need to happen afterwards?
 
 Funny you mention it... :) We were just having this conversation on IRC
 about Nikesh's issues with some Tempest volume tests and a token
 expiration problem.
 
 So, yes, a Glance upload operation makes a series of HTTP calls in the
 course of the upload:
 
  POST $registry/images -- Creates the queued image record
  ...  upload of chunked body of HTTP request to backend like Swift ..
  PUT $registry/images/IMAGE_ID -- update image status and checksum
 
 So, what seems to be happening here is that the PUT call at the end of
 uploading the snapshot is using the same token that was created in the
 keystone client of the tempest test case during the test classes'
 setUpClass() method, and the test class ends up running for 1 hour, and
 by the time the PUT call is reached, the token has expired.

Yes... and there is this whole unresolved dev thread on this -
http://lists.openstack.org/pipermail/openstack-dev/2014-September/045567.html

-Sean

-- 
Sean Dague
http://dague.net

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] 2 Minute tokens

2014-09-30 Thread Adam Young

On 09/30/2014 12:21 PM, Sean Dague wrote:

On 09/30/2014 11:58 AM, Jay Pipes wrote:

On 09/30/2014 11:37 AM, Adam Young wrote:

On 09/30/2014 11:06 AM, Louis Taylor wrote:

On Tue, Sep 30, 2014 at 10:44:51AM -0400, Adam Young wrote:

What are the uses that require long lived tokens?

Glance has operations which can take a long time, such as uploading and
downloading large images.

Yes, but the token is only authenticated at the start of the operation.
Does anything need to happen afterwards?

Funny you mention it... :) We were just having this conversation on IRC
about Nikesh's issues with some Tempest volume tests and a token
expiration problem.

So, yes, a Glance upload operation makes a series of HTTP calls in the
course of the upload:

  POST $registry/images -- Creates the queued image record
  ...  upload of chunked body of HTTP request to backend like Swift ..
  PUT $registry/images/IMAGE_ID -- update image status and checksum

So, what seems to be happening here is that the PUT call at the end of
uploading the snapshot is using the same token that was created in the
keystone client of the tempest test case during the test classes'
setUpClass() method, and the test class ends up running for 1 hour, and
by the time the PUT call is reached, the token has expired.

Yes... and there is this whole unresolved dev thread on this -
http://lists.openstack.org/pipermail/openstack-dev/2014-September/045567.html

-Sean



This is a test case, so the tempest test has enough information to 
request a new token, it just does not request it?






___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] 2 Minute tokens

2014-09-30 Thread Matthew Treinish
On Tue, Sep 30, 2014 at 04:23:37PM -0400, Adam Young wrote:
 On 09/30/2014 12:21 PM, Sean Dague wrote:
 On 09/30/2014 11:58 AM, Jay Pipes wrote:
 On 09/30/2014 11:37 AM, Adam Young wrote:
 On 09/30/2014 11:06 AM, Louis Taylor wrote:
 On Tue, Sep 30, 2014 at 10:44:51AM -0400, Adam Young wrote:
 What are the uses that require long lived tokens?
 Glance has operations which can take a long time, such as uploading and
 downloading large images.
 Yes, but the token is only authenticated at the start of the operation.
 Does anything need to happen afterwards?
 Funny you mention it... :) We were just having this conversation on IRC
 about Nikesh's issues with some Tempest volume tests and a token
 expiration problem.
 
 So, yes, a Glance upload operation makes a series of HTTP calls in the
 course of the upload:
 
   POST $registry/images -- Creates the queued image record
   ...  upload of chunked body of HTTP request to backend like Swift ..
   PUT $registry/images/IMAGE_ID -- update image status and checksum
 
 So, what seems to be happening here is that the PUT call at the end of
 uploading the snapshot is using the same token that was created in the
 keystone client of the tempest test case during the test classes'
 setUpClass() method, and the test class ends up running for 1 hour, and
 by the time the PUT call is reached, the token has expired.
 Yes... and there is this whole unresolved dev thread on this -
 http://lists.openstack.org/pipermail/openstack-dev/2014-September/045567.html
 
  -Sean
 
 
 This is a test case, so the tempest test has enough information to request a
 new token, it just does not request it?
 

No, I don't think that's the case. The tempest auth code handles token
expiration and prempts it. See:

http://git.openstack.org/cgit/openstack/tempest/tree/tempest/auth.py#n340

and 

http://git.openstack.org/cgit/openstack/tempest/tree/tempest/auth.py#n464

depending on which keystone api version is being used. These get called before
each outgoing http call from tempest is made and if it's expired it'll get a
new token. There could be a bug in that code, but I think it is probably
something else.

I think the issue here is probably caused by the token that glance is using is
expiring because the upload takes too long and it doesn't know how to handle
that. (unless I'm misreading Jay's comment) So things fail when it tries to use
the same token for another operation which is part of the same api request after
the upload is finished. This is the topic of that thread which Sean pointed out
above.

-Matt Treinish


pgpbcLO_fAH5b.pgp
Description: PGP signature
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] 2 Minute tokens

2014-09-30 Thread Adam Young

On 09/30/2014 12:21 PM, Sean Dague wrote:

On 09/30/2014 11:58 AM, Jay Pipes wrote:

On 09/30/2014 11:37 AM, Adam Young wrote:

On 09/30/2014 11:06 AM, Louis Taylor wrote:

On Tue, Sep 30, 2014 at 10:44:51AM -0400, Adam Young wrote:

What are the uses that require long lived tokens?

Glance has operations which can take a long time, such as uploading and
downloading large images.

Yes, but the token is only authenticated at the start of the operation.
Does anything need to happen afterwards?

Funny you mention it... :) We were just having this conversation on IRC
about Nikesh's issues with some Tempest volume tests and a token
expiration problem.

So, yes, a Glance upload operation makes a series of HTTP calls in the
course of the upload:

  POST $registry/images -- Creates the queued image record
  ...  upload of chunked body of HTTP request to backend like Swift ..
  PUT $registry/images/IMAGE_ID -- update image status and checksum

So, what seems to be happening here is that the PUT call at the end of
uploading the snapshot is using the same token that was created in the
keystone client of the tempest test case during the test classes'
setUpClass() method, and the test class ends up running for 1 hour, and
by the time the PUT call is reached, the token has expired.

Yes... and there is this whole unresolved dev thread on this -
http://lists.openstack.org/pipermail/openstack-dev/2014-September/045567.html

-Sean



This is comparable to the HEAT use case that Keystone Trusts were 
originally designed to solve.


If the glance client knows the roles required to perform those 
operations, it could create the trust up front, with the  Glance Service 
user as the trustee; the trustee execute the trust when it needs the token.


Are there other cases besides the glance one that require long lived tokens?

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] 2 Minute tokens

2014-09-30 Thread Andrew Laski


On 09/30/2014 05:33 PM, Adam Young wrote:

On 09/30/2014 12:21 PM, Sean Dague wrote:

On 09/30/2014 11:58 AM, Jay Pipes wrote:

On 09/30/2014 11:37 AM, Adam Young wrote:

On 09/30/2014 11:06 AM, Louis Taylor wrote:

On Tue, Sep 30, 2014 at 10:44:51AM -0400, Adam Young wrote:

What are the uses that require long lived tokens?
Glance has operations which can take a long time, such as 
uploading and

downloading large images.
Yes, but the token is only authenticated at the start of the 
operation.

Does anything need to happen afterwards?

Funny you mention it... :) We were just having this conversation on IRC
about Nikesh's issues with some Tempest volume tests and a token
expiration problem.

So, yes, a Glance upload operation makes a series of HTTP calls in the
course of the upload:

  POST $registry/images -- Creates the queued image record
  ...  upload of chunked body of HTTP request to backend like Swift ..
  PUT $registry/images/IMAGE_ID -- update image status and checksum

So, what seems to be happening here is that the PUT call at the end of
uploading the snapshot is using the same token that was created in the
keystone client of the tempest test case during the test classes'
setUpClass() method, and the test class ends up running for 1 hour, 
and

by the time the PUT call is reached, the token has expired.

Yes... and there is this whole unresolved dev thread on this -
http://lists.openstack.org/pipermail/openstack-dev/2014-September/045567.html 



-Sean



This is comparable to the HEAT use case that Keystone Trusts were 
originally designed to solve.


If the glance client knows the roles required to perform those 
operations, it could create the trust up front, with the  Glance 
Service user as the trustee; the trustee execute the trust when it 
needs the token.


Are there other cases besides the glance one that require long lived 
tokens?


Another potential case would be Nova interactions with Cinder when Nova 
is asked to create a volume on a users behalf in order to boot an 
instance from it.  The creation of the volume can take a long time and 
token expiration could be an issue in that process.





___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] 2 Minute tokens

2014-09-30 Thread Duncan Thomas
On Oct 1, 2014 12:37 AM, Adam Young ayo...@redhat.com wrote:

 On 09/30/2014 12:21 PM, Sean Dague wrote:

 On 09/30/2014 11:58 AM, Jay Pipes wrote:

 On 09/30/2014 11:37 AM, Adam Young wrote:

 On 09/30/2014 11:06 AM, Louis Taylor wrote:

 On Tue, Sep 30, 2014 at 10:44:51AM -0400, Adam Young wrote:

 What are the uses that require long lived tokens?

 Glance has operations which can take a long time, such as uploading
and
 downloading large images.

 Yes, but the token is only authenticated at the start of the operation.
 Does anything need to happen afterwards?

 Funny you mention it... :) We were just having this conversation on IRC
 about Nikesh's issues with some Tempest volume tests and a token
 expiration problem.

 So, yes, a Glance upload operation makes a series of HTTP calls in the
 course of the upload:

   POST $registry/images -- Creates the queued image record
   ...  upload of chunked body of HTTP request to backend like Swift ..
   PUT $registry/images/IMAGE_ID -- update image status and checksum

 So, what seems to be happening here is that the PUT call at the end of
 uploading the snapshot is using the same token that was created in the
 keystone client of the tempest test case during the test classes'
 setUpClass() method, and the test class ends up running for 1 hour, and
 by the time the PUT call is reached, the token has expired.

 Yes... and there is this whole unresolved dev thread on this -

http://lists.openstack.org/pipermail/openstack-dev/2014-September/045567.html

 -Sean


 This is comparable to the HEAT use case that Keystone Trusts were
originally designed to solve.

 If the glance client knows the roles required to perform those
operations, it could create the trust up front, with the  Glance Service
user as the trustee; the trustee execute the trust when it needs the token.

 Are there other cases besides the glance one that require long lived
tokens?

Cinder backups. These do many swift operations over a long period, often
hours. They should probably be converted to trusts, but I'd need some
guidance do so.
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] 2 Minute tokens

2014-09-30 Thread Adam Young


 This is comparable to the HEAT use case that Keystone Trusts were 
originally designed to solve.


 If the glance client knows the roles required to perform those 
operations, it could create the trust up front, with the Glance 
Service user as the trustee; the trustee execute the trust when it 
needs the token.


 Are there other cases besides the glance one that require long lived 
tokens?


Cinder backups. These do many swift operations over a long period, 
often hours. They should probably be converted to trusts, but I'd need 
some guidance do so.




1.  Identify the roles for the APIs that Cinder is going to be calling 
on swift based on Swifts policy.json
2.  The user Creates trust:  this could be done by the cinder client.  
The create trust call is here:


trustee user is the Cinder service user
trustor user is the actual human being (or reasonable facimile thereof) 
that needs this work done.



https://github.com/openstack/identity-api/blob/master/v3/src/markdown/identity-api-v3-os-trust-ext.md#create-trust-post-os-trusttrusts

impersonation should only be set if the API requires the tokens user_id 
to reflect the trustor.  I know that swift was one of the use cases for 
impersonation, but I don't think it is required.


Expiration date time is optional, but probably appropriate;  chose a 
value that is guaranteed to be long enough for the workflow, but that 
still limits the window for attack.   If this backup routinely takes 12 
hours, perhaps 24 hours is appropriate.


When the user requests the Cinder backup, they will have to include the 
trust ID in the request.  The cinder endpoint, prior to making a call to 
Swift, will get a token from Keystone, passing in the trust id  as per


https://github.com/openstack/identity-api/blob/master/v3/src/markdown/identity-api-v3-os-trust-ext.md#consuming-a-trust-with-post-authtokens




___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev