Re: [openstack-dev] masking X-Auth-Token in debug output - proposed consistency
On Fri, Sep 12, 2014 at 12:02 PM, Tripp, Travis S
wrote:
From Jamie Lennox:
We handle this in the keystoneclient Session object by just
printing
REDACTED or something similar.
The problem with using a SHA1 is that for backwards compatability
we
often use the SHA1 of a PKI token
as if it were a UUID token and so this is still sensitive data.
There
is working in keystone by morganfainberg
(which i think was merged) to add a new audit_it which will be
able to
identify a token across calls without
exposing any sensitive information. We will support this in
session
when available.
From Sean Dague
So the problem is that means we are currently leaking secrets and
making
the logs unreadable.
It seems like we should move forward with the {SHA1} ... and if
that is
still sensitive, address that later.
Not addressing it basically keeps the exposure and destroys
usability of
the code because there is so much garbage printed out.
I understand Sean's point about debugging. Right now the
glanceclient is just printing ***. So it isn't printing a lot of
excess and isn't leaking anything sensitive. The other usability
concern with the *** that Sean previously mentioned was having a
short usable string might be useful for debugging.
Morgan and Jamie, You think switching to SHA1 in actually adds a
potential security vulnerability to glanceclient that doesn't exist
now. If that is true, I think it would override the additional
debugging concern of using
SHA1 for now. Can you please confirm?
If only for consistency sake, I could switch to TOKEN_REDACTED
like the code sample Morgan sent. [1]
[1]
https://github.com/openstack/python-keystoneclient/blob/01cabf6bbbee
8b5340295f3be5e1fa7111387e7d/keystoneclient/session.py#L126-L131
As the person who proposed the change to print TOKEN_REDACTED, I'd be
happy to see it printed as {SHA1} instead. I only had it print
TOKEN_REDACTED because I was concerned that we were still logging
tokens and wanted to get something merged that didn't do that rather
than waiting for the perfect solution to come along.
Since we have configurable token hashing algorithm support in keystone
and auth_token middleware, it's possible that someone could lose their
sanity and decide to use sha1 as the hash algorithm (it defaults to
MD5, which some security standards say is inadequate), and now your
logs have usable token IDs instead of an unusable hash, ***,
TOKEN_REDACTED, or whatever. We could accept this as a risk, and we
could mitigate the risk some by changing keystone to reject sha1 as a hashing
algorithm.
- Brant
From: Morgan Fainberg [mailto:morgan.fainb...@gmail.com]
Ideally, I want to see the use of the audit_ids (in each token as of Juno)
as the end goal. If we can get there as fast as changing to the {SHA1},
I’d advocate for that. Brant nicely outlined why we didn’t go with SHA1
earlier on in the cycle.
I think we are close to solving this in a better way than using sha1,
but if we need a stop-gap we can go towards that for the short term (and
disallow sha1 as a hash for Keystone).
Thanks for all the input. Stuart McLaren pointed out the patches in Nova and
Swift clients where using the SHA1 was done in place of the token, so I've gone
ahead and posted up a glanceclient patch using the same approach.
https://review.openstack.org/#/c/121692/
Hopefully this at least makes these client work similarly for now and we can
switch everything to the audit ID soon.
Thanks,
Travis
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] masking X-Auth-Token in debug output - proposed consistency
On 09/11/2014 08:49 PM, Jamie Lennox wrote:
- Original Message -
From: Travis S Tripp travis.tr...@hp.com
To: OpenStack Development Mailing List (not for usage questions)
openstack-dev@lists.openstack.org
Sent: Friday, 12 September, 2014 10:30:53 AM
Subject: [openstack-dev] masking X-Auth-Token in debug output - proposed
consistency
Hi All,
I’m just helping with bug triage in Glance and we’ve got a bug to update how
tokens are redacted in the glanceclient [1]. It says to update to whatever
cross-project approach is agreed upon and references this thread:
http://lists.openstack.org/pipermail/openstack-dev/2014-June/037345.html
I just went through the thread and as best as I can tell there wasn’t a
conclusion in the ML. However, if we are going to do anything, IMO the
thread leans toward {SHA1}, with Morgan Fainberg dissenting.
However, he references a patch that was ultimately abandoned.
If there was a conclusion to this, please let me know so I can update and
work on closing this bug.
We handle this in the keystoneclient Session object by just printing REDACTED
or something similar. The problem with using a SHA1 is that for backwards
compatability we often use the SHA1 of a PKI token as if it were a UUID token
and so this is still sensitive data. There is working in keystone by
morganfainberg (which i think was merged) to add a new audit_it which will be
able to identify a token across calls without exposing any sensitive
information. We will support this in session when available.
So the problem is that means we are currently leaking secrets and making
the logs unreadable.
It seems like we should move forward with the {SHA1} ... and if that is
still sensitive, address that later. Not addressing it basically keeps
the exposure and destroys usability of the code because there is so much
garbage printed out.
The best i can say for standardization is that when glanceclient adopts the
session it will be handled the same way as all the other clients and
improvements can happen there without you having to worry about it.
Please don't let this be the perfect is the enemy of the good here.
Debugging OpenStack is hard. Not fixing this keeps it harder.
-Sean
--
Sean Dague
http://dague.net
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] masking X-Auth-Token in debug output - proposed consistency
From Jamie Lennox:
We handle this in the keystoneclient Session object by just printing
REDACTED or something similar.
The problem with using a SHA1 is that for backwards compatability we often
use the SHA1 of a PKI token
as if it were a UUID token and so this is still sensitive data. There is
working in keystone by morganfainberg
(which i think was merged) to add a new audit_it which will be able to
identify a token across calls without
exposing any sensitive information. We will support this in session when
available.
From Sean Dague
So the problem is that means we are currently leaking secrets and making the
logs unreadable.
It seems like we should move forward with the {SHA1} ... and if that is still
sensitive, address that later.
Not addressing it basically keeps the exposure and destroys usability of the
code because there is so much garbage printed out.
I understand Sean's point about debugging. Right now the glanceclient is just
printing ***. So it isn't printing a lot of excess and isn't leaking anything
sensitive. The other usability concern with the *** that Sean previously
mentioned was having a short usable string might be useful for debugging.
Morgan and Jamie, You think switching to SHA1 in actually adds a potential
security vulnerability to glanceclient that doesn't exist now. If that is true,
I think it would override the additional debugging concern of using SHA1 for
now. Can you please confirm?
If only for consistency sake, I could switch to TOKEN_REDACTED like the code
sample Morgan sent. [1]
[1]
https://github.com/openstack/python-keystoneclient/blob/01cabf6bbbee8b5340295f3be5e1fa7111387e7d/keystoneclient/session.py#L126-L131
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] masking X-Auth-Token in debug output - proposed consistency
On Fri, Sep 12, 2014 at 12:02 PM, Tripp, Travis S travis.tr...@hp.com
wrote:
From Jamie Lennox:
We handle this in the keystoneclient Session object by just printing
REDACTED or something similar.
The problem with using a SHA1 is that for backwards compatability we
often use the SHA1 of a PKI token
as if it were a UUID token and so this is still sensitive data. There
is working in keystone by morganfainberg
(which i think was merged) to add a new audit_it which will be able to
identify a token across calls without
exposing any sensitive information. We will support this in session
when available.
From Sean Dague
So the problem is that means we are currently leaking secrets and making
the logs unreadable.
It seems like we should move forward with the {SHA1} ... and if that is
still sensitive, address that later.
Not addressing it basically keeps the exposure and destroys usability of
the code because there is so much garbage printed out.
I understand Sean's point about debugging. Right now the glanceclient is
just printing ***. So it isn't printing a lot of excess and isn't leaking
anything sensitive. The other usability concern with the *** that Sean
previously mentioned was having a short usable string might be useful for
debugging.
Morgan and Jamie, You think switching to SHA1 in actually adds a potential
security vulnerability to glanceclient that doesn't exist now. If that is
true, I think it would override the additional debugging concern of using
SHA1 for now. Can you please confirm?
If only for consistency sake, I could switch to TOKEN_REDACTED like the
code sample Morgan sent. [1]
[1]
https://github.com/openstack/python-keystoneclient/blob/01cabf6bbbee8b5340295f3be5e1fa7111387e7d/keystoneclient/session.py#L126-L131
As the person who proposed the change to print TOKEN_REDACTED, I'd be happy
to see it printed as {SHA1}token-sha1 instead. I only had it print
TOKEN_REDACTED because I was concerned that we were still logging tokens
and wanted to get something merged that didn't do that rather than waiting
for the perfect solution to come along.
Since we have configurable token hashing algorithm support in keystone and
auth_token middleware, it's possible that someone could lose their sanity
and decide to use sha1 as the hash algorithm (it defaults to MD5, which
some security standards say is inadequate), and now your logs have usable
token IDs instead of an unusable hash, ***, TOKEN_REDACTED, or whatever. We
could accept this as a risk, and we could mitigate the risk some by
changing keystone to reject sha1 as a hashing algorithm.
- Brant
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] masking X-Auth-Token in debug output - proposed consistency
-Original Message-
From: Brant Knudson b...@acm.org
Reply: OpenStack Development Mailing List (not for usage questions)
openstack-dev@lists.openstack.org
Date: September 12, 2014 at 14:32:20
To: OpenStack Development Mailing List (not for usage questions)
openstack-dev@lists.openstack.org
Subject: Re: [openstack-dev] masking X-Auth-Token in debug output - proposed
consistency
On Fri, Sep 12, 2014 at 12:02 PM, Tripp, Travis S
wrote:
From Jamie Lennox:
We handle this in the keystoneclient Session object by just printing
REDACTED or something similar.
The problem with using a SHA1 is that for backwards compatability we
often use the SHA1 of a PKI token
as if it were a UUID token and so this is still sensitive data. There
is working in keystone by morganfainberg
(which i think was merged) to add a new audit_it which will be able to
identify a token across calls without
exposing any sensitive information. We will support this in session
when available.
From Sean Dague
So the problem is that means we are currently leaking secrets and making
the logs unreadable.
It seems like we should move forward with the {SHA1} ... and if that is
still sensitive, address that later.
Not addressing it basically keeps the exposure and destroys usability of
the code because there is so much garbage printed out.
I understand Sean's point about debugging. Right now the glanceclient is
just printing ***. So it isn't printing a lot of excess and isn't leaking
anything sensitive. The other usability concern with the *** that Sean
previously mentioned was having a short usable string might be useful for
debugging.
Morgan and Jamie, You think switching to SHA1 in actually adds a potential
security vulnerability to glanceclient that doesn't exist now. If that is
true, I think it would override the additional debugging concern of using
SHA1 for now. Can you please confirm?
If only for consistency sake, I could switch to TOKEN_REDACTED like the
code sample Morgan sent. [1]
[1]
https://github.com/openstack/python-keystoneclient/blob/01cabf6bbbee8b5340295f3be5e1fa7111387e7d/keystoneclient/session.py#L126-L131
As the person who proposed the change to print TOKEN_REDACTED, I'd be happy
to see it printed as {SHA1} instead. I only had it print
TOKEN_REDACTED because I was concerned that we were still logging tokens
and wanted to get something merged that didn't do that rather than waiting
for the perfect solution to come along.
Since we have configurable token hashing algorithm support in keystone and
auth_token middleware, it's possible that someone could lose their sanity
and decide to use sha1 as the hash algorithm (it defaults to MD5, which
some security standards say is inadequate), and now your logs have usable
token IDs instead of an unusable hash, ***, TOKEN_REDACTED, or whatever. We
could accept this as a risk, and we could mitigate the risk some by
changing keystone to reject sha1 as a hashing algorithm.
- Brant
Ideally, I want to see the use of the audit_ids (in each token as of Juno) as
the end goal. If we can get there as fast as changing to the {SHA1}, I’d
advocate for that. Brant nicely outlined why we didn’t go with SHA1 earlier on
in the cycle.
I think we are close to solving this in a better way than using sha1, but if we
need a stop-gap we can go towards that for the short term (and disallow sha1 as
a hash for Keystone).
—Morgan
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] masking X-Auth-Token in debug output - proposed consistency
Hi Travis,
By and large we have addressed this in the Session code within Keystoneclient
via the function here (and other similar cases):
https://github.com/openstack/python-keystoneclient/blob/01cabf6bbbee8b5340295f3be5e1fa7111387e7d/keystoneclient/session.py#L126-L131
If/when Glanceclient is moved to consuming the session code, it should help
alleviate the issues with printing the Token ID’s in the logs themselves.
Along with the changes for the session code, all tokens issued from Keystone
(Juno and beyond) will also include audit_id fields that are safe to use in
logging (they are part of the token data). There are two elements to the
audit_ids field, the first (will always exist) and is the local token’s
audit_id (audit ids are randomly generated and should be considered as globally
unique as a UUID). The second element will exist if the token has ever been
part of a rescope (exchange of a token for another token of a different scope,
e.g. changing to a new project/tenant). The second audit_id is the audit_id of
the first token in the chain (unique for the entire chain of tokens).
I don’t believe we’re exposing the audit_ids yet to the services behind the
auth_token middleware nor using them for logging in cases such as the above
linked logging function. I would like to eventually see the audit_ids used
(where they exist) for logging cases like this.
I’m sure Jamie Lennox can chime in and provide a bit more insight as to the
status of converting Glanceclient to using session as I know he’s been working
on the client front in this regard. I hope that sometime within the K
development cycle timeline we will be converting the logging over to audit_ids
where possible (but that has not been 100% decided on).
Cheers,
Morgan
—
Morgan Fainberg
-Original Message-
From: Tripp, Travis S travis.tr...@hp.com
Reply: OpenStack Development Mailing List (not for usage questions)
openstack-dev@lists.openstack.org
Date: September 11, 2014 at 17:35:30
To: OpenStack Development Mailing List (not for usage questions)
openstack-dev@lists.openstack.org
Subject: [openstack-dev] masking X-Auth-Token in debug output - proposed
consistency
Hi All,
I'm just helping with bug triage in Glance and we've got a bug to update how
tokens are redacted
in the glanceclient [1]. It says to update to whatever cross-project approach
is agreed
upon and references this thread:
http://lists.openstack.org/pipermail/openstack-dev/2014-June/037345.html
I just went through the thread and as best as I can tell there wasn't a
conclusion in the
ML. However, if we are going to do anything, IMO the thread leans toward
{SHA1},
with Morgan Fainberg dissenting. However, he references a patch that was
ultimately
abandoned.
If there was a conclusion to this, please let me know so I can update and
work on closing
this bug.
[1] https://bugs.launchpad.net/python-glanceclient/+bug/1329301
Thanks,
Travis
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] masking X-Auth-Token in debug output - proposed consistency
- Original Message -
From: Travis S Tripp travis.tr...@hp.com
To: OpenStack Development Mailing List (not for usage questions)
openstack-dev@lists.openstack.org
Sent: Friday, 12 September, 2014 10:30:53 AM
Subject: [openstack-dev] masking X-Auth-Token in debug output - proposed
consistency
Hi All,
I’m just helping with bug triage in Glance and we’ve got a bug to update how
tokens are redacted in the glanceclient [1]. It says to update to whatever
cross-project approach is agreed upon and references this thread:
http://lists.openstack.org/pipermail/openstack-dev/2014-June/037345.html
I just went through the thread and as best as I can tell there wasn’t a
conclusion in the ML. However, if we are going to do anything, IMO the
thread leans toward {SHA1}, with Morgan Fainberg dissenting.
However, he references a patch that was ultimately abandoned.
If there was a conclusion to this, please let me know so I can update and
work on closing this bug.
We handle this in the keystoneclient Session object by just printing REDACTED
or something similar. The problem with using a SHA1 is that for backwards
compatability we often use the SHA1 of a PKI token as if it were a UUID token
and so this is still sensitive data. There is working in keystone by
morganfainberg (which i think was merged) to add a new audit_it which will be
able to identify a token across calls without exposing any sensitive
information. We will support this in session when available.
The best i can say for standardization is that when glanceclient adopts the
session it will be handled the same way as all the other clients and
improvements can happen there without you having to worry about it.
Jamie
[1] https://bugs.launchpad.net/python-glanceclient/+bug/1329301
Thanks,
Travis
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] masking X-Auth-Token in debug output - proposed consistency
If these tokens are variable length up to 4k, it will make the search space much to large to construct any kind of useful table. They become infeasible for A-z0-9 variable-length password sets above 10 chars if you include every permutation. Assuming the tokens are generated in a very predictable manner that exclude a ton of possibilities, we shouldn't have to worry about rainbow tables. -- Kevin Benton On Fri, Jun 13, 2014 at 12:52 AM, Robert Collins robe...@robertcollins.net wrote: On 12 June 2014 23:59, Sean Dague s...@dague.net wrote: The only thing it makes harder is you have to generate your own token to run the curl command. The rest is there. Because everyone is running our servers at debug levels, it means the clients are going to be running debug level as well (yay python logging!), so this is something I don't think people realized was a huge issue. Anyway I have sent a patch for swiftclient for this in : https://review.openstack.org/#/c/99632/1 Personally I don't think I like much that SHA1 and i'd rather use the first 16 bytes of the token (like we did in swift server) Using a well known hash means you can verify it was the right thing if you have access to the original data. Just taking the first 16 bytes doesn't give you that, so I think the hash provides slightly more debugability. Would it be possible to salt it? e.g. make a 128bit salt and use that. The same token used twice will log with the same salt, but you won't have the rainbow table weakness. The length of tokens isn't a particularly strong defense against rainbow tables AIUI: if folk realise we have tokens exposed, they will just use a botnet to build a table specifically targetting us. -Rob -- Robert Collins rbtcoll...@hp.com Distinguished Technologist HP Converged Cloud ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev -- Kevin Benton ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] masking X-Auth-Token in debug output - proposed consistency
Morgan Fainberg wrote:
I’ve been looking over the code for this and it turns out plain old SHA1
is a bad idea. We recently had a patch land in keystone client and
keystone to let us configure the hashing algorithm used for token
revocation list and the short-token ids.
I’ve updated my patch set to use ‘{OBSCURED}%(token)s’ instead of
specifying a specific obscuring algorithm. This means that if we ever
update the way we obscure the data in the future, we’re not lying about
what was done in the log. The proposed approach can be found
here: https://review.openstack.org/#/c/99432
Looks good!
--
Thierry Carrez (ttx)
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] masking X-Auth-Token in debug output - proposed consistency
On Wed, Jun 11, 2014 at 9:47 PM, Sean Dague s...@dague.net wrote: Actually swiftclient is one of the biggest offenders in the gate - http://logs.openstack.org/96/99396/1/check/check-tempest-dsvm-full/4501fc8/logs/screen-g-api.txt.gz#_2014-06-11_15_20_11_078 I'd be happy to fix that but that would make the --debug option innefective right? Is it addressed in a different way in other clients? Chmouel ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] masking X-Auth-Token in debug output - proposed consistency
On Thu, Jun 12, 2014 at 12:58 PM, Chmouel Boudjnah chmo...@enovance.com wrote: On Wed, Jun 11, 2014 at 9:47 PM, Sean Dague s...@dague.net wrote: Actually swiftclient is one of the biggest offenders in the gate - http://logs.openstack.org/96/99396/1/check/check-tempest-dsvm-full/4501fc8/logs/screen-g-api.txt.gz#_2014-06-11_15_20_11_078 I'd be happy to fix that but that would make the --debug option innefective right? Is it addressed in a different way in other clients? Anyway I have sent a patch for swiftclient for this in : https://review.openstack.org/#/c/99632/1 Personally I don't think I like much that SHA1 and i'd rather use the first 16 bytes of the token (like we did in swift server) Chmouel ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] masking X-Auth-Token in debug output - proposed consistency
On 06/12/2014 07:42 AM, Chmouel Boudjnah wrote: On Thu, Jun 12, 2014 at 12:58 PM, Chmouel Boudjnah chmo...@enovance.com mailto:chmo...@enovance.com wrote: On Wed, Jun 11, 2014 at 9:47 PM, Sean Dague s...@dague.net mailto:s...@dague.net wrote: Actually swiftclient is one of the biggest offenders in the gate - http://logs.openstack.org/96/99396/1/check/check-tempest-dsvm-full/4501fc8/logs/screen-g-api.txt.gz#_2014-06-11_15_20_11_078 I'd be happy to fix that but that would make the --debug option innefective right? Is it addressed in a different way in other clients? The only thing it makes harder is you have to generate your own token to run the curl command. The rest is there. Because everyone is running our servers at debug levels, it means the clients are going to be running debug level as well (yay python logging!), so this is something I don't think people realized was a huge issue. Anyway I have sent a patch for swiftclient for this in : https://review.openstack.org/#/c/99632/1 Personally I don't think I like much that SHA1 and i'd rather use the first 16 bytes of the token (like we did in swift server) Using a well known hash means you can verify it was the right thing if you have access to the original data. Just taking the first 16 bytes doesn't give you that, so I think the hash provides slightly more debugability. -Sean -- Sean Dague http://dague.net signature.asc Description: OpenPGP digital signature ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] masking X-Auth-Token in debug output - proposed consistency
On Thu, Jun 12, 2014 at 1:59 PM, Sean Dague s...@dague.net wrote: The only thing it makes harder is you have to generate your own token to run the curl command. The rest is there. Well I would have imagine that the curl command debug are here so people can easily copy and paste them and/or tweak them, but sure it would just make it a bit harder. Because everyone is running our servers at debug levels, it means the clients are going to be running debug level as well (yay python logging!), so this is something I don't think people realized was a huge issue. so maybe the issue is that those curl commands shows up in server log when they should only output when running swift/nova/etc/client --debug, right? Chmouel ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] masking X-Auth-Token in debug output - proposed consistency
I'm hoping we can just ACK this approach, and get folks to start moving patches through the clients to clean this all up. just an fyi, in pyCADF, we obfuscate tokens similar to how credit cards are handled: by capturing a percentage of leading and trailing characters and substituting the middle ie. 4724 8478. whatever we decide here, i'm all for having a consistent way of masking and minimising tokens in OpenStack. cheers, gordon chung openstack, ibm software standards___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] masking X-Auth-Token in debug output - proposed consistency
Sorry to interrupt this discussion. Sean, Since I'm working the neutron client code change, by looking at your code change to nova client, looks like only X-Auth-Token is taken care of in http_log_req. There is also password in header and token id in response. Any particular reason that they are not being taken care of? Thanks, Xu Han — Sent from Mailbox for iPhone On Fri, Jun 13, 2014 at 8:47 AM, Gordon Chung chu...@ca.ibm.com wrote: I'm hoping we can just ACK this approach, and get folks to start moving patches through the clients to clean this all up. just an fyi, in pyCADF, we obfuscate tokens similar to how credit cards are handled: by capturing a percentage of leading and trailing characters and substituting the middle ie. 4724 8478. whatever we decide here, i'm all for having a consistent way of masking and minimising tokens in OpenStack. cheers, gordon chung openstack, ibm software standards___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] masking X-Auth-Token in debug output - proposed consistency
This stems a bit further than just reduction in noise in the logs. Think of
this from a case of security (with centralized logging or lower privileged
users able to read log files). If we aren’t putting passwords into these log
files, we shouldn’t be putting tokens in. The major functional different
between a token and a password is that the token has a fixed life span. Barring
running over the TTL of the token, the token grants all rights and privileges
that user has (some exceptions, such as trusts), even allowing for a rescope of
token to another project/tenant. In this light, tokens
are only marginally less exposure than a password in a log file.
I firmly believe that we should avoid putting information that conveys
authorization (e.g. username/password or bearer token id) in the logs.
—
Morgan Fainberg
From: Sean Dague s...@dague.net
Reply: OpenStack Development Mailing List (not for usage questions)
openstack-dev@lists.openstack.org
Date: June 11, 2014 at 12:02:20
To: OpenStack Development Mailing List (not for usage questions)
openstack-dev@lists.openstack.org
Subject: [openstack-dev] masking X-Auth-Token in debug output - proposed
consistency
We've had a few reviews recently going around to mask out X-Auth-Token
from the python clients in the debug output. Currently there are a mix
of ways this is done.
In glanceclient (straight stricken)
X-Auth-Token: ***
The neutronclient proposal -
https://review.openstack.org/#/c/93866/9/neutronclient/client.py is to
use 'REDACTED'
There is a novaclient patch in the gate that uses SHA1(sha1oftoken) -
https://review.openstack.org/#/c/98443/
Morgan was working on keystone.session patch -
https://review.openstack.org/#/c/98443/
after some back and forth we landed on {SHA1}sha1oftoken because
that's actually LDAP standard for such things, and SHA1(...) looks too
much like a function. I think that should probably be our final solution
here.
Why SHA1?
While we want to get rid of the token from the logs, for both security
and size reasons (5 - 10% of the logs in a gate run by bytes are
actually keystone tokens), it's actually sometimes important to
understand that *the same* token was used between 2 requests, or that 2
different tokens were used. This is especially try with expiration times
defaulting to 1 hr, and the fact that sometimes we have tests take
longer than that (so we need to debug that we didn't rotate tokens when
we should have).
Because the keystone token is long (going north of 4k), and variable
data length, and with different site data, these values should not be
susceptible to a generic rainbow attack, so a single SHA1 seems
sufficient. If there are objections to that, we can field something else
there. It also has the advantage of being batteries included with all
our supported versions of python.
I'm hoping we can just ACK this approach, and get folks to start moving
patches through the clients to clean this all up.
If you have concerns, please bring them up now.
-Sean
--
Sean Dague
http://dague.net
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] masking X-Auth-Token in debug output - proposed consistency
For both the security and the log line length, Swift is by default just
displaying the first 16 bytes of the token.
--John
On Jun 11, 2014, at 12:39 PM, Morgan Fainberg morgan.fainb...@gmail.com wrote:
This stems a bit further than just reduction in noise in the logs. Think of
this from a case of security (with centralized logging or lower privileged
users able to read log files). If we aren’t putting passwords into these log
files, we shouldn’t be putting tokens in. The major functional different
between a token and a password is that the token has a fixed life span.
Barring running over the TTL of the token, the token grants all rights and
privileges that user has (some exceptions, such as trusts), even allowing for
a rescope of token to another project/tenant. In this light, tokens
are only marginally less exposure than a password in a log file.
I firmly believe that we should avoid putting information that conveys
authorization (e.g. username/password or bearer token id) in the logs.
—
Morgan Fainberg
From: Sean Dague s...@dague.net
Reply: OpenStack Development Mailing List (not for usage questions)
openstack-dev@lists.openstack.org
Date: June 11, 2014 at 12:02:20
To: OpenStack Development Mailing List (not for usage questions)
openstack-dev@lists.openstack.org
Subject: [openstack-dev] masking X-Auth-Token in debug output - proposed
consistency
We've had a few reviews recently going around to mask out X-Auth-Token
from the python clients in the debug output. Currently there are a mix
of ways this is done.
In glanceclient (straight stricken)
X-Auth-Token: ***
The neutronclient proposal -
https://review.openstack.org/#/c/93866/9/neutronclient/client.py is to
use 'REDACTED'
There is a novaclient patch in the gate that uses SHA1(sha1oftoken) -
https://review.openstack.org/#/c/98443/
Morgan was working on keystone.session patch -
https://review.openstack.org/#/c/98443/
after some back and forth we landed on {SHA1}sha1oftoken because
that's actually LDAP standard for such things, and SHA1(...) looks too
much like a function. I think that should probably be our final solution
here.
Why SHA1?
While we want to get rid of the token from the logs, for both security
and size reasons (5 - 10% of the logs in a gate run by bytes are
actually keystone tokens), it's actually sometimes important to
understand that *the same* token was used between 2 requests, or that 2
different tokens were used. This is especially try with expiration times
defaulting to 1 hr, and the fact that sometimes we have tests take
longer than that (so we need to debug that we didn't rotate tokens when
we should have).
Because the keystone token is long (going north of 4k), and variable
data length, and with different site data, these values should not be
susceptible to a generic rainbow attack, so a single SHA1 seems
sufficient. If there are objections to that, we can field something else
there. It also has the advantage of being batteries included with all
our supported versions of python.
I'm hoping we can just ACK this approach, and get folks to start moving
patches through the clients to clean this all up.
If you have concerns, please bring them up now.
-Sean
--
Sean Dague
http://dague.net
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
signature.asc
Description: Message signed with OpenPGP using GPGMail
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] masking X-Auth-Token in debug output - proposed consistency
Actually swiftclient is one of the biggest offenders in the gate -
http://logs.openstack.org/96/99396/1/check/check-tempest-dsvm-full/4501fc8/logs/screen-g-api.txt.gz#_2014-06-11_15_20_11_078
On 06/11/2014 03:44 PM, John Dickinson wrote:
For both the security and the log line length, Swift is by default just
displaying the first 16 bytes of the token.
--John
On Jun 11, 2014, at 12:39 PM, Morgan Fainberg morgan.fainb...@gmail.com
wrote:
This stems a bit further than just reduction in noise in the logs. Think of
this from a case of security (with centralized logging or lower privileged
users able to read log files). If we aren’t putting passwords into these log
files, we shouldn’t be putting tokens in. The major functional different
between a token and a password is that the token has a fixed life span.
Barring running over the TTL of the token, the token grants all rights and
privileges that user has (some exceptions, such as trusts), even allowing
for a rescope of token to another project/tenant. In this light, tokens
are only marginally less exposure than a password in a log file.
I firmly believe that we should avoid putting information that conveys
authorization (e.g. username/password or bearer token id) in the logs.
—
Morgan Fainberg
From: Sean Dague s...@dague.net
Reply: OpenStack Development Mailing List (not for usage questions)
openstack-dev@lists.openstack.org
Date: June 11, 2014 at 12:02:20
To: OpenStack Development Mailing List (not for usage questions)
openstack-dev@lists.openstack.org
Subject: [openstack-dev] masking X-Auth-Token in debug output - proposed
consistency
We've had a few reviews recently going around to mask out X-Auth-Token
from the python clients in the debug output. Currently there are a mix
of ways this is done.
In glanceclient (straight stricken)
X-Auth-Token: ***
The neutronclient proposal -
https://review.openstack.org/#/c/93866/9/neutronclient/client.py is to
use 'REDACTED'
There is a novaclient patch in the gate that uses SHA1(sha1oftoken) -
https://review.openstack.org/#/c/98443/
Morgan was working on keystone.session patch -
https://review.openstack.org/#/c/98443/
after some back and forth we landed on {SHA1}sha1oftoken because
that's actually LDAP standard for such things, and SHA1(...) looks too
much like a function. I think that should probably be our final solution
here.
Why SHA1?
While we want to get rid of the token from the logs, for both security
and size reasons (5 - 10% of the logs in a gate run by bytes are
actually keystone tokens), it's actually sometimes important to
understand that *the same* token was used between 2 requests, or that 2
different tokens were used. This is especially try with expiration times
defaulting to 1 hr, and the fact that sometimes we have tests take
longer than that (so we need to debug that we didn't rotate tokens when
we should have).
Because the keystone token is long (going north of 4k), and variable
data length, and with different site data, these values should not be
susceptible to a generic rainbow attack, so a single SHA1 seems
sufficient. If there are objections to that, we can field something else
there. It also has the advantage of being batteries included with all
our supported versions of python.
I'm hoping we can just ACK this approach, and get folks to start moving
patches through the clients to clean this all up.
If you have concerns, please bring them up now.
-Sean
--
Sean Dague
http://dague.net
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
--
Sean Dague
http://dague.net
signature.asc
Description: OpenPGP digital signature
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] masking X-Auth-Token in debug output - proposed consistency
On 06/11/2014 03:01 PM, Sean Dague wrote:
We've had a few reviews recently going around to mask out X-Auth-Token
from the python clients in the debug output. Currently there are a mix
of ways this is done.
In glanceclient (straight stricken)
X-Auth-Token: ***
The neutronclient proposal -
https://review.openstack.org/#/c/93866/9/neutronclient/client.py is to
use 'REDACTED'
There is a novaclient patch in the gate that uses SHA1(sha1oftoken) -
https://review.openstack.org/#/c/98443/
Morgan was working on keystone.session patch -
https://review.openstack.org/#/c/98443/
after some back and forth we landed on {SHA1}sha1oftoken because
that's actually LDAP standard for such things, and SHA1(...) looks too
much like a function. I think that should probably be our final solution
here.
Why SHA1?
While we want to get rid of the token from the logs, for both security
and size reasons (5 - 10% of the logs in a gate run by bytes are
actually keystone tokens), it's actually sometimes important to
understand that *the same* token was used between 2 requests, or that 2
different tokens were used. This is especially try with expiration times
defaulting to 1 hr, and the fact that sometimes we have tests take
longer than that (so we need to debug that we didn't rotate tokens when
we should have).
Because the keystone token is long (going north of 4k), and variable
data length, and with different site data, these values should not be
susceptible to a generic rainbow attack, so a single SHA1 seems
sufficient. If there are objections to that, we can field something else
there. It also has the advantage of being batteries included with all
our supported versions of python.
I'm hoping we can just ACK this approach, and get folks to start moving
patches through the clients to clean this all up.
If you have concerns, please bring them up now.
Sounds like an excellent plan, thx for the update, Sean.
Best,
-jay
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] masking X-Auth-Token in debug output - proposed consistency
I’ve been looking over the code for this and it turns out plain old SHA1 is a
bad idea. We recently had a patch land in keystone client and keystone to let
us configure the hashing algorithm used for token revocation list and the
short-token ids.
I’ve updated my patch set to use ‘{OBSCURED}%(token)s’ instead of specifying a
specific obscuring algorithm. This means that if we ever update the way we
obscure the data in the future, we’re not lying about what was done in the log.
The proposed approach can be found here: https://review.openstack.org/#/c/99432
Cheers,
Morgan
—
Morgan Fainberg
From: Jay Pipes jaypi...@gmail.com
Reply: OpenStack Development Mailing List (not for usage questions)
openstack-dev@lists.openstack.org
Date: June 11, 2014 at 12:49:35
To: openstack-dev@lists.openstack.org openstack-dev@lists.openstack.org
Subject: Re: [openstack-dev] masking X-Auth-Token in debug output - proposed
consistency
On 06/11/2014 03:01 PM, Sean Dague wrote:
We've had a few reviews recently going around to mask out X-Auth-Token
from the python clients in the debug output. Currently there are a mix
of ways this is done.
In glanceclient (straight stricken)
X-Auth-Token: ***
The neutronclient proposal -
https://review.openstack.org/#/c/93866/9/neutronclient/client.py is to
use 'REDACTED'
There is a novaclient patch in the gate that uses SHA1(sha1oftoken) -
https://review.openstack.org/#/c/98443/
Morgan was working on keystone.session patch -
https://review.openstack.org/#/c/98443/
after some back and forth we landed on {SHA1}sha1oftoken because
that's actually LDAP standard for such things, and SHA1(...) looks too
much like a function. I think that should probably be our final solution
here.
Why SHA1?
While we want to get rid of the token from the logs, for both security
and size reasons (5 - 10% of the logs in a gate run by bytes are
actually keystone tokens), it's actually sometimes important to
understand that *the same* token was used between 2 requests, or that 2
different tokens were used. This is especially try with expiration times
defaulting to 1 hr, and the fact that sometimes we have tests take
longer than that (so we need to debug that we didn't rotate tokens when
we should have).
Because the keystone token is long (going north of 4k), and variable
data length, and with different site data, these values should not be
susceptible to a generic rainbow attack, so a single SHA1 seems
sufficient. If there are objections to that, we can field something else
there. It also has the advantage of being batteries included with all
our supported versions of python.
I'm hoping we can just ACK this approach, and get folks to start moving
patches through the clients to clean this all up.
If you have concerns, please bring them up now.
Sounds like an excellent plan, thx for the update, Sean.
Best,
-jay
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
