Re: [openstack-dev] [keystone] [keystoneauth] Debug data isn't sanitized - bug 1638978

2017-10-03 Thread Jamie Lennox 
Another option, pass log=False which we currently do for all the auth
requests. This will prevent debug printing the body at all, so con, by
default you can't see that message, but it's there because I never wanted
to mess around with masking individual service's secrets like this.

On 29 Sep. 2017 11:49 pm, "Lance Bragstad"  wrote:

>
>
> On 09/27/2017 06:38 AM, Bhor, Dinesh wrote:
>
> Hi Team,
>
>
>
> There are four solutions to fix the below bug:
>
> https://bugs.launchpad.net/keystoneauth/+bug/1638978
>
>
>
> 1) Carry a copy of mask_password() method to keystoneauth from oslo_utils
> [1]:
>
> *Pros:*
>
> A. keystoneauth will use already tested and used version of mask_password.
>
>
>
> *Cons:*
>
> A. keystoneauth will have to keep the version of mask_password() method
> sync with oslo_utils version.
>
>  If there are any new "_SANITIZE_KEYS" added to oslo_utils
> mask_password then those should be added in keystoneauth mask_password also.
>
> B. Copying the "mask_password" will also require to copy its supporting
> code [2] which is huge.
>
>
>
>
> I'm having flashbacks of the oslo-incubator days...
>
>
>
> 2) Use Oslo.utils mask_password() method in keystoneauth:
>
> *Pros:*
>
> A) No synching issue as described in solution #1. keystoneauth will
> directly use mask_password() method from Oslo.utils.
>
>
>
> *Cons:*
>
> A) You will need oslo.utils library to use keystoneauth.
>
> Objection by community:
>
> - keystoneauth community don't want any dependency on any of OpenStack
> common oslo libraries.
>
> Please refer to the comment from Morgan: https://bugs.launchpad.net/
> keystoneauth/+bug/1700751/comments/3
>
>
>
>
>
> 3) Add a custom logging filter in oslo logger
>
> Please refer to POC sample here: http://paste.openstack.org/show/617093/
>
> OpenStack core services using any OpenStack individual python-*client (for
> e.g python-cinderclient used in nova service) will need to pass oslo_logger
> object during it’s
>
> initialization which will do the work of masking sensitive information.
>
> Note: In nova, oslo.logger object is not passed during cinder client
> initialization (https://github.com/openstack/nova/blob/master/nova/volume/
> cinder.py#L135-L141),
>
> In this case, sensitive information will not be masked as it isn’t using
> Oslo.logger.
>
>
>
> *Pros:*
>
> A) No changes required in oslo.logger or any OpenStack services if
> mask_password method is modified in oslo.utils.
>
>
>
> *Cons:*
>
> A) Every log message will be scanned for certain password fields degrading
> the performance.
>
> B) If consumer of keystoneauth doesn’t use oslo_logger, then the sensitive
> information will not be masked.
>
> C) Will need to make changes wherever applicable to the OpenStack core
> services to pass oslo.logger object during python-novaclient initialization.
>
>
>
>
>
> 4) Add mask_password formatter parameter in oslo_log:
>
> Add "mask_password" formatter to sanitize sensitive data and pass it as a
> keyword argument to the log statement.
>
> If the mask_password is set, then only the sensitive information will be
> masked at the time of logging.
>
> The log statement will look like below:
>
>
>
> logger.debug("'adminPass': 'Now you see me'"), mask_password=True)
>
>
>
> Please refer to the POC code here: http://paste.openstack.org/show/618019/
>
>
>
> *Pros:  *
>
> A) No changes required in oslo.logger or any OpenStack services if
> mask_password method is modified in oslo.utils.
>
>
>
> *Cons:*
>
> A) If consumer of keystoneauth doesn’t use oslo_logger, then the sensitive
> information will not be masked.
>
> B) If you forget to pass mask_password=True for logging messages where
> sensitive information is present, then those fields won't be masked with
> ***.
>
>  But this can be clearly documented as suggested by Morgan and Lance.
>
> C) This solution requires you to add a below check in keystoneauth to
> avoid from an exception being raised in case logger is pure python Logger
> as it
>
>   doesn’t accept mask_password keyword argument.
>
>
>
> if isinstance(logger, logging.Logger):
>
> logger.debug(' '.join(string_parts))
>
> else:
>
> logger.debug(' '.join(string_parts), mask_password=True)
>
>
>
> This check assumes that the logger instance will be oslo_log only if it is
> not of python default logging.Logger.
>
> Keystoneauth community is not ready to have any dependency on any oslo-*
> lib, so it seems this solution has low acceptance chances.
>
>
> Options 2, 3, and 4 all require dependencies on oslo in order to work,
> which is a non-starter according to Morgan's comment in the bug [0].
> Options 3 and 4 will require a refactor to get keystoneauth to use oslo.log
> (today it uses the logging module from Python's standard library).
>
> [0] https://bugs.launchpad.net/keystoneauth/+bug/1700751/comments/3
>
>
>
> Please let me know your opinions about the above four approaches. Which
> one should we adopt?
>
>
>
> [1]

Re: [openstack-dev] [keystone] [keystoneauth] Debug data isn't sanitized - bug 1638978

2017-09-29 Thread Lance Bragstad 


On 09/27/2017 06:38 AM, Bhor, Dinesh wrote:
>
> Hi Team,
>
>  
>
> There are four solutions to fix the below bug:
>
> https://bugs.launchpad.net/keystoneauth/+bug/1638978
>
>  
>
> 1) Carry a copy of mask_password() method to keystoneauth from
> oslo_utils [1]:
>
> *Pros:*
>
> A. keystoneauth will use already tested and used version of mask_password.
>
>    
>
> *Cons:*
>
> A. keystoneauth will have to keep the version of mask_password()
> method sync with oslo_utils version.
>
>  If there are any new "_SANITIZE_KEYS" added to oslo_utils
> mask_password then those should be added in keystoneauth mask_password
> also.
>
> B. Copying the "mask_password" will also require to copy its
> supporting code [2] which is huge.
>
>  
>

I'm having flashbacks of the oslo-incubator days...

>  
>
> 2) Use Oslo.utils mask_password() method in keystoneauth:
>
> *Pros:*
>
> A) No synching issue as described in solution #1. keystoneauth will
> directly use mask_password() method from Oslo.utils.
>
>    
>
> *Cons:*
>
> A) You will need oslo.utils library to use keystoneauth.
>
> Objection by community:
>
> - keystoneauth community don't want any dependency on any of OpenStack
> common oslo libraries.
>
> Please refer to the comment from Morgan:
> https://bugs.launchpad.net/keystoneauth/+bug/1700751/comments/3
>
>  
>
>  
>
> 3) Add a custom logging filter in oslo logger
>
> Please refer to POC sample here:
> http://paste.openstack.org/show/617093/
> 
>
> OpenStack core services using any OpenStack individual python-*client
> (for e.g python-cinderclient used in nova service) will need to pass
> oslo_logger object during it’s
>
> initialization which will do the work of masking sensitive information.
>
> Note: In nova, oslo.logger object is not passed during cinder client
> initialization
> (https://github.com/openstack/nova/blob/master/nova/volume/cinder.py#L135-L141),
>
>
> In this case, sensitive information will not be masked as it isn’t
> using Oslo.logger.
>
>    
>
> *Pros:*
>
> A) No changes required in oslo.logger or any OpenStack services if
> mask_password method is modified in oslo.utils.
>
>    
>
> *Cons:*
>
> A) Every log message will be scanned for certain password fields
> degrading the performance.
>
> B) If consumer of keystoneauth doesn’t use oslo_logger, then the
> sensitive information will not be masked.
>
> C) Will need to make changes wherever applicable to the OpenStack core
> services to pass oslo.logger object during python-novaclient
> initialization.
>
>  
>
>  
>
> 4) Add mask_password formatter parameter in oslo_log:
>
> Add "mask_password" formatter to sanitize sensitive data and pass it
> as a keyword argument to the log statement.
>
> If the mask_password is set, then only the sensitive information will
> be masked at the time of logging.
>
> The log statement will look like below:
>
>  
>
> logger.debug("'adminPass': 'Now you see me'"), mask_password=True)
>
>  
>
> Please refer to the POC code here:
> http://paste.openstack.org/show/618019/
> 
>
>    
>
> *Pros:  *
>
> A) No changes required in oslo.logger or any OpenStack services if
> mask_password method is modified in oslo.utils.
>
>  
>
> *Cons:*
>
> A) If consumer of keystoneauth doesn’t use oslo_logger, then the
> sensitive information will not be masked.
>
> B) If you forget to pass mask_password=True for logging messages where
> sensitive information is present, then those fields won't be masked
> with ***.
>
>  But this can be clearly documented as suggested by Morgan and Lance.
>
> C) This solution requires you to add a below check in keystoneauth to
> avoid from an exception being raised in case logger is pure python
> Logger as it
>
>   doesn’t accept mask_password keyword argument.
>
>  
>
> if isinstance(logger, logging.Logger):
>
>     logger.debug(' '.join(string_parts))
>
> else:
>
>     logger.debug(' '.join(string_parts), mask_password=True)
>
>    
>
> This check assumes that the logger instance will be oslo_log only if
> it is not of python default logging.Logger.
>
> Keystoneauth community is not ready to have any dependency on any
> oslo-* lib, so it seems this solution has low acceptance chances.
>

Options 2, 3, and 4 all require dependencies on oslo in order to work,
which is a non-starter according to Morgan's comment in the bug [0].
Options 3 and 4 will require a refactor to get keystoneauth to use
oslo.log (today it uses the logging module from Python's standard library).

[0] https://bugs.launchpad.net/keystoneauth/+bug/1700751/comments/3

>  
>
> Please let me know your opinions about the above four approaches.
> Which one should we adopt?
>
>  
>
> [1]
> https://github.com/openstack/oslo.utils/blob/master/oslo_utils/strutils.py#L248-L313
>
> [2]
>

[openstack-dev] [keystone] [keystoneauth] Debug data isn't sanitized - bug 1638978

2017-09-27 Thread Bhor, Dinesh 
Hi Team,

There are four solutions to fix the below bug:
https://bugs.launchpad.net/keystoneauth/+bug/1638978

1) Carry a copy of mask_password() method to keystoneauth from oslo_utils [1]:
Pros:
A. keystoneauth will use already tested and used version of mask_password.

Cons:
A. keystoneauth will have to keep the version of mask_password() method sync 
with oslo_utils version.
 If there are any new "_SANITIZE_KEYS" added to oslo_utils mask_password 
then those should be added in keystoneauth mask_password also.
B. Copying the "mask_password" will also require to copy its supporting code 
[2] which is huge.


2) Use Oslo.utils mask_password() method in keystoneauth:
Pros:
A) No synching issue as described in solution #1. keystoneauth will directly 
use mask_password() method from Oslo.utils.

Cons:
A) You will need oslo.utils library to use keystoneauth.
Objection by community:
- keystoneauth community don't want any dependency on any of OpenStack common 
oslo libraries.
Please refer to the comment from Morgan: 
https://bugs.launchpad.net/keystoneauth/+bug/1700751/comments/3


3) Add a custom logging filter in oslo logger
Please refer to POC sample here: http://paste.openstack.org/show/617093/
OpenStack core services using any OpenStack individual python-*client (for e.g 
python-cinderclient used in nova service) will need to pass oslo_logger object 
during it's
initialization which will do the work of masking sensitive information.
Note: In nova, oslo.logger object is not passed during cinder client 
initialization 
(https://github.com/openstack/nova/blob/master/nova/volume/cinder.py#L135-L141),
In this case, sensitive information will not be masked as it isn't using 
Oslo.logger.

Pros:
A) No changes required in oslo.logger or any OpenStack services if 
mask_password method is modified in oslo.utils.

Cons:
A) Every log message will be scanned for certain password fields degrading the 
performance.
B) If consumer of keystoneauth doesn't use oslo_logger, then the sensitive 
information will not be masked.
C) Will need to make changes wherever applicable to the OpenStack core services 
to pass oslo.logger object during python-novaclient initialization.


4) Add mask_password formatter parameter in oslo_log:
Add "mask_password" formatter to sanitize sensitive data and pass it as a 
keyword argument to the log statement.
If the mask_password is set, then only the sensitive information will be masked 
at the time of logging.
The log statement will look like below:

logger.debug("'adminPass': 'Now you see me'"), mask_password=True)

Please refer to the POC code here: http://paste.openstack.org/show/618019/

Pros:
A) No changes required in oslo.logger or any OpenStack services if 
mask_password method is modified in oslo.utils.

Cons:
A) If consumer of keystoneauth doesn't use oslo_logger, then the sensitive 
information will not be masked.
B) If you forget to pass mask_password=True for logging messages where 
sensitive information is present, then those fields won't be masked with ***.
 But this can be clearly documented as suggested by Morgan and Lance.
C) This solution requires you to add a below check in keystoneauth to avoid 
from an exception being raised in case logger is pure python Logger as it
  doesn't accept mask_password keyword argument.

if isinstance(logger, logging.Logger):
logger.debug(' '.join(string_parts))
else:
logger.debug(' '.join(string_parts), mask_password=True)

This check assumes that the logger instance will be oslo_log only if it is not 
of python default logging.Logger.
Keystoneauth community is not ready to have any dependency on any oslo-* lib, 
so it seems this solution has low acceptance chances.

Please let me know your opinions about the above four approaches. Which one 
should we adopt?

[1] 
https://github.com/openstack/oslo.utils/blob/master/oslo_utils/strutils.py#L248-L313
[2] 
https://github.com/openstack/oslo.utils/blob/6e04f882c4308ff64fa199d1b127ad225e0a30c4/oslo_utils/strutils.py#L56-L96

Thanks and Regards,
Dinesh Bhor | App. Software Dev. Cnslt.
dinesh.b...@nttdata.com | VOIP. 8833.8395I | 
nttdata.com/americas
NTT DATA, Inc.
Consulting | Digital | Managed Services | Industry Solutions
Learn more:
[Description: Description: 
cid:image005.jpg@01D193F0.F70B44C0]

[Description: Description: 
cid:image009.jpg@01D193F0.F70B44C0]

[Description: Description: 
cid:image010.jpg@01D193F0.F70B44C0]

[Description: Description: 
cid:image011.jpg@01D193F0.F70B44C0]



__
Disclaimer: This email and any attachments are sent in strictest confidence
for the sole use of the addressee and may contain legally privileged,
confidential, and proprietary data. If you are not the