[
https://issues.apache.org/jira/browse/FLUME-2442?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14097673#comment-14097673
]
Roshan Naik edited comment on FLUME-2442 at 8/14/14 11:13 PM:
--------------------------------------------------------------
There is an initiative in Hadoop to create an API to address clear text
passwords that some projects seem to be adopting.
https://issues.apache.org/jira/browse/HADOOP-10607
https://issues.apache.org/jira/browse/HADOOP-10904
To summarize my understanding of it:
- Its an api for accessing passwords which are stored elsewhere
- It will have several "providers" which will determine where the password
actually gets stored
- A simple provider will basically put the password in a local/hdfs file .. the
api has an option to use a default "none" password to decrypt the local/hdfs
file. this functionality is very similar to what we do currently when we put
the keys in a local keystore.
- A more sophisticated "provider" will user kerberos keytab to acquire the
password from a remote "credential store service"
- The api interface will support a few different providers and each provider
may require some unique configuration settings.. but the API interface would
remain the same.
*Benefits of going with this API for flume:*
- Give flume users the option to use more sophisticated secure providers as
they become available.
- with kerberos based provider we completely move away from clear text passwords
*Concerns that I have:*
- Initially only the simple local store provider will be available. So it will
not offer any new level of security than the current local keystore based
approach we have in say file channel and jms source. A minor benefit is that it
has the option of using a default "none" password which means no explicit
cleartext password is required in config file (although it has hard code in
some java source file)
- Once the kerberos based provider becomes available, level of security does
not appear to be much stronger than the local keystore approach. Since, in both
case the security is governed by the file permission of keytab or local
keystore. so once the hacker gets access to the keytab or keystore file, the
security has been compromised. Nevertheless it will still be a solution that
does not use cleartext passwords (which is concern for some security sensitive
institutions like banks)
- *Hadoop Dependency* - The credential provider api is part of hadoop-common
and consequently it would require hadoop to be installed along side flume in
many cases (sources/sinks that use that feature)
was (Author: roshan_naik):
There is an initiative in Hadoop to create an API to address clear text
passwords that some projects seem to be adopting.
https://issues.apache.org/jira/browse/HADOOP-10607
https://issues.apache.org/jira/browse/HADOOP-10904
To summarize my understanding of it:
- Its an api for accessing passwords which are stored elsewhere
- It will have several "providers" which will determine where the password
actually gets stored
- A simple provider will basically put the password in a local/hdfs file .. the
api has an option to use a default "none" password to decrypt the local/hdfs
file. this functionality is very similar to what we do currently when we put
the keys in a local keystore.
- A more sophisticated "provider" will user kerberos keytab to acquire the
password from a remote "credential store service"
- The api interface will support a few different providers and each provider
may require some unique configuration settings.. but the API interface would
remain the same.
*Benefits of going with this API for flume:*
- Give flume users the option to use more sophisticated secure providers as
they become available.
- with kerberos based provider we completely move away from clear text passwords
*Concerns that I have:*
- Initially only the simple local store provider will be available. So it will
not offer any new level of security than the current local keystore based
approach we have in say file channel and jms source. A minor benefit is that it
has the option of using a default "none" password which means no explicit
cleartext password is required in config file (although it has hard code in
some java source file)
- Once the kerberos based provider becomes available, level of security does
not appear to be much stronger than the local keystore approach. Since, in both
case the security is governed by the file permission of keytab or local
keystore. so once the hacker gets access to the keytab or keystore file, the
security has been compromised. Nevertheless it will still be a solution that
does not use cleartext passwords (which is concern for some security sensitive
institutions like banks)
> Need an alternative to providing clear text passwords in flume config
> ---------------------------------------------------------------------
>
> Key: FLUME-2442
> URL: https://issues.apache.org/jira/browse/FLUME-2442
> Project: Flume
> Issue Type: Bug
> Components: Sinks+Sources
> Affects Versions: v1.5.0.1
> Reporter: Roshan Naik
> Assignee: Roshan Naik
> Labels: Security
>
> For some sources and sinks, currently, passwords to keystores/other are
> specified in clear text in the flume config file. Since flume config files
> are often easily accessible to a broader audience (like in source control for
> instance), the visibility of these passwords can be too much and risky for
> institutions where security is too critical (like banks)
> To help address this visibility issue it would be desirable to do the
> following two things :
> 1) Store the password in a separate file and provide the path of that
> password file in the flume config. this will enable the flume config to be
> shared with a wider audience and reduce risk. the password file will need to
> be very tightly guarded. Some components like file channel & JMS source
> already do this.
> 2) As an additional measure, obfuscate the password in the external password
> file. A simple command line tool can be used to generate the obfuscated
> password file. Flume source/sink configuration will read the password file
> and de-obfuscate the password before using it to access the keystore. This
> obfuscation step IMO is nice but unclear to me if it is essential.
> The following Sources and Sinks appear to use inline cleartext passwords:
> - Avro Source
> - Avro sink
> - HTTP(S) source
> JDBC channel also uses inline passwords but i am not aware of anybody who
> uses JDBC channel. So it may not be an issue.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
