Re: Store password for config safely?

[Ferenc Szabo]

Hi Helmut,

here is the documentation part of the PR:
https://github.com/szaboferee/flume/blob/fa13593baa06c9d770a21fc970110e0c9abf2ef8/flume-ng-doc/sphinx/FlumeUserGuide.rst#configuration-filters

it is basically a variable substitution where the value comes from an
external source.
the PR contains 3 implementations to have a few to start with:
- Environment variable ( there was already another method to use env vars,
however, I figured it would be nice to have it in the new format to be
consistent)
- External Process where you can call a command that returns the value
- Hadoop credential store API where you can configure credential providers
documented here:
https://hadoop.apache.org/docs/stable/hadoop-project-dist/hadoop-common/CredentialProviderAPI.html#Provider_Types

feel free to review it and give feedback if You like.

Best regards,
Ferenc

On Wed, Feb 14, 2018 at 10:45 AM, Wahrmann, Helmut 
wrote:

> That sounds good. Need to have a closer look tough, how it can be used.
>
> Best regards,
> Helmut
>
> -Original Message-
> From: Mike Percy [mailto:mpe...@apache.org]
> Sent: Mittwoch, 14. Februar 2018 00:32
> To: dev@flume.apache.org
> Subject: Re: Store password for config safely?
>
> I think Ferenc has been looking at something related to this, or perhaps
> is trying to get an existing patch merged (FLUME-2442 <
> https://issues.apache.org/jira/browse/FLUME-2442>, PR 197 <
> https://github.com/apache/flume/pull/197>). I haven't been following that
> work closely so I don't know if it's exactly what you're looking for, but
> maybe he can chime in here.
>
> Mike
>
> On Mon, Feb 12, 2018 at 1:16 AM, Wahrmann, Helmut  >
> wrote:
>
> > Hi,
> >
> > Do we have a way of storing a password safely, i.e. not in clear text?
> > When e.g. an Elasticsearch cluster is protected by X-Pack Security, I
> > need to specify a userid / password when connecting.
> > The userid / password could be specified in the config, but then the
> > password would be available in readable form.
> >
> > Do we have other sinks or sources, where we are dealing with passwords
> > and were a suitable method exists?
> >
> > best regards,
> >
> > Helmut
> >
>

RE: Store password for config safely?

[Wahrmann, Helmut]

That sounds good. Need to have a closer look tough, how it can be used.

Best regards,
Helmut

-Original Message-
From: Mike Percy [mailto:mpe...@apache.org] 
Sent: Mittwoch, 14. Februar 2018 00:32
To: dev@flume.apache.org
Subject: Re: Store password for config safely?

I think Ferenc has been looking at something related to this, or perhaps is 
trying to get an existing patch merged (FLUME-2442 
<https://issues.apache.org/jira/browse/FLUME-2442>, PR 197 
<https://github.com/apache/flume/pull/197>). I haven't been following that work 
closely so I don't know if it's exactly what you're looking for, but maybe he 
can chime in here.

Mike

On Mon, Feb 12, 2018 at 1:16 AM, Wahrmann, Helmut 
wrote:

> Hi,
>
> Do we have a way of storing a password safely, i.e. not in clear text?
> When e.g. an Elasticsearch cluster is protected by X-Pack Security, I 
> need to specify a userid / password when connecting.
> The userid / password could be specified in the config, but then the 
> password would be available in readable form.
>
> Do we have other sinks or sources, where we are dealing with passwords 
> and were a suitable method exists?
>
> best regards,
>
> Helmut
>

Re: Store password for config safely?

[Mike Percy]

I think Ferenc has been looking at something related to this, or perhaps is
trying to get an existing patch merged (FLUME-2442
, PR 197
). I haven't been following that
work closely so I don't know if it's exactly what you're looking for, but
maybe he can chime in here.

Mike

On Mon, Feb 12, 2018 at 1:16 AM, Wahrmann, Helmut 
wrote:

> Hi,
>
> Do we have a way of storing a password safely, i.e. not in clear text?
> When e.g. an Elasticsearch cluster is protected by X-Pack Security, I need
> to specify a userid / password when connecting.
> The userid / password could be specified in the config, but then the
> password would be available in readable form.
>
> Do we have other sinks or sources, where we are dealing with passwords and
> were a suitable method exists?
>
> best regards,
>
> Helmut
>