Re: Re: List Files

[Thad Guidry]

It should be currently possible to use the ExecuteScript processor (with
Groovy language) and set those needed user variables like Paths,etc using
whatever System environment variables you wish to use and retrieve later.

In Groovy scripts you can do this:

System.setProperty("socksProxyPort","2003")
System.setProperty("socksProxyHost","localhost")
System.setProperty("ourPathToHell","DonaldTrump")

Thad
+ThadGuidry 

Re: Re: List Files

[Charlie Frasure]

Thanks Matthew and Joe,
I must have missed the automatic environment variable mapping in expression
language.  I will plan on utilizing this method until the Variable Registry
is available.


On Fri, Mar 4, 2016 at 1:57 PM, Joe Witt <joe.w...@gmail.com> wrote:

> Charlie,
>
> Yeah you're right that is a viable option now too.  And we support
> environment variables from the system too.  So for things that support
> expression language that helps.
>
> The variable registry obviously will give us a nice step forward.
>
> Thanks
> Joe
>
> On Fri, Mar 4, 2016 at 1:55 PM, Charlie Frasure
> <charliefras...@gmail.com> wrote:
> > Thanks Joe,
> >
> > That option is certainly more comprehensive than what I was looking to
> do.
> > An UpdateAttribute at the beginning of a flow (or even before a process
> > group) could establish the variables I need.  One case that either
> > UpdateAttribute or the Variable Registry should consider is being able to
> > mark sensitive properties so that things like passwords can remain
> secure.
> >
> >
> > On Fri, Mar 4, 2016 at 1:24 PM, Joe Witt <joe.w...@gmail.com> wrote:
> >>
> >> No.  But we need to and plan to
> >> https://cwiki.apache.org/confluence/display/NIFI/Variable+Registry
> >>
> >> On Fri, Mar 4, 2016 at 1:22 PM, Uwe Geercken <uwe.geerc...@web.de>
> wrote:
> >> > Matthew,
> >> >
> >> > does NiFi itself allow to define such things as constants? The idea
> >> > would be
> >> > to re-use these constants (e.g. a path) in different processors.
> >> >
> >> > Rgds,
> >> >
> >> > Uwe
> >> >
> >> > Gesendet: Freitag, 04. März 2016 um 19:04 Uhr
> >> > Von: "Matthew Clarke" <matt.clarke@gmail.com>
> >> > An: users@nifi.apache.org, "Joe Percivall" <joeperciv...@yahoo.com>
> >> > Betreff: Re: List Files
> >> >
> >> > The expression language can also be used to return values from both
> JVM
> >> > properties set in the NiFi bootstrap.conf and system environment
> >> > variables
> >> > for the user running NiFi.
> >> >
> >> > On Mar 4, 2016 11:00 AM, "Joe Percivall" <joeperciv...@yahoo.com>
> wrote:
> >> >>
> >> >> Hello,
> >> >>
> >> >> ListFile is a source processor so this behavior is expected. It
> >> >> supports
> >> >> expression language so that it can be configured to utilize certains
> >> >> methods. For example, some people may want to get a list of files
> from
> >> >> a
> >> >> rotating directory that gets created every hour. To do that they
> would
> >> >> need
> >> >> to use the date functions included in Expression Language.
> >> >>
> >> >> Does forbidding input hinder a specific use-case you have?
> >> >>
> >> >> Joe
> >> >>
> >> >> - - - - - -
> >> >> Joseph Percivall
> >> >> linkedin.com/in/Percivall
> >> >> e: joeperciv...@yahoo.com
> >> >>
> >> >>
> >> >>
> >> >> On Friday, March 4, 2016 10:38 AM, Charlie Frasure
> >> >> <charliefras...@gmail.com> wrote:
> >> >>
> >> >>
> >> >>
> >> >> I'm using the 0.5.1 build and having what I think is odd behavior for
> >> >> ListFile.  The processor supports expression language in the Input
> >> >> Directory
> >> >> property, however I can't figure out how to configure an attribute as
> >> >> input.
> >> >>
> >> >> I tried using UpdateAttribute prior to ListFile, but ListFile doesn't
> >> >> seem
> >> >> to allow incoming connections.  I also tried creating an attribute
> >> >> within
> >> >> ListFile, but it doesn't seem to be available for referencing within
> >> >> the
> >> >> same processor.
> >> >>
> >> >> Is this expected?  If so, what is the intended purpose of allowing
> >> >> expression language in this attribute?
> >
> >
>

Re: Re: List Files

[Joe Witt]

Charlie,

Yeah you're right that is a viable option now too.  And we support
environment variables from the system too.  So for things that support
expression language that helps.

The variable registry obviously will give us a nice step forward.

Thanks
Joe

On Fri, Mar 4, 2016 at 1:55 PM, Charlie Frasure
<charliefras...@gmail.com> wrote:
> Thanks Joe,
>
> That option is certainly more comprehensive than what I was looking to do.
> An UpdateAttribute at the beginning of a flow (or even before a process
> group) could establish the variables I need.  One case that either
> UpdateAttribute or the Variable Registry should consider is being able to
> mark sensitive properties so that things like passwords can remain secure.
>
>
> On Fri, Mar 4, 2016 at 1:24 PM, Joe Witt <joe.w...@gmail.com> wrote:
>>
>> No.  But we need to and plan to
>> https://cwiki.apache.org/confluence/display/NIFI/Variable+Registry
>>
>> On Fri, Mar 4, 2016 at 1:22 PM, Uwe Geercken <uwe.geerc...@web.de> wrote:
>> > Matthew,
>> >
>> > does NiFi itself allow to define such things as constants? The idea
>> > would be
>> > to re-use these constants (e.g. a path) in different processors.
>> >
>> > Rgds,
>> >
>> > Uwe
>> >
>> > Gesendet: Freitag, 04. März 2016 um 19:04 Uhr
>> > Von: "Matthew Clarke" <matt.clarke@gmail.com>
>> > An: users@nifi.apache.org, "Joe Percivall" <joeperciv...@yahoo.com>
>> > Betreff: Re: List Files
>> >
>> > The expression language can also be used to return values from both JVM
>> > properties set in the NiFi bootstrap.conf and system environment
>> > variables
>> > for the user running NiFi.
>> >
>> > On Mar 4, 2016 11:00 AM, "Joe Percivall" <joeperciv...@yahoo.com> wrote:
>> >>
>> >> Hello,
>> >>
>> >> ListFile is a source processor so this behavior is expected. It
>> >> supports
>> >> expression language so that it can be configured to utilize certains
>> >> methods. For example, some people may want to get a list of files from
>> >> a
>> >> rotating directory that gets created every hour. To do that they would
>> >> need
>> >> to use the date functions included in Expression Language.
>> >>
>> >> Does forbidding input hinder a specific use-case you have?
>> >>
>> >> Joe
>> >>
>> >> - - - - - -
>> >> Joseph Percivall
>> >> linkedin.com/in/Percivall
>> >> e: joeperciv...@yahoo.com
>> >>
>> >>
>> >>
>> >> On Friday, March 4, 2016 10:38 AM, Charlie Frasure
>> >> <charliefras...@gmail.com> wrote:
>> >>
>> >>
>> >>
>> >> I'm using the 0.5.1 build and having what I think is odd behavior for
>> >> ListFile.  The processor supports expression language in the Input
>> >> Directory
>> >> property, however I can't figure out how to configure an attribute as
>> >> input.
>> >>
>> >> I tried using UpdateAttribute prior to ListFile, but ListFile doesn't
>> >> seem
>> >> to allow incoming connections.  I also tried creating an attribute
>> >> within
>> >> ListFile, but it doesn't seem to be available for referencing within
>> >> the
>> >> same processor.
>> >>
>> >> Is this expected?  If so, what is the intended purpose of allowing
>> >> expression language in this attribute?
>
>

Re: Re: List Files

[Charlie Frasure]

Thanks Joe,

That option is certainly more comprehensive than what I was looking to do.
An UpdateAttribute at the beginning of a flow (or even before a process
group) could establish the variables I need.  One case that either
UpdateAttribute or the Variable Registry should consider is being able to
mark sensitive properties so that things like passwords can remain secure.


On Fri, Mar 4, 2016 at 1:24 PM, Joe Witt <joe.w...@gmail.com> wrote:

> No.  But we need to and plan to
> https://cwiki.apache.org/confluence/display/NIFI/Variable+Registry
>
> On Fri, Mar 4, 2016 at 1:22 PM, Uwe Geercken <uwe.geerc...@web.de> wrote:
> > Matthew,
> >
> > does NiFi itself allow to define such things as constants? The idea
> would be
> > to re-use these constants (e.g. a path) in different processors.
> >
> > Rgds,
> >
> > Uwe
> >
> > Gesendet: Freitag, 04. März 2016 um 19:04 Uhr
> > Von: "Matthew Clarke" <matt.clarke....@gmail.com>
> > An: users@nifi.apache.org, "Joe Percivall" <joeperciv...@yahoo.com>
> > Betreff: Re: List Files
> >
> > The expression language can also be used to return values from both JVM
> > properties set in the NiFi bootstrap.conf and system environment
> variables
> > for the user running NiFi.
> >
> > On Mar 4, 2016 11:00 AM, "Joe Percivall" <joeperciv...@yahoo.com> wrote:
> >>
> >> Hello,
> >>
> >> ListFile is a source processor so this behavior is expected. It supports
> >> expression language so that it can be configured to utilize certains
> >> methods. For example, some people may want to get a list of files from a
> >> rotating directory that gets created every hour. To do that they would
> need
> >> to use the date functions included in Expression Language.
> >>
> >> Does forbidding input hinder a specific use-case you have?
> >>
> >> Joe
> >>
> >> - - - - - -
> >> Joseph Percivall
> >> linkedin.com/in/Percivall
> >> e: joeperciv...@yahoo.com
> >>
> >>
> >>
> >> On Friday, March 4, 2016 10:38 AM, Charlie Frasure
> >> <charliefras...@gmail.com> wrote:
> >>
> >>
> >>
> >> I'm using the 0.5.1 build and having what I think is odd behavior for
> >> ListFile.  The processor supports expression language in the Input
> Directory
> >> property, however I can't figure out how to configure an attribute as
> input.
> >>
> >> I tried using UpdateAttribute prior to ListFile, but ListFile doesn't
> seem
> >> to allow incoming connections.  I also tried creating an attribute
> within
> >> ListFile, but it doesn't seem to be available for referencing within the
> >> same processor.
> >>
> >> Is this expected?  If so, what is the intended purpose of allowing
> >> expression language in this attribute?
>

Re: Re: List Files

[Joe Witt]

No.  But we need to and plan to
https://cwiki.apache.org/confluence/display/NIFI/Variable+Registry

On Fri, Mar 4, 2016 at 1:22 PM, Uwe Geercken <uwe.geerc...@web.de> wrote:
> Matthew,
>
> does NiFi itself allow to define such things as constants? The idea would be
> to re-use these constants (e.g. a path) in different processors.
>
> Rgds,
>
> Uwe
>
> Gesendet: Freitag, 04. März 2016 um 19:04 Uhr
> Von: "Matthew Clarke" <matt.clarke@gmail.com>
> An: users@nifi.apache.org, "Joe Percivall" <joeperciv...@yahoo.com>
> Betreff: Re: List Files
>
> The expression language can also be used to return values from both JVM
> properties set in the NiFi bootstrap.conf and system environment variables
> for the user running NiFi.
>
> On Mar 4, 2016 11:00 AM, "Joe Percivall" <joeperciv...@yahoo.com> wrote:
>>
>> Hello,
>>
>> ListFile is a source processor so this behavior is expected. It supports
>> expression language so that it can be configured to utilize certains
>> methods. For example, some people may want to get a list of files from a
>> rotating directory that gets created every hour. To do that they would need
>> to use the date functions included in Expression Language.
>>
>> Does forbidding input hinder a specific use-case you have?
>>
>> Joe
>>
>> - - - - - -
>> Joseph Percivall
>> linkedin.com/in/Percivall
>> e: joeperciv...@yahoo.com
>>
>>
>>
>> On Friday, March 4, 2016 10:38 AM, Charlie Frasure
>> <charliefras...@gmail.com> wrote:
>>
>>
>>
>> I'm using the 0.5.1 build and having what I think is odd behavior for
>> ListFile.  The processor supports expression language in the Input Directory
>> property, however I can't figure out how to configure an attribute as input.
>>
>> I tried using UpdateAttribute prior to ListFile, but ListFile doesn't seem
>> to allow incoming connections.  I also tried creating an attribute within
>> ListFile, but it doesn't seem to be available for referencing within the
>> same processor.
>>
>> Is this expected?  If so, what is the intended purpose of allowing
>> expression language in this attribute?

Aw: Re: List Files

[Uwe Geercken]

Matthew,

 

does NiFi itself allow to define such things as constants? The idea would be to re-use these constants (e.g. a path) in different processors.

 

Rgds,

 

Uwe

 

Gesendet: Freitag, 04. März 2016 um 19:04 Uhr
Von: "Matthew Clarke" <matt.clarke@gmail.com>
An: users@nifi.apache.org, "Joe Percivall" <joeperciv...@yahoo.com>
Betreff: Re: List Files


The _expression_ language can also be used to return values from both JVM properties set in the NiFi bootstrap.conf and system environment variables for the user running NiFi.

On Mar 4, 2016 11:00 AM, "Joe Percivall" <joeperciv...@yahoo.com> wrote:

Hello,

ListFile is a source processor so this behavior is expected. It supports _expression_ language so that it can be configured to utilize certains methods. For example, some people may want to get a list of files from a rotating directory that gets created every hour. To do that they would need to use the date functions included in _expression_ Language.

Does forbidding input hinder a specific use-case you have?

Joe

- - - - - -
Joseph Percivall
linkedin.com/in/Percivall
e: joeperciv...@yahoo.com



On Friday, March 4, 2016 10:38 AM, Charlie Frasure <charliefras...@gmail.com> wrote:



I'm using the 0.5.1 build and having what I think is odd behavior for ListFile.  The processor supports _expression_ language in the Input Directory property, however I can't figure out how to configure an attribute as input.

I tried using UpdateAttribute prior to ListFile, but ListFile doesn't seem to allow incoming connections.  I also tried creating an attribute within ListFile, but it doesn't seem to be available for referencing within the same processor.

Is this expected?  If so, what is the intended purpose of allowing _expression_ language in this attribute?

Re: List Files

[Matthew Clarke]

The expression language can also be used to return values from both JVM
properties set in the NiFi bootstrap.conf and system environment variables
for the user running NiFi.
On Mar 4, 2016 11:00 AM, "Joe Percivall"  wrote:

> Hello,
>
> ListFile is a source processor so this behavior is expected. It supports
> expression language so that it can be configured to utilize certains
> methods. For example, some people may want to get a list of files from a
> rotating directory that gets created every hour. To do that they would need
> to use the date functions included in Expression Language.
>
> Does forbidding input hinder a specific use-case you have?
>
> Joe
>
> - - - - - -
> Joseph Percivall
> linkedin.com/in/Percivall
> e: joeperciv...@yahoo.com
>
>
>
> On Friday, March 4, 2016 10:38 AM, Charlie Frasure <
> charliefras...@gmail.com> wrote:
>
>
>
> I'm using the 0.5.1 build and having what I think is odd behavior for
> ListFile.  The processor supports expression language in the Input
> Directory property, however I can't figure out how to configure an
> attribute as input.
>
> I tried using UpdateAttribute prior to ListFile, but ListFile doesn't seem
> to allow incoming connections.  I also tried creating an attribute within
> ListFile, but it doesn't seem to be available for referencing within the
> same processor.
>
> Is this expected?  If so, what is the intended purpose of allowing
> expression language in this attribute?
>

Re: List Files

[Joe Percivall]

Hello,

ListFile is a source processor so this behavior is expected. It supports 
expression language so that it can be configured to utilize certains methods. 
For example, some people may want to get a list of files from a rotating 
directory that gets created every hour. To do that they would need to use the 
date functions included in Expression Language. 

Does forbidding input hinder a specific use-case you have?
 
Joe

- - - - - - 
Joseph Percivall
linkedin.com/in/Percivall
e: joeperciv...@yahoo.com



On Friday, March 4, 2016 10:38 AM, Charlie Frasure  
wrote:



I'm using the 0.5.1 build and having what I think is odd behavior for ListFile. 
 The processor supports expression language in the Input Directory property, 
however I can't figure out how to configure an attribute as input.

I tried using UpdateAttribute prior to ListFile, but ListFile doesn't seem to 
allow incoming connections.  I also tried creating an attribute within 
ListFile, but it doesn't seem to be available for referencing within the same 
processor.

Is this expected?  If so, what is the intended purpose of allowing expression 
language in this attribute?

List Files

[Charlie Frasure]

I'm using the 0.5.1 build and having what I think is odd behavior for
ListFile.  The processor supports expression language in the Input
Directory property, however I can't figure out how to configure an
attribute as input.

I tried using UpdateAttribute prior to ListFile, but ListFile doesn't seem
to allow incoming connections.  I also tried creating an attribute within
ListFile, but it doesn't seem to be available for referencing within the
same processor.

Is this expected?  If so, what is the intended purpose of allowing
expression language in this attribute?