Re: [DISCUSS] 1.0 Beta Release

[Jennifer Barnabee]

+1

Sent from my iPhone

> On Aug 1, 2016, at 10:14 PM, Joe Percivall  
> wrote:
> 
> Hey Team,
> 
> The big ticket elements of 1.0 look to be in a pretty good state. Lots of 
> things left to tighten up, primarily docs and bug fixes, but, it seems wise 
> for a major release like this to first kick out a Beta[1]. The relevant 
> section of the Apache.org link:
> 
>Releases that are believed to be usable by testers and developers outside 
> the project, but perhaps not yet stable in terms of features or 
> functionality, are usually referred to as "beta" or "unstable". 
> 
> This will enable us to will open up to the broader base of users that 
> normally don't build from source to help test and give us a couple weeks of 
> feedback to work with before we do a formal 1.0 release vote. 
> 
> If no one has any objections, I will start an RC vote for it later this week.
> 
> [1] http://www.apache.org/dev/release.html#release-types
> 
> Joe
> - - - - - - 
> Joseph Percivall
> linkedin.com/in/Percivall
> e: joeperciv...@yahoo.com

Re: Wiki privileges request

[Andrew Lim]

Thanks!  I’m all set.


> On Aug 3, 2016, at 4:58 PM, Andy LoPresto  wrote:
> 
> Drew, 
> 
> Please try now. I believe I gave you the right permissions. 
> 
> Andy LoPresto
> alopre...@apache.org
> alopresto.apa...@gmail.com
> PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69
> 
>> On Aug 3, 2016, at 12:35 PM, Andrew Lim  wrote:
>> 
>> I’d like to contribute content to the NiFi Wiki, specifically the Migration 
>> Guide (https://cwiki.apache.org/confluence/display/NIFI/Migration+Guidance) 
>> for the 1.0.0 release.  Could someone give me the necessary privileges 
>> please?
>> 
>> My username is:  andrewlim.apache
>> 
>> Thanks,
>> 
>> Drew
> 

Re: Wiki privileges request

[Andy LoPresto]

Drew,

Please try now. I believe I gave you the right permissions.

Andy LoPresto
alopre...@apache.org
alopresto.apa...@gmail.com
PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69

> On Aug 3, 2016, at 12:35 PM, Andrew Lim  wrote:
> 
> I’d like to contribute content to the NiFi Wiki, specifically the Migration 
> Guide (https://cwiki.apache.org/confluence/display/NIFI/Migration+Guidance) 
> for the 1.0.0 release.  Could someone give me the necessary privileges please?
> 
> My username is:  andrewlim.apache
> 
> Thanks,
> 
> Drew



signature.asc
Description: Message signed with OpenPGP using GPGMail

TestListFile.testAttributesSet fails in AppVeyor

[Tarou, Kirk]

I submitted a pull request that failed in AppVeyor on 
TestListFile.testAttributesSet

Tests run: 12, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 4.347 sec <<< 
FAILURE! - in org.apache.nifi.processors.standard.TestListFile
testAttributesSet(org.apache.nifi.processors.standard.TestListFile)
 Time elapsed: 0.2 sec <<< FAILURE!
java.lang.AssertionError:
 null
at
 org.junit.Assert.fail(Assert.java:86)
at
 org.junit.Assert.assertTrue(Assert.java:41)
at
 org.junit.Assert.assertTrue(Assert.java:52)
at
 
org.apache.nifi.processors.standard.TestListFile.testAttributesSet(TestListFile.java:675)

That passes on my Mac, but AppVeyor ran the tests on a Windows machine:
Build started
git
 clone -q https://github.com/apache/nifi.git C:\projects\nifi
git
 fetch -q origin +refs/pull/774/merge:
git
 checkout -qf FETCH_HEAD
Running
 Install scripts
Add-Type
 -AssemblyName System.IO.Compression.FileSystem
if
 (!(Test-Path -Path "C:\maven" )) {
(new-object
 System.Net.WebClient).DownloadFile(
'http://www.us.apache.org/dist/maven/maven-3/3.1.1/binaries/apache-maven-3.1.1-bin.zip',
'C:\maven-bin.zip'
)
[System.IO.Compression.ZipFile]::ExtractToDirectory("C:\maven-bin.zip",
 "C:\maven")
}

SET
 PATH=C:\maven\apache-maven-3.1.1\bin;%JAVA_HOME%\bin;%PATH%
SET
 MAVEN_OPTS=-XX:MaxPermSize=2g -Xmx4g
SET
 JAVA_OPTS=-XX:MaxPermSize=2g -Xmx4g

I don’t have easy access to a Windows machine, so I can’t really troubleshoot 
this.
My guess is that the value of System.getProperty(“user.name”) may be null & 
would need to either be set in appveyor.yml or devise some other method to 
determine the username of the process.

- Kirk Tarou

Wiki privileges request

[Andrew Lim]

I’d like to contribute content to the NiFi Wiki, specifically the Migration 
Guide (https://cwiki.apache.org/confluence/display/NIFI/Migration+Guidance) for 
the 1.0.0 release.  Could someone give me the necessary privileges please?

My username is:  andrewlim.apache

Thanks,

Drew

Re: NiFi List/Fetch SFTP processors making 100 plus connections

[Bryan Bende]

Hi Kislay,

If you have a 3 node cluster and you setup ListSFTP -> FetchSFTP, it is
actually running this on all 3 nodes and likely fetching the same files
three times.
I'm not sure that would equate to 100s of connections, but it would
definitely be more connections then needed.

Typically the way you use the List + Fetch processors in a cluster is to
schedule the List processor to run on primary node only, and connect it to
a Remote Process Group.
The Remote Process Group points back to the same cluster and connects to an
Input Port port, this is essentially distributing all of the filenames
equally across your three nodes.
The Input Port then connects to FetchFile so all three nodes fetch in
parallel, but fetch different files.

Some resources that show how this work are here:
https://community.hortonworks.com/storage/attachments/1989-03-pulling-with-redistribution.png
http://www.slideshare.net/BryanBende/building-data-pipelines-for-solr-with-apache-nifi/31?src=clipshare

Let us know if this helps reduce the connections.

Thanks,

Bryan


On Wed, Aug 3, 2016 at 12:38 PM, Kislay Kumar 
wrote:

> Hi Bryan
>
> We have 3 node nifi cluster.
>
> And we are trying to connect to one sftp server.
>
> Flow is listsftp then fetchsftp.
>
> But when we check no. Of connections it shows 100+.
>
> Regards
> Kislay
>
> Sent from my iPhone
>
> On 03-Aug-2016, at 8:11 PM, Bryan Bende  wrote:
>
> Hi Rahul,
>
> A couple of questions to better understand the scenario...
>
> How many nodes in your NiFi cluster?
>
> Are you running ListSFPT on the primary node and using site-to-site to
> redistribute to the other nodes?
>
> Have you increased the concurrent tasks on any of the FetchSFPT processors?
>
> Thanks,
>
> Bryan
>
> On Wed, Aug 3, 2016 at 2:05 AM, Rahul Dahiya 
> wrote:
>
>> Hi Team,
>>
>>
>> We are using NiFi for downloading files from a remote machine. We are
>> using fetch and list sftp processors to download the files. We are able to
>> download the files from the remote machine but we see 100+ connections on
>> the remote machine from the NiFi cluster. We are using NiFi version -0.6.1 .
>>
>> Can you help us in finding out why so many connections are being made and
>> how can we limit the number of connections to the remote machine ? . Any
>> help is appreciated . Thanks in advance.
>>
>>
>> Regards ,
>>
>> Rahul
>>
>> 
>>
>>
>>
>>
>>
>>
>> NOTE: This message may contain information that is confidential,
>> proprietary, privileged or otherwise protected by law. The message is
>> intended solely for the named addressee. If received in error, please
>> destroy and notify the sender. Any use of this email is prohibited when
>> received in error. Impetus does not represent, warrant and/or guarantee,
>> that the integrity of this communication has been maintained nor that the
>> communication is free of errors, virus, interception or interference.
>>
>
>
> --
>
>
>
>
>
>
> NOTE: This message may contain information that is confidential,
> proprietary, privileged or otherwise protected by law. The message is
> intended solely for the named addressee. If received in error, please
> destroy and notify the sender. Any use of this email is prohibited when
> received in error. Impetus does not represent, warrant and/or guarantee,
> that the integrity of this communication has been maintained nor that the
> communication is free of errors, virus, interception or interference.
>

Re: [DISCUSS] 1.0 Beta Release

[Matt Burgess]

+1


> On Aug 1, 2016, at 10:14 PM, Joe Percivall  
> wrote:
> 
> Hey Team,
> 
> The big ticket elements of 1.0 look to be in a pretty good state. Lots of 
> things left to tighten up, primarily docs and bug fixes, but, it seems wise 
> for a major release like this to first kick out a Beta[1]. The relevant 
> section of the Apache.org link:
> 
>Releases that are believed to be usable by testers and developers outside 
> the project, but perhaps not yet stable in terms of features or 
> functionality, are usually referred to as "beta" or "unstable". 
> 
> This will enable us to will open up to the broader base of users that 
> normally don't build from source to help test and give us a couple weeks of 
> feedback to work with before we do a formal 1.0 release vote. 
> 
> If no one has any objections, I will start an RC vote for it later this week.
> 
> [1] http://www.apache.org/dev/release.html#release-types
> 
> Joe
> - - - - - - 
> Joseph Percivall
> linkedin.com/in/Percivall
> e: joeperciv...@yahoo.com

Re: [DISCUSS] 1.0 Beta Release

[Yolanda Davis]

+1

On Wed, Aug 3, 2016 at 11:01 AM, Jeff  wrote:

> +1
>
> On Tue, Aug 2, 2016 at 9:17 AM Joe Skora  wrote:
>
> > +1
> >
> > On Mon, Aug 1, 2016 at 10:49 PM, Tony Kurc  wrote:
> >
> > > seems reasonable to me
> > >
> > > On Mon, Aug 1, 2016 at 10:14 PM, Joe Percivall <
> > > joeperciv...@yahoo.com.invalid> wrote:
> > >
> > > > Hey Team,
> > > >
> > > > The big ticket elements of 1.0 look to be in a pretty good state.
> Lots
> > of
> > > > things left to tighten up, primarily docs and bug fixes, but, it
> seems
> > > wise
> > > > for a major release like this to first kick out a Beta[1]. The
> relevant
> > > > section of the Apache.org link:
> > > >
> > > > Releases that are believed to be usable by testers and developers
> > > > outside the project, but perhaps not yet stable in terms of
> > features
> > > or
> > > > functionality, are usually referred to as "beta" or "unstable".
> > > >
> > > > This will enable us to will open up to the broader base of users that
> > > > normally don't build from source to help test and give us a couple
> > weeks
> > > of
> > > > feedback to work with before we do a formal 1.0 release vote.
> > > >
> > > > If no one has any objections, I will start an RC vote for it later
> this
> > > > week.
> > > >
> > > > [1] http://www.apache.org/dev/release.html#release-types
> > > >
> > > > Joe
> > > > - - - - - -
> > > > Joseph Percivall
> > > > linkedin.com/in/Percivall
> > > > e: joeperciv...@yahoo.com
> > > >
> > >
> >
>



-- 
--
yolanda.m.da...@gmail.com
@YolandaMDavis

Re: NiFi List/Fetch SFTP processors making 100 plus connections

[Kislay Kumar]

Hi Bryan

We have 3 node nifi cluster.

And we are trying to connect to one sftp server.

Flow is listsftp then fetchsftp.

But when we check no. Of connections it shows 100+.

Regards
Kislay

Sent from my iPhone

On 03-Aug-2016, at 8:11 PM, Bryan Bende 
mailto:bbe...@gmail.com>> wrote:

Hi Rahul,

A couple of questions to better understand the scenario...

How many nodes in your NiFi cluster?

Are you running ListSFPT on the primary node and using site-to-site to 
redistribute to the other nodes?

Have you increased the concurrent tasks on any of the FetchSFPT processors?

Thanks,

Bryan

On Wed, Aug 3, 2016 at 2:05 AM, Rahul Dahiya 
mailto:rahul.dah...@impetus.co.in>> wrote:
Hi Team,


We are using NiFi for downloading files from a remote machine. We are using 
fetch and list sftp processors to download the files. We are able to download 
the files from the remote machine but we see 100+ connections on the remote 
machine from the NiFi cluster. We are using NiFi version -0.6.1 .

Can you help us in finding out why so many connections are being made and how 
can we limit the number of connections to the remote machine ? . Any help is 
appreciated . Thanks in advance.


Regards ,

Rahul








NOTE: This message may contain information that is confidential, proprietary, 
privileged or otherwise protected by law. The message is intended solely for 
the named addressee. If received in error, please destroy and notify the 
sender. Any use of this email is prohibited when received in error. Impetus 
does not represent, warrant and/or guarantee, that the integrity of this 
communication has been maintained nor that the communication is free of errors, 
virus, interception or interference.









NOTE: This message may contain information that is confidential, proprietary, 
privileged or otherwise protected by law. The message is intended solely for 
the named addressee. If received in error, please destroy and notify the 
sender. Any use of this email is prohibited when received in error. Impetus 
does not represent, warrant and/or guarantee, that the integrity of this 
communication has been maintained nor that the communication is free of errors, 
virus, interception or interference.

Re: escape functions in Expression Language

[Devin Fisher]

Done. Excited to hear any feedback when someone has time.

Devin

On Tue, Aug 2, 2016 at 9:16 PM, Bryan Bende  wrote:

> Hi Devin,
>
> Glad to hear you were able to implement the functionality you were looking
> for. Yes submitting a pull request against the master branch is
> probably the easiest way to submit a contribution. Right now the community
> is working towards closing out tickets for an upcoming 1.0 release, so it
> may take some time before PMCs/committers have time to review, but it is
> still good to get it posted and available.
>
> Thanks,
>
> Bryan
>
> On Tuesday, August 2, 2016, Devin Fisher <
> devin.fis...@perfectsearchcorp.com>
> wrote:
>
> > Forgot to mention that I made a JIRA ticket too.
> >
> > https://issues.apache.org/jira/browse/NIFI-2460
> >
> > Devin
> >
> > On Tue, Aug 2, 2016 at 10:18 AM, Devin Fisher <
> > devin.fis...@perfectsearchcorp.com > wrote:
> >
> > > I've added the functions following the instructions Bryan point me to
> and
> > > ready to at least have someone else look at it.
> > >
> > > Do I do a pull request to the master or some other branch?
> > >
> > > Devin
> > >
> > > On Fri, Jun 24, 2016 at 11:33 AM, Devin Fisher <
> > > devin.fis...@perfectsearchcorp.com > wrote:
> > >
> > >> Thanks for point out the README (did not find it in my searching).
> I'll
> > >> have a look at it.
> > >>
> > >> Devin
> > >>
> > >> On Fri, Jun 24, 2016 at 7:24 PM, Bryan Bende  > > wrote:
> > >>
> > >>> Hi Devin,
> > >>>
> > >>> I'm not aware of a built in escape function in NiFi's EL, although
> you
> > >>> might be able to do some of it with the replace functions.
> > >>>
> > >>> NiFi's expression language is in this module:
> > >>>
> > >>>
> >
> https://github.com/apache/nifi/tree/master/nifi-commons/nifi-expression-language
> > >>>
> > >>> The README has some good details about how to make changes. Let us
> know
> > >>> if
> > >>> it isn't clear.
> > >>>
> > >>> Thanks,
> > >>>
> > >>> Bryan
> > >>>
> > >>>
> > >>> On Fri, Jun 24, 2016 at 1:15 PM, Devin Fisher <
> > >>> devin.fis...@perfectsearchcorp.com > wrote:
> > >>>
> > >>> > I'm looking to create a bit of JSON using Nifi Expression Language
> so
> > >>> that
> > >>> > I can send it to a web service. But It does not look like Nifi
> > >>> Expression
> > >>> > Language have a function for escaping text to put into JSON.  I've
> > >>> looked
> > >>> > around in the wiki and I don't see any documentation now how to
> > extend
> > >>> > the Expression Language (not even sure if it is possible).
> > >>> >
> > >>> > Looks like nifi makes use of Apache commons-lang which has some
> > >>> functions
> > >>> > that would do the heavy lifting. [1]
> > >>> >
> > >>> > So if someone can help me understand the Expression Language end
> > >>> points I
> > >>> > could do the work fairly easily I think. And add functions for the
> > >>> other
> > >>> > formats supported by StringEscapeUtils (XML, csv, html).
> > >>> >
> > >>> > I think this could be useful.
> > >>> >
> > >>> > Devin
> > >>> >
> > >>> > [1]
> > >>> >
> > >>> >
> > >>>
> >
> https://commons.apache.org/proper/commons-lang/apidocs/org/apache/commons/lang3/StringEscapeUtils.html
> > >>> >
> > >>>
> > >>
> > >>
> > >
> >
>
>
> --
> Sent from Gmail Mobile
>

Re: [DISCUSS] 1.0 Beta Release

[Jeff]

+1

On Tue, Aug 2, 2016 at 9:17 AM Joe Skora  wrote:

> +1
>
> On Mon, Aug 1, 2016 at 10:49 PM, Tony Kurc  wrote:
>
> > seems reasonable to me
> >
> > On Mon, Aug 1, 2016 at 10:14 PM, Joe Percivall <
> > joeperciv...@yahoo.com.invalid> wrote:
> >
> > > Hey Team,
> > >
> > > The big ticket elements of 1.0 look to be in a pretty good state. Lots
> of
> > > things left to tighten up, primarily docs and bug fixes, but, it seems
> > wise
> > > for a major release like this to first kick out a Beta[1]. The relevant
> > > section of the Apache.org link:
> > >
> > > Releases that are believed to be usable by testers and developers
> > > outside the project, but perhaps not yet stable in terms of
> features
> > or
> > > functionality, are usually referred to as "beta" or "unstable".
> > >
> > > This will enable us to will open up to the broader base of users that
> > > normally don't build from source to help test and give us a couple
> weeks
> > of
> > > feedback to work with before we do a formal 1.0 release vote.
> > >
> > > If no one has any objections, I will start an RC vote for it later this
> > > week.
> > >
> > > [1] http://www.apache.org/dev/release.html#release-types
> > >
> > > Joe
> > > - - - - - -
> > > Joseph Percivall
> > > linkedin.com/in/Percivall
> > > e: joeperciv...@yahoo.com
> > >
> >
>

Re: StandardSSLContextService Not Being shown

[Bryan Bende]

Hello,

Adding the dev alias as well to see if anyone else knows the answer.

-Bryan

On Fri, Jul 29, 2016 at 10:37 AM, Mariama Barr-Dallas  wrote:

> Hello,
> I am attempting to add a Controller Service to a processor property via
> the rest API by changing the descriptors field and the properties field to
> the correct Id of the controller Service for the ProcessorEntity before
> uploading it;
> For some reason the first step is giving unexpected results: I cannot
> access any of the controller services via a GET rest api call until all of
> the other components (processors, process groups, connections) are loaded.
> Is there a certain order to when context services can be accessed and
> applied to a processor via the rest api?
>
>

Re: NiFi List/Fetch SFTP processors making 100 plus connections

[Bryan Bende]

Hi Rahul,

A couple of questions to better understand the scenario...

How many nodes in your NiFi cluster?

Are you running ListSFPT on the primary node and using site-to-site to
redistribute to the other nodes?

Have you increased the concurrent tasks on any of the FetchSFPT processors?

Thanks,

Bryan

On Wed, Aug 3, 2016 at 2:05 AM, Rahul Dahiya 
wrote:

> Hi Team,
>
>
> We are using NiFi for downloading files from a remote machine. We are
> using fetch and list sftp processors to download the files. We are able to
> download the files from the remote machine but we see 100+ connections on
> the remote machine from the NiFi cluster. We are using NiFi version -0.6.1 .
>
> Can you help us in finding out why so many connections are being made and
> how can we limit the number of connections to the remote machine ? . Any
> help is appreciated . Thanks in advance.
>
>
> Regards ,
>
> Rahul
>
> 
>
>
>
>
>
>
> NOTE: This message may contain information that is confidential,
> proprietary, privileged or otherwise protected by law. The message is
> intended solely for the named addressee. If received in error, please
> destroy and notify the sender. Any use of this email is prohibited when
> received in error. Impetus does not represent, warrant and/or guarantee,
> that the integrity of this communication has been maintained nor that the
> communication is free of errors, virus, interception or interference.
>

Re: UI translation into Simplified Chinese

[Andy LoPresto]

Hi Frank,

That would be a valuable effort. Thanks for volunteering. There is a current 
effort to translate the documentation to Spanish [NIFI-2116]. Translating the 
UI would probably require some refactoring to support internationalization, 
decoupling the messages to be displayed from the source code directly. 

Andy LoPresto
alopre...@apache.org
alopresto.apa...@gmail.com
PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69

> On Aug 2, 2016, at 22:59, Frank Ouyang  wrote:
> 
> Hello, 
> Is there anybody working on translating Nifi UI into Simplified Chinese?  If 
> not, I would like to assess the feasibility and apprecite on help with the 
> list of files that need to be translated. I’m based in Shanghai and have 
> extensive software globalization experience at IBM
> 
> Regards, 
> Frank
> 
> 

NiFi List/Fetch SFTP processors making 100 plus connections

[Rahul Dahiya]

Hi Team,


We are using NiFi for downloading files from a remote machine. We are using 
fetch and list sftp processors to download the files. We are able to download 
the files from the remote machine but we see 100+ connections on the remote 
machine from the NiFi cluster. We are using NiFi version -0.6.1 .

Can you help us in finding out why so many connections are being made and how 
can we limit the number of connections to the remote machine ? . Any help is 
appreciated . Thanks in advance.


Regards ,

Rahul








NOTE: This message may contain information that is confidential, proprietary, 
privileged or otherwise protected by law. The message is intended solely for 
the named addressee. If received in error, please destroy and notify the 
sender. Any use of this email is prohibited when received in error. Impetus 
does not represent, warrant and/or guarantee, that the integrity of this 
communication has been maintained nor that the communication is free of errors, 
virus, interception or interference.

UI translation into Simplified Chinese

[Frank Ouyang]

Hello, 
Is there anybody working on translating Nifi UI into Simplified Chinese?  If 
not, I would like to assess the feasibility and apprecite on help with the list 
of files that need to be translated. I’m based in Shanghai and have extensive 
software globalization experience at IBM

Regards, 
Frank