Re: Odd inter-operation behavior when using clustered NiFi against secure MapR cluster

I had same issue with Secure MapR Cluster + NiFi cluster with embedded zk + PutHDFS setup. Switched back to non-cluster NiFi to avoid conflict between NiFi enabled zk and MapR. Waiting for stable solution to get back NiFi cluster. Thanks Sumo Sent from my iPad > On Oct 26, 2016, at

Re: 0.7.1 download items

JoeW, I made the link changes. Following the 1.0.0 link pattern I changed the source and binary file links but not the hash file links. Is there a commit job that pushes from the repository to the website? Regards, JoeS On Wed, Oct 26, 2016 at 3:45 PM, Joe Skora wrote: >

Re: [ANNOUNCE] New Apache NiFi Committer Bryan Rosander

Welcome aboard Byan! Your contributions are much appreciated by many NiFi users (TLS Toolkit I am looking at you..) and I am looking forward for your future contributions. Regarding the use of my name by Joe W, Rest assured that was not a copy and paste mistake... Joe is just trying to make

Re: [ANNOUNCE] New Apache NiFi Committer Bryan Rosander

Great to have you aboard, Bryan! On Wed, Oct 26, 2016 at 3:05 PM, Bryan Bende wrote: > Congrats Bryan! > > On Wed, Oct 26, 2016 at 1:20 PM, Andy LoPresto > wrote: > > > Congratulations Bryan. I can say the TLS toolkit has made my life much > > easier and

Re: 0.7.1 download items

No problem. Will do! On Wed, Oct 26, 2016 at 8:35 PM, Joe Witt wrote: > JoeS, > > Do you mind updating the download page for the 0.7.1 sources and > binaries to use the mirror based URLs rather than the specific dist > download site. See the 1.0 download links for

0.7.1 download items

JoeS, Do you mind updating the download page for the 0.7.1 sources and binaries to use the mirror based URLs rather than the specific dist download site. See the 1.0 download links for reference. It should end up looking like this

Re: [ANNOUNCE] New Apache NiFi Committer Bryan Rosander

Congrats Bryan! On Wed, Oct 26, 2016 at 1:20 PM, Andy LoPresto wrote: > Congratulations Bryan. I can say the TLS toolkit has made my life much > easier and I think that goes for a lot of people. Looking forward to more > great contributions. > > Andy LoPresto >

Re: [ANNOUNCE] New Apache NiFi Committer Bryan Rosander

Congrats Bryan! (and I definitely agree with Andy) 2016-10-26 19:20 GMT+02:00 Andy LoPresto : > Congratulations Bryan. I can say the TLS toolkit has made my life much > easier and I think that goes for a lot of people. Looking forward to more > great contributions. > > Andy

Re: [ANNOUNCE] New Apache NiFi Committer Bryan Rosander

Congrats Bryan! On Wed, Oct 26, 2016 at 1:15 PM, Joe Witt wrote: > On behalf of the Apache NiFI PMC, I am very pleased to announce that > Bryan Rosander has accepted the PMC's invitation to become a committer > on the Apache NiFi project. We greatly appreciate all of Andre's

Re: [ANNOUNCE] New Apache NiFi Committer Bryan Rosander

On behalf of the Apache NiFI PMC, I am very pleased to announce that Bryan Rosander has accepted the PMC's invitation to become a committer on the Apache NiFi project. We greatly appreciate all of Bryan's hard work and generous contributions and look forward to continued involvement in the

Re: [ANNOUNCE] New Apache NiFi Committer Bryan Rosander

Congrats Bryan! Glad to see you have Andre doing all your hard work ;p On Wed, Oct 26, 2016 at 1:16 PM, Oleg Zhurakousky < ozhurakou...@hortonworks.com> wrote: > Congrats Bryan! > Glad to have you! > > On Oct 26, 2016, at 1:15 PM, Joe Witt wrote: > > > > On behalf of the

Re: [ANNOUNCE] New Apache NiFi Committer Bryan Rosander

Congrats Bryan! Glad to have you! > On Oct 26, 2016, at 1:15 PM, Joe Witt wrote: > > On behalf of the Apache NiFI PMC, I am very pleased to announce that > Bryan Rosander has accepted the PMC's invitation to become a committer > on the Apache NiFi project. We greatly

[ANNOUNCE] New Apache NiFi Committer Bryan Rosander

On behalf of the Apache NiFI PMC, I am very pleased to announce that Bryan Rosander has accepted the PMC's invitation to become a committer on the Apache NiFi project. We greatly appreciate all of Andre's hard work and generous contributions and look forward to continued involvement in the

Re: PutS3Object submitting file many times

I agree with Joe, it sounds like the NiFi commit was failing after the file was sent. If the processor was administratively yielding, possibly due to the disk issues you mentioned, after completing the upload to S3, the file would be left in the queue and picked up again for processing later,

Re: Odd inter-operation behavior when using clustered NiFi against secure MapR cluster

Bryan, My apologies as the original email wasn't explicit about this: Your assumption is correct: My flow contains a processor (PutHDFS) with a core-site.xml configured. The file contains the property you refer to (as this is a cleaner way to force NiFi to connect to the secure MapR cluster).

Re: PutS3Object submitting file many times

Joe I have not looked at this code but can say that processors are or should be built such that in error cases we prefer duplication over loss. So a case like you describe could be a good example where session commit fails or something like it causing retries but in the mean time we keep sending

PutS3Object submitting file many times

Has anyone run into problems where PutS3Object submits the same file many (hundreds) of times? This turned out to be a problem for me since we have bucket versioning enabled, so we have many copies of the same file. I can't reproduce this behavior anymore in the normal flow of things, but I'm

Re: Odd inter-operation behavior when using clustered NiFi against secure MapR cluster

Meant to say the config instance somehow got "hadoop.security.authentication" set to "kerberos" On Wed, Oct 26, 2016 at 9:50 AM, Bryan Bende wrote: > Andre, > > This definitely seems weird that somehow using embedded ZooKeeper is > causing this. > > One thing I can say

Re: Replace Text for updating the flow file content

Arsalan, If I understand your use case correctly, you could use SplitJson with a JSONPath of "$.Records", this will emit one flow file for each entry in the array. Then you could use ConvertJSONtoAvro (it should infer the schema but you could either use InferAvroSchema or supply one if you know

Replace Text for updating the flow file content

Hi I am not very familiar with Regexs. The following is the json format data being delivered to the replace text processor. {"NrRecords":1,"FileRecords":-1,"Records":[{"ISN":250,"AA":"1122","AF":"S","AG":"F"}]} I want to replace this with [{"ISN":250,"AA":"1122","AF":"S","AG":"F"}]

Odd inter-operation behavior when using clustered NiFi against secure MapR cluster

Hi there, I've notice an odd behavior when using embedded Zookeeper on a NiFi cluster with MapR compatible processors: I noticed that every time I enable embedded zookeeper, NiFi's HDFS processors (e.g. PutHDFS) start complaining about Kerberos identities: 2016-10-26 20:07:22,376 ERROR