Re: how to merge attributes?

2018-10-23 Thread Koji Kawamura
Hello, InvokeHttp creates a new FlowFile for "Result" relationship from the incoming FlowFile. That means, the FlowFile for "Result" carries all attributes copied that the incoming one has. You just need to connect the "Result" relationship to ReplaceTest. "Original" can be auto-terminated in

Re: NiFi Toolkit CLI issues with NiFi/Registry SSL handshake

2018-10-23 Thread ara m.
I tried this >>> You could also just use the keystore from NiFi since NiFi is already successfully talking to registry. I used the CLI from inside NiFi and provided the truststore as well as the keystore that NiFi uses, still same error message. /ERROR: Error executing command 'list-buckets'

how to merge attributes?

2018-10-23 Thread l vic
I have to create sql query from results of rest call (InvokeHttp ) and original request parameters... Any idea how i can merge attributes from both in single flowfile for passing it to "ReplaceText"? If I pass both "Original" and "Result" from "InvokeHttp" to "ReplaceText" I end up with one

Re: NiFi Toolkit CLI issues with NiFi/Registry SSL handshake

2018-10-23 Thread Bryan Bende
In order to perform any action against a secure registry or NiFi you will need to authenticate, just like if you went to one of the UI's in your browser and you provided a client cert in your browser, or logged in with LDAP/Kerberos/etc. By specifying a truststore you are only verifying the

NiFi Toolkit CLI issues with NiFi/Registry SSL handshake

2018-10-23 Thread ara m.
Hi I've setup NiFi and NiFi registry to talk securely with 2-way SSL auth (tls), and so have 2 separate keystores and truststores. The truststores of each service include the cert from the other service. I am able to go to https:// NiFI url and https:// Registry url, and see both UI's, and I am

Re: NiFi fails on cluster nodes

2018-10-23 Thread Bryan Bende
The only way I could see that happening is if the ZK config on the second node pointed at a different ZK, or at a different location in the same ZK. For example, if node 1 had: nifi.zookeeper.connect.string=node-1:2181 nifi.zookeeper.connect.timeout=3 secs nifi.zookeeper.session.timeout=3 secs

RE: NiFi fails on cluster nodes

2018-10-23 Thread Saip, Alexander (NIH/CC/BTRIS) [C]
That's exactly the case. -Original Message- From: Bryan Bende Sent: Tuesday, October 23, 2018 2:44 PM To: users@nifi.apache.org Subject: Re: NiFi fails on cluster nodes So you can get into each node's UI and they each show 1/1 for cluster nodes? It doesn't really make sense how the

Re: NiFi fails on cluster nodes

2018-10-23 Thread Bryan Bende
Also, not sure if this provides anything additional to what has already been mentioned on this thread, but this morning I wrote up the exact steps I followed to create a secure 2 node cluster to test the 1.8.0 release candidate.

Re: NiFi fails on cluster nodes

2018-10-23 Thread Bryan Bende
So you can get into each node's UI and they each show 1/1 for cluster nodes? It doesn't really make sense how the second node would form its own cluster. On Tue, Oct 23, 2018 at 2:20 PM Saip, Alexander (NIH/CC/BTRIS) [C] wrote: > > I copied over users.xml, authorizers.xml and authorizations.xml

Re: NiFi fails on cluster nodes

2018-10-23 Thread Bryan Bende
Probably easiest to copy the files over since you have other existing users/policies and you know the first node is working. On Tue, Oct 23, 2018 at 1:12 PM Saip, Alexander (NIH/CC/BTRIS) [C] wrote: > > Embarrassingly enough, there was a missing whitespace in the host DN in the > users.xml file.

Re: NiFi fails on cluster nodes

2018-10-23 Thread Bryan Bende
That means the user representing host-1 does not have permissions to proxy. You can look in authorizations.xml on nifi-1 for a policy like: That user identifier should point to a user in users.xml like: All of the user identities are case sensitive and white space

Re: Problem with remote groups reconnecting to input port after restart.

2018-10-23 Thread Joe Witt
bill can you share more about what is observed on restart? backlogs? can you share screenshots and or logs. thanks On Tue, Oct 23, 2018, 7:28 AM William Gosse wrote: > Hi, > > > > I’m trying to implement a Listfile>FetchFile pattern similar to the one > described on the following link: > >

Re: Potential Use Case

2018-10-23 Thread Mike Thomsen
I worked on a team that was packaging NiFi for distribution for people to use the flow as a service, and what they did to make it easy was export the flow.xml.gz file and add it to a custom Docker image. That way it became essentially a lift-and-shift operation. Once you do something like that,

Problem with remote groups reconnecting to input port after restart.

2018-10-23 Thread William Gosse
Hi, I'm trying to implement a Listfile>FetchFile pattern similar to the one described on the following link: https://pierrevillard.com/2017/02/23/listfetch-pattern-and-remote-process-group-in-apache-nifi/ I have multiple Listfiles process groups each containing a remote process group that

Re: Potential Use Case

2018-10-23 Thread Martijn Dekkers
FYI, we are just about to deploy NiFi to 100+ windows machines where we have to collect data, do some small transforms, and immediately write to an S3 compatible storage device. We would eventually like to use MiNiFi, but have not yet had the time to look at integrating the AWS processors into

Re: Who uses NiFi Cluster in Docker ?

2018-10-23 Thread Pierre Villard
Hi Guillaume, Vote for RC3 is in progress, if everything goes well, it could be released before the end of the week. Pierre Le mar. 23 oct. 2018 à 10:31, PICHARD, Guillaume < guillaume.pich...@sogeti.com> a écrit : > Mike, > > > > Do you know when this 1.8 version will be released ? > > > >

RE: Who uses NiFi Cluster in Docker ?

2018-10-23 Thread PICHARD, Guillaume
Mike, Do you know when this 1.8 version will be released ? De : Mike Thomsen Envoyé : vendredi 19 octobre 2018 12:22 À : users@nifi.apache.org Objet : Re: Who uses NiFi Cluster in Docker ? Guillaume, We also have a patch coming in 1.8 that exposes the clustering settings through Docker, so

Re: CaptureChangeMySQL - throwing Binlog connector communications failure

2018-10-23 Thread Koji Kawamura
Hello, Thanks for reporting the issue and the detailed analysis. I was able to reproduce by setting short wait_timeout as 30 sec. I'm not aware of any work-around to make keep alive CaptureChangeMySQL at the moment. But you can write a script to stop/start CaptureChangeMySQL processor using NiFi