Re: FetchSFTP vs GetSFTP

Yep that's exactly how I have it set up with a push to RPG. Is that preferred? I just started playing with it to be honest. I can see how it could be tricky if you have to pull from multiple servers each flow file could potentially have a different sftp host address in the queues. All together we

Re: Did 1.4.0 introduce a "breaking change" ?

Matt, Yes. It seems to be the case. Reason being the Config Mgmt techniques used to populate the files are different. Cheers On Wed, Nov 1, 2017 at 12:07 AM, Matt Gilman wrote: > Andre, > > Based on the error, it looks like you've retained your existing >

Re: FetchSFTP vs GetSFTP

Joe/Bryan Thanks! I believe the one specific file per concurrent task/connection (and too many threads) is the issue I have we have a lot of small files and often times backed up . I'm going to drop the task count to take advantage of the pooling. Is it possible to have Fetch do batches vs a

Re: SSL and Zookeeper

Hi Mark, Just adding to what Andy said below. Currently there is support for Kerberos/SASL authentication with Zookeeper. The admin guide provides details on that configuration and setup for embedded zookeeper and links for guidance on external Zookeeper installations.

Re: SSL and Zookeeper

Hi Mark, I believe SSL ZK connections are only supported in 3.5.0+ [1] and currently NiFi uses ZK 3.4.6 [2]. I don’t know the details on making a TLS connection to ZK, but my first thoughts would be to update the host/port combination in your NiFi configs to reference the ZK HTTPS port, and

Re: FetchSFTP vs GetSFTP

Ryan, Personally I don't have experience running these processors at scale, but from a code perspective they are fundamentally different... GetSFTP is a source processor, meaning is not being fed by an upstream connection, so when it executes it can create a connection and retrieve up to

SSL and Zookeeper

Has anyone setup zookeeper connections for a NiFi Cluster over SSL? We have ZK itself running over SSL. How do we get the NiFi to ZK connections secure? Is this possible? Advice, suggestions and/or documentation greatly appreciated. Thanks, Mark

Re: Data anonymization in Nifi

Vyshali, I would love to help, but I've never used ARX so I'm not at all familiar with their APIs. They do have an examples page though [1]. Regards, Matt [1] http://arx.deidentifier.org/overview/#a3 On Tue, Oct 31, 2017 at 1:11 PM, Vyshali wrote: > Hi Matt, > >

Re: Data anonymization in Nifi

Hi Matt, Thanks for your valuable comment. Is it possible to anonymize data without specifying generalization hierarchies in ARX.? Also,can you please help me with some basic examples using ARX APIs. Regards, Vyshali -- Sent from: http://apache-nifi-developer-list.39713.n7.nabble.com/

Re: FetchSFTP vs GetSFTP

Ryan - dont know the code specifics behind FetchSFTP off-hand but i can confirm there are users at that range for it. Thanks On Tue, Oct 31, 2017 at 11:38 AM, Ryan Ward wrote: > I've found that on a single node getSFTP is able to pull more files off a > remote server than

FetchSFTP vs GetSFTP

I've found that on a single node getSFTP is able to pull more files off a remote server than Fetch in a cluster. I noticed Fetch doesn't have a max selects so it is requiring way more connections (one per file?) and concurrent threads to keep up. Was wondering if anyone is using List/Fetch at

Re: Nifi Embedded Zookeeper Ports

Andy, Thank you for getting back to me so quickly. If I understand it correctly, I think I’m asking about something slightly different. When I run a secure Nifi cluster in a more open firewall environment the configuration works and using tcpdump it looks roughly like this: - fqdn1.37186 >

Re: Did 1.4.0 introduce a "breaking change" ?

Andre, Based on the error, it looks like you've retained your existing authorizers.xml but upgraded the nifi.security.user.authorizer property in your nifi.properties. These two are related. The authorizers.xml file defines the available Authorizers (and now UserGroupProviders and