Re: GRPC processors missing after upgrade to 1.11.3

2020-03-09 Thread Joe Witt
Dirk Apologies on the grpc bundle being included. Just a mistake on my part during the RM process. Thanks Joe On Mon, Mar 9, 2020 at 9:18 AM Bryan Bende wrote: > Hello, > > We likely need Joe to confirm about grpc since he was the RM for the last > few releases, but I believe that it was

Re: NiFi Registry tarball modes

2020-03-09 Thread Bryan Bende
Hello, It looks like most of the file modes in NiFi Registry were set to 0660, where as NiFi sets them to 0664. I created this JIRA [1] and submitted a PR [2]. -Bryan [1] https://issues.apache.org/jira/browse/NIFIREG-369 [2] https://github.com/apache/nifi-registry/pull/265 On Mon, Mar 9,

Re: GRPC processors missing after upgrade to 1.11.3

2020-03-09 Thread Bryan Bende
Hello, We likely need Joe to confirm about grpc since he was the RM for the last few releases, but I believe that it was probably just a mistake in the build process that caused it to not be excluded. For the other ones you referenced, those were purposely excluded due to space limitations with

Re: Exception is showing in nifi UI users page

2020-03-09 Thread Bryan Bende
There is no file that contains this mapping, the users and groups are periodically loaded from LDAP into memory on each nifi node. The id of each user and group is a UUID that is seeded from the corresponding name. UUID.nameUUIDFromBytes(seed.getBytes(StandardCharsets.UTF_8)).toString(); For a

Re: Listing a folder with millions of files

2020-03-09 Thread Jeremy Pemberton-Pigott
Thanks for the suggestions guys. The pre-filtered list is possibly one I can use. Regards, Jeremy > On 9 Mar 2020, at 20:16, Shawn Weeks wrote: >  When I’ve had to do this I just skip trying to use ListFile and instead create a text file containing a list of all the files that can be

GRPC processors missing after upgrade to 1.11.3

2020-03-09 Thread Dirk Arends
Hey, I recently upgraded from 1.11.1 to 1.11.3 and found that the GRPC processors (ListenGRPC and InvokeGRPC) had seemingly been removed from the packaged release. But it seems that the source code in the Nifi repo for the “nifi-grpc-bundle” is still there and has not been removed. I have a

Re: Listing a folder with millions of files

2020-03-09 Thread Shawn Weeks
When I’ve had to do this I just skip trying to use ListFile and instead create a text file containing a list of all the files that can be used with the SplitFile and FetchFile processors to pull things in in batches. Even with filtering ListFile will iterate through a lot of files. Thanks

Re: Nifi installation - port 8080 being used even nifi installed with a different port

2020-03-09 Thread Shawn Weeks
You’ll need to add this line to your bootstrap.conf file to disable the Zookeeper Admin Server. We’ve done this for you in NiFi 1.11.x java.arg.17=-Dzookeeper.admin.enableServer=false From: Samarendra Sahoo Reply-To: "users@nifi.apache.org" Date: Friday, March 6, 2020 at 11:17 PM To:

Re: Listing a folder with millions of files

2020-03-09 Thread Edward Armes
Hi Jeremy, In this case I don't think there is an easy answer here. You may have some luck with adjusting the max runtime of the processor but without checking the the processors implementation I couldn't know for certain if that would have any effect. Edward On Mon, 9 Mar 2020, 06:34 Jeremy

NiFi Registry tarball modes

2020-03-09 Thread Johannes J. Meixner
Moin, The jars inside of nifi-registry-0.5.0-bin.tar.gz are owned by bbende:staff with mode 0660. In a future version of the tarball, would you mind chmodding the files in lib/ and subdirectories to 0644, so I don't have to? Thanks Johannes signature.asc Description: OpenPGP digital

Re: Exception is showing in nifi UI users page

2020-03-09 Thread sanjeet rath
Hi team, I have the AD group name from the authoriser.xml file i want to delete the corresponding Unique group dientifier value from authorisation.xml file. Could you please help me to find out the file where we are keeping the relation map between AD group name and corresponding Unique group

Listing a folder with millions of files

2020-03-09 Thread Jeremy Pemberton-Pigott
Hi, I need to list a sub-set (few 100,000) of files in a folder with millions of files (to do some historical processing). What's the best way I can do that? ListFiles is taking way too long and seems to try to dump the entire list to the flow when I test it on a smaller folder list. It would