Re: Anti-Virus Scanning

2019-08-14 Thread Edward Armes
Hi Jason, This isn't explicitly documented anywhere, however the locations of all the key paths for Nifi can be found in the documentation in general. Hopefully a combination of this email thread and official documentation should be enough for your client to give the AV exemptions you need. Getti

Re: Anti-Virus Scanning

2019-08-14 Thread Jason Csencsits
Edward, Thank you . Appreciate the help and details. Jason Sent via the Samsung Galaxy Note8, an AT&T 4G LTE smartphone Original message From: Edward Armes Date: 8/14/19 5:28 AM (GMT-05:00) To: users@nifi.apache.org Subject: Re: Anti-Virus Scanning Hi Jason, This isn't ex

Re: My nifi no more serve admin interface

2019-08-14 Thread Edward Armes
Hi Nicolas, This is another dump question. As I've only ever seen this before when I've accidentally connect to a secured Nifi cluster over HTTP and not HTTPS. >From I've seen Nifi won't ask your browser to do a connection upgrade (HTTP -> HTTPS), When you type in the address are you sure your br

Re: Custom Processor Upgrade

2019-08-14 Thread Bryan Bende
Without access to the code for your NAR I can only really guess, but it sounds like an exception is happening when trying to call the constructor of your processor and then it bounces into creating a ghost processor. What is in the logs at the time you get the ghost processor? On Tue, Aug 13, 20

Re: My nifi no more serve admin interface

2019-08-14 Thread Nicolas Delsaux
Oh damn It appeared (after a long search) that my keystore was incorrectly built. Indeed, it contained the server certificate as a trusted certificate, where it should had been a key pair (with both private and public keys in) as is explained in Jetty documentation (https://www.eclipse.org/jetty

Re: My nifi no more serve admin interface

2019-08-14 Thread Pierre Villard
Glad you sorted it out and thanks for letting us know! In case you missed it, you might be interested by the NiFi toolkit [1] containing a TLS toolkit to help you with certificates [2]. [1] https://nifi.apache.org/download.html [2] https://nifi.apache.org/docs/nifi-docs/html/toolkit-guide.html#tls

Re: My nifi no more serve admin interface

2019-08-14 Thread Edward Armes
Hmm, I wonder if there's a change that could be made to expose this error so its a bit more obvious, maybe one for the Dev mailing list? Edward On Wed, Aug 14, 2019 at 3:12 PM Pierre Villard wrote: > Glad you sorted it out and thanks for letting us know! > In case you missed it, you might be in

Specifying formatters at a record field level

2019-08-14 Thread Mike Thomsen
If there any way to specify a timestamp format string on each field that is a TIMESTAMP (long, logical type timestamp-millis)? We have a case where we would need at least three, possibly half a dozen timestamp formats to read a record set. Thanks, Mike

FlowFile Repository can't checkpoint, out of heap space.

2019-08-14 Thread Peter Wicks (pwicks)
I have a node in a cluster whose FlowFile repository grew so fast that it exceeded the amount of available heap space and now can't checkpoint. Or that is my interpretation of the error. "Cannot update journal file flowfile_repository/journals/.journal because this journal has already enco

RE: [EXT] Specifying formatters at a record field level

2019-08-14 Thread Peter Wicks (pwicks)
Not that I’m aware of. We implemented something custom that lets you specify it with attributes on the FlowFile (something like data.field.#.format=….), we do the same thing for binary/hex fields. But we didn’t contribute it as it’s part of a custom record processing processor that’s application

Re: Custom Processor Upgrade

2019-08-14 Thread Bimal Mehta
For the custom processor we have, we are extending the AbstractNifiProcessor.java. The processor is used to scan metadata of an incoming flow file. The error we get in logs is as below. 2019-08-13 23:21:21,529 ERROR [main] o.a.nifi.controller.ExtensionBuilder Could not create Processor of type org.

Re: Custom Processor Upgrade

2019-08-14 Thread Bryan Bende
Can you edit logback.xml and add the following, the get the stacktrace again? This should include a root cause exception which we are missing right now. I think it takes about 20-30 seconds for logback to pick up the edits to logback.xml. On Wed, Aug 14, 2019 at 12:53 PM Bimal Mehta wrote: >

Re: Custom Processor Upgrade

2019-08-14 Thread Bimal Mehta
Hi Bryan, I did what you said. This is what I got 2019-08-14 20:16:18,948 DEBUG [Validate Components Thread-3] o.a.n.controller.AbstractComponentNode Computed validation errors with Validation Context StandardValidationContext[componentId= 6fbe2407-7799-3908-f4c4-bf2f8940bf1e , properties={Proper

Re: Custom Processor Upgrade

2019-08-14 Thread Bryan Bende
You have to add another instance of the processor which should generate the same stracktrace you sent earlier, except this time there should be a second part to it with "Caused by " and then more of the stacktrace that wasn't there before. On Wed, Aug 14, 2019 at 4:41 PM Bimal Mehta wrote: >

Re: Custom Processor Upgrade

2019-08-14 Thread Bimal Mehta
Ahh, seems like a Springboot error. Is it to do with upgraded Jetty server ? Caused by: org.springframework.beans.factory.BeanDefinitionStoreException: Unexpected exception parsing XML document from class path resource [application-context.xml]; nested exception is org.springframework.beans.FatalB

Re: [EXT] Specifying formatters at a record field level

2019-08-14 Thread Mike Thomsen
Hypothetically, I think the path of least resistance might be to refactor the readers and writers to allow custom properties like "timestamp.format.1...N" and have those properties evaluated until one is found that can parse the incoming text. Thoughts? On Wed, Aug 14, 2019 at 12:34 PM Peter Wicks

Re: Custom Processor Upgrade

2019-08-14 Thread Bryan Bende
Where is application-context.xml in your NAR? And how are you trying to load it in com.thinkbiganalytics.nifi.processor.AbstractNiFiProcessor ? I would expect it to be packaged into the jar that contains your processors, most likely in src/main/resources of the processors module which then ends