RE: Weird ListFile Issue

2019-03-22 Thread William Gosse
Yep exactly. It probably shouldn’t do anything if its null or empty. From: Andrew Grande Sent: Friday, March 22, 2019 5:04 PM To: users@nifi.apache.org Subject: Re: Weird ListFile Issue [CAUTION: This email originated from outside of Kodak Alaris. Do not click links or open attachments unless

Re: Weird ListFile Issue

2019-03-22 Thread Andrew Grande
Looks like the processor started listing CWD On Fri, Mar 22, 2019, 2:00 PM William Gosse wrote: > The real var name was aimuploaddir and the error was aimduploaddir. Think > I have a work around to prevent the tragedy was wiping out Nifi. I did this: > >

RE: Weird ListFile Issue

2019-03-22 Thread William Gosse
The real var name was aimuploaddir and the error was aimduploaddir. Think I have a work around to prevent the tragedy was wiping out Nifi. I did this: ${aimuploaddir:replaceEmpty('')} From: Joe Witt Sent: Friday, March 22, 2019 4:06 PM To: users@nifi.apache.org Subject: Re:

Re: Weird ListFile Issue

2019-03-22 Thread Joe Witt
William What was the real dir name vs the erroneous name? What might happens depends on many factors such as os behaviors. thanks On Fri, Mar 22, 2019, 12:47 PM William Gosse wrote: > I ran into kind of a weird issue with the ListFile processor. I was > referencing a variable for my input

Weird ListFile Issue

2019-03-22 Thread William Gosse
I ran into kind of a weird issue with the ListFile processor. I was referencing a variable for my input directory and had enter the variable name incorrectly. So I assumed that the with my incorrect variable name the value of the Input Directory would be null or an empty string. When I

Re: No load on second node

2019-03-22 Thread Jean-Sebastien Vachon
Ah that's what is missing from my configuration. Like I said, I've upgraded from 1.7 and didn't check for new configuration options. Thanks, that will make it a lot easier to fix From: Bryan Bende Sent: Friday, March 22, 2019 1:38 PM To: users@nifi.apache.org

Re: No load on second node

2019-03-22 Thread Bryan Bende
Hello, The host and port used for load balancing are defined in nifi.properties of each node with the following properties: # cluster load balancing properties # nifi.cluster.load.balance.host= nifi.cluster.load.balance.port=6342 nifi.cluster.load.balance.connections.per.node=4

Re: No load on second node

2019-03-22 Thread Jean-Sebastien Vachon
I am living in a very strict environment... is a new port being used for load balancing? The logs show 8088 (which is my Jetty port) which is open and available for the node not receiving anything. I've read these two articles:

Re: Best way to convert a record to a Map

2019-03-22 Thread Matt Burgess
Mike, In JoltTransformRecord I make the following call to do that: Map recordMap = (Map) DataTypeUtils.convertRecordFieldtoObject(record, RecordFieldType.RECORD.getRecordDataType(record.getSchema())); Regards, Matt On Fri, Mar 22, 2019 at 8:55 AM Mike Thomsen wrote: > > What's the best way to

No load on second node

2019-03-22 Thread Jean-Sebastien Vachon
Hi all, What's the best strategy to get load balancing working properly? I've configure one of the very first connection of my flow to use one of the load balancing option so that flows are processed on both machines. However, one of my two nodes is not doing anything. The load one the first

Re: Problem with load balancing option

2019-03-22 Thread Jean-Sebastien Vachon
Hi, FYI, I managed to get my node back by removing the node from the cluster, deleting the local flow and restart Nifi. Hope this helps identify the issue From: Jean-Sebastien Vachon Sent: Friday, March 22, 2019 10:56 AM To: users@nifi.apache.org Subject: Re:

Empty "nifi users" page.

2019-03-22 Thread DEHAY Aurelien
Hello. My "nifi users" page is empty, with only the spinner spinning at the bottom of the page. I've tried a lot of conf, and for now everything work fine, except that. There is nothing in the log, even when I tried to put "debug" for authorization parts in logback. I had the same behavior

Re: Problem with load balancing option

2019-03-22 Thread Jean-Sebastien Vachon
Hi again, I thought everything was fine but one of my node can not start.. 2019-03-22 14:51:27,811 INFO [main] o.a.n.wali.SequentialAccessWriteAheadLog Successfully recovered 10396 records in 367 milliseconds. Now checkpointing to ensure that Write-Ahead Log is in a consistent state 2019-03-22

Re: Problem with load balancing option

2019-03-22 Thread Jean-Sebastien Vachon
Hi, I stopped each node one by one and the queue is now empty. Not sure if this is a bug or intended but it does look strange from a user point of view Thanks From: Jean-Sebastien Vachon Sent: Friday, March 22, 2019 10:28 AM To: users@nifi.apache.org Subject:

Problem with load balancing option

2019-03-22 Thread Jean-Sebastien Vachon
Hi all, I've configured one of my connection to use the "partition by attribute" load balancing option. It was not working as expected and after a few tests I realized I was missing some dependencies on the cluster nodes so I stopped everything (not related to the load balancing or Nifi at

Re: Best way to convert a record to a Map

2019-03-22 Thread Andy LoPresto
I would look at ConvertRecord to understand the process and then wrap those values in a map. Andy LoPresto alopre...@apache.org alopresto.apa...@gmail.com PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4 BACE 3C6E F65B 2F7D EF69 > On Mar 22, 2019, at 14:26, Mike Thomsen wrote: > > Inside the

Re: Best way to convert a record to a Map

2019-03-22 Thread Mike Thomsen
Inside the processor. On Fri, Mar 22, 2019 at 9:24 AM Andy LoPresto wrote: > Probably using a ScriptedWriter with a couple lines of Groovy to form and > serialize the Java map. Or are you asking about inside the processor > execution? > > Andy LoPresto > alopre...@apache.org >

Re: Best way to convert a record to a Map

2019-03-22 Thread Andy LoPresto
Probably using a ScriptedWriter with a couple lines of Groovy to form and serialize the Java map. Or are you asking about inside the processor execution? Andy LoPresto alopre...@apache.org alopresto.apa...@gmail.com PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4 BACE 3C6E F65B 2F7D EF69 > On Mar

Best way to convert a record to a Map

2019-03-22 Thread Mike Thomsen
What's the best way to get a Map representation of a record? Thanks, Mike

NIFI: Transformation into new XML structure.

2019-03-22 Thread Ling, Marcus
Folks, I am struggling to get to grips with JOLT/AVRO to convert files into XML. My flow is: 1) ConsumeKAFTA (KAFTA queue consumes JSON based files from filebeat, the flowfile created is a plain/text file.) 2) UpdateAttribute. (Creates an attribute mimie.type = application/json 3)

Re: sensitive variable values ?

2019-03-22 Thread Andy LoPresto
I have done some proposals for sensitive variables storage in the variable registry, but haven’t been able to focus on it yet due to other priorities. You can see some light example mock-up, workflow, and bullet points towards the end of this presentation [1]. If you just want to write a