Re: Title: Feature needed - ConvertJSONToCSV processor

2018-08-08 Thread Mahendra prabhu
I could write JQ for direct json values, but for dynamic json nested object I couldn't achieve with JQ. Can you please share your suggestion? major goal is to move complex nested json values into SQL after using transformation with expression language. If I get the intermediate result into CSV, it

Re: NiFi processor to execute a Java class

2018-08-08 Thread Matt Burgess
Pierre, That could very well be, I guess it just depends on what the desired feature set is. Calling a Java class and giving it access to the flow file content and attributes is easy in ExecuteScript with Groovy, so I was imagining ExecuteClass working more like ExecuteStreamCommand. I was hoping

Re: NiFi processor to execute a Java class

2018-08-08 Thread Pierre Villard
Quick remark: just wondering if (InputStream, OutputStream, Map) would be a better approach to consider flow files attributes + dynamic properties configured at processor level. Pierre 2018-08-08 17:06 GMT+02:00 Matt Burgess : > Vitaly, > > I am indeed back from vacation and I started on the

Re: NiFi processor to execute a Java class

2018-08-08 Thread Matt Burgess
Vitaly, I am indeed back from vacation and I started on the ExecuteClass processor but never wrote the Jira because I ran into a snag, namely how to get the FlowFile content into the class. In ExecuteStreamCommand and ExecuteProcess we create an external Process and route the content to the

RE: NiFi processor to execute a Java class

2018-08-08 Thread Vitaly Krivoy
Hi Matt, I just saw you message. In case you are back from your vacation and still want to talk software, may I ask you a couple of questions?  There is no description of ExecuteClass NiFi processor in version 1.7 and googling for it returns nothing. Is this something you wrote? Where can I

RE: PutDatabaseRecord runs out of connections

2018-08-08 Thread Carlos Manuel Fernandes (DSI)
I walter, try setting validation query property on database controller service. Probably in you installation there is some firewall which drops idle connections. With this property set (ex: select 1 for Sqlserver) if the processor get an invalid connection that connection is dropped and a new

PutDatabaseRecord runs out of connections

2018-08-08 Thread Vos, Walter
Hi, I'm an application admin for NiFi (among others) within my company. NiFi is very new to me and to my colleagues as well. We're running into an issue with one of our flows where a PutDatabaseRecord processor halts with the following error: " Cannot get a connection, pool error Timeout

NiFi 1.6.0 cluster stability with Site-to-Site

2018-08-08 Thread Joe Gresock
I am running a 7-node NiFi 1.6.0 cluster that performs fairly well when it's simply processing its own data (putting records in Elasticsearch, MongoDB, running transforms, etc.). However, when we add receiving Site-to-Site traffic to the mix, the CPU spikes to the point that the nodes can't talk

Re: GetMongo Truncates Dates

2018-08-08 Thread Mike Thomsen
Yes. Use a custom validator. On Tue, Aug 7, 2018 at 1:58 PM Ryan Hendrickson < ryan.andrew.hendrick...@gmail.com> wrote: > Yea, just finished testing, it works.. I added an additional property > called "Date Format" and I passed in a java date format there. Now I'm > just wondering if there's a