Re: MergeContent Demarcator Question

2015-04-23 Thread Joe Witt
Are you sure you're not sending the [ , ] over UDP as well ;-) Can you create a template of your flow and send it over? Perhaps just attach to a JIRA for this. MergeContent is a powerful and useful thing so if you're seeing funky behavior we want to sort it out quickly. On Thu, Apr 23, 2015 at

MergeContent Demarcator Question

2015-04-23 Thread Bryan Bende
I'm trying to use MergeContent to merge json documents. I have the Header. Demarcator, and Footer properties pointing to files with [ , ] respectively. I left all other properties the same, and set Max Entries to 5 and Max Bin Age to 10 seconds. I have a simple flow with ListenUDP -> MergeContent

Re: NiFi Processor Invalid State Issues

2015-04-23 Thread Jim Reilly
Matt, That was it, tkx!!! I hovered in the wrong place  :-)   Tkx, Jim From: Matt Gilman To: Jim Reilly ; "dev@nifi.incubator.apache.org" Sent: Thursday, April 23, 2015 12:11 PM Subject: Re: NiFi Processor Invalid State Issues Jim, If you hover over the invalid icon it should

Re: Expression Language Extensions/Plugins

2015-04-23 Thread Mark Payne
Brian, I would encourage you to create a JIRA ticket with these ideas. They are good ideas that we don't want to lose. As far as allowing pluggable functions, that's likely a pretty big task and not likely to have too soon. But these functions could be implemented much sooner than that, I wou

Re: ExecuteStreamCommand with output directory

2015-04-23 Thread Brandon DeVries
Mike, Yes, it's a bit of a compromise. But honestly, when you use ExecuteStreamCommand you're stepping a bit outside of NiFi, and its probably going to be less than ideal. As Joe said, writing your own processor would allow you to handle things a bit more directly and efficiently. However,

Re: ExecuteStreamCommand with output directory

2015-04-23 Thread Mark Payne
Mike, Any chance you can control where the data is written to by the script? If you write it for instance to ".myDir" your "wrapper" script could just rename it form ".myDir" to "myDir". GetFile by default does not pick up any file that begins with a . -- Original Message -- From: "

Re: ExecuteStreamCommand with output directory

2015-04-23 Thread mbwagne
Brandon's approach did work. Thanks Brandon! My concern is I'm tarring up a directory in a shell script just to unpack in a nifi processor. That seems like a lot of unnecessary IO. I wish I could trigger GetFile on the completion of the ExecuteStreamCommand via the success route. -- View this m

Re: Expression Language Extensions/Plugins

2015-04-23 Thread Brian Ghigiarelli
Also, we'd like to be able to format a timestamp (number) into a particular timezone regardless of the timezone set on the host. Since NiFi uses SimpleDateFormat, that's exposed programmatically via sdf.setTimeZone(...). On Thu, Apr 23, 2015 at 10:20 AM, Brian Ghigiarelli wrote: > Thanks! A fe

Re: NiFi Processor Invalid State Issues

2015-04-23 Thread Jennifer Barnabee
Jim, When you hover the mouse over the invalid icon, does it tell you anything? -Jenn Sent from my iPhone > On Apr 23, 2015, at 12:02 PM, Jim Reilly > wrote: > > Hi, > When using NiFi and adding a Processor (say for example in my case: > PostHTTP), how do you know what the issue is when creat

Re: NiFi Processor Invalid State Issues

2015-04-23 Thread Mark Payne
Jim, In your browser, if you hover over the "Invalid" icon on the Processor, it will give you a tooltip explaining why the Processor is not valid. Thanks -Mark -- Original Message -- From: "Jim Reilly" To: "dev@nifi.incubator.apache.org" Sent: 4/23/2015 12:02:16 PM Subject: NiFi Pro

NiFi Processor Invalid State Issues

2015-04-23 Thread Jim Reilly
Hi, When using NiFi and adding a Processor (say for example in my case: PostHTTP), how do you know what the issue is when creating a processor, when its not valid?   It shows the invalid icon, the log files says something like: "processor is not in a valid state", but, it does not say what the i

Re: Expression Language Extensions/Plugins

2015-04-23 Thread Brian Ghigiarelli
Thanks! A few options for extension language enhancements I can think of off-hand are: - Capitalize - Pretty Date ( e.g., "yesterday", "an hour ago", "a few seconds ago" ) - Phone Formatter - IP v4 and v6 Formatters While a few of those can be done with some chaining of the existing