Joe, Appreciate the guidance. Yeah, the standard processor is close but not exactly what we needed, as we need it to find all files the first time around, then only modified files from that point forward, along with some other flexibilities (and preferred to avoid scripting, so the processor is self-contained).
What's odd is that I haven't change the I/O aspect of the original ExecuteProcess java code, just some properties and how commands get structured. I suspect it has something to do with the timing associated with that "Batch" property, which seems a bit touchy from a timing standpoint... will track it down. Thanks Rick -----Original Message----- From: Joe Witt [mailto:[email protected]] Sent: Wednesday, September 02, 2015 7:24 PM To: [email protected] Subject: Re: ExecuteProcess and stdout flush? Rick, That function, calling a process from Java and reliably interacting with the streams, was a surprisingly tricky thing to get right. One of the key things is to make sure you're always fully consuming the stream and such. Pay close attention to the trickery in the standard processor. Now instead of modifying the processor in this manner you may consider simply executing a script instead and invoking that. It may give you more control and more natural control. Not trying to discourage you from building a NiFi processor by any means but in this case you're sort of on the 'edge of java and system specific commands'. Tricky road there. Thanks Joe On Wed, Sep 2, 2015 at 6:59 PM, Rick Braddy <[email protected]> wrote: > Further troubleshooting today... used same "find" command via ExecuteProcess > standard processor. It works fine, so there's something wrong with my > customized processor... will debug it to resolve. > > -----Original Message----- > From: Rick Braddy [mailto:[email protected]] > Sent: Tuesday, September 01, 2015 10:04 PM > To: [email protected] > Subject: ExecuteProcess and stdout flush? > > Hi, > > I have a slightly modified version of ExecuteProcess that's been customized > to do various "find targetdir -print" style commands, which generates > standard output that results in a FlowFile. Large outputs (long directory > listings of 100 lines or more) work perfectly; however, it appears that brief > outputs from find to stdout (e.g., 8 to 10 lines) are not being picked up at > all (the FlowFile length is zero after the find command runs). In the > debugger I can see the find command is properly formed, then after it runs > the FlowFile is zero bytes in length. > > I suspect this is some kind of issue with stdout not being flushed > sufficiently by "find" to be picked up as input, but not sure at this point. > The process flow is a bit of a mystery at this point, so wondering if anyone > might shed some light on how to troubleshoot/resolve (this most likely > happens on a standard ExecuteProcess processor, but haven't tried to > reproduce it on that processor yet). > > Rick
