Oleksandr Lobunets created NIFI-4943:
----------------------------------------

             Summary: Batch Duration capability from ExecuteProcess added to 
ExecuteStreamCommand
                 Key: NIFI-4943
                 URL: https://issues.apache.org/jira/browse/NIFI-4943
             Project: Apache NiFi
          Issue Type: Improvement
    Affects Versions: 1.5.0
            Reporter: Oleksandr Lobunets


It would be great to let the ExecuteStreamCommand processor to send FlowFiles 
per chunk of stdout using a given separator (common case: for each line from 
stdout).

I have a case  of running the 3rd party CLI (linux) with the following 
behaviour:
- Should be executed upon a FlowFile with attributes/content containing 
parameters to CLI
- Accepts params via flags or environment variables 
- Writes output to stdout as a stream of JSON objects
- The output might be huge (millions and millions of objects), which means 
caching stdout is not an option - each line/object should be sent as a separate 
FlowFile
- The errors/log is written to stderr (might be very chatty)

Using ExecuteProcessor is not an option (cannot be trigger by incoming 
FlowFile), but the way it treats stdout is what is desired.
Using ExecuteStreamCommand is not an option as it buffers the output until the 
binary exists with a status code 0.

 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Reply via email to