[
https://issues.apache.org/jira/browse/NIFI-421?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14378127#comment-14378127
]
Mark Payne commented on NIFI-421:
---------------------------------
Toivo,
I went ahead and assigned the ticket to you.
There are a few ways to go about tackling this. The ticket lays out a couple of
options. I would go with the "Stop reading from Output Stream" if backpressure
is applied. I think this is generally going to be the most desirable solution,
but it's the most involved.
The way that I would approach this is as follows:
In the onTrigger, when a Process object is created, I would save that off to a
member variable (marked volatile)
Then, where we create Runnable that we submit to the Executor, those currently
run in a 'while' loop. I would change these Runnables so that instead of
running in a while loop, it creates a single FlowFile and then returns.
Then we have to modify the log a bit for creating/handling the process. We will
first do something like:
{code}
final Process runningProcess = this.process;
if ( runningProcess == null ) {
// create the process like we do now.
}
// get the Error Stream and the Input Stream from runningProcess and handle
them like we currently do.
// if process is finished then set this.process = null;
{code}
So the idea here is instead of creating a Process and just reading from it
until the Process terminates, we read from it until we've finished creating a
single FlowFile. This way, when backpressure occurs we can stop reading from it.
Just noticed that this ticket has a 'beginner' label. It should not. This is
not a beginner ticket at all. If it seems a bit overwhelming with your level of
familiarity with the framework you can let me know and it's cool. But if you're
up for it, I'm more than happy to work with you to get it finished! :)
> ExecuteProcess does not honor back pressure setting
> ---------------------------------------------------
>
> Key: NIFI-421
> URL: https://issues.apache.org/jira/browse/NIFI-421
> Project: Apache NiFi
> Issue Type: Bug
> Components: Extensions
> Reporter: Joseph Witt
> Assignee: Toivo Adams
> Labels: beginner
> Fix For: 0.1.0
>
>
> ExecuteProcess does not pay attention to back pressure settings when
> operating on a continuous stream of output. It needs to be able to do so.
> It should also document or make optional its behavior in the event of a back
> pressure condition. One it can kill the process it is executing or two it
> can simply stop reading from the output stream. There may be valid cases for
> both options.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
