Hi,

Sometimes long running processors cannot be easily stopped.
See nifi-users, Mon, 23 Nov, ExecuteProcess processor cannot be
stopped/started

And often it is desirable to be able set running timeout before processor
start.
https://issues.apache.org/jira/browse/NIFI-528  Add support to specify a
timeout on ExecuteStreamCommand, ExecuteScript, and ExecuteProcess

Such problems may be in other processors also. For example ExecuteSQL does
have timeout support but cannot be stopped in the middle of long running
query.

Both timeouts and stopping can be maybe solved together?
There are strong similarities and some code can be reused?
And some processor internal redesign is needed for both timeouts and stops.

More possibly related things:
https://issues.apache.org/jira/browse/NIFI-1183
Failed to stop failing processor

I looked ExecuteProcess code and it supports some sort stopping/cancelling

        if (!isScheduled()) {
            getLogger().info("User stopped processor; will terminate process
immediately");
            longRunningProcess.cancel(true);
            return;
        }

But this works only at certain time. After processor starts streaming
stopping don’t work anymore.

Both timeouts and prompt stopping/cancelling can be tricky.
And can be implemented in different ways.
1.      Will stop sometime in the future, we don't guarantee immediate stop.
2.      We guarantee immediate stop, this can be sometimes very difficult.

And we must accept some limitations. For example JDBC can be forced to
timeout/stop. But database server may continue to run query anyway for long
time after our cancel in client.

How to move on?
Or we have other higher priority tasks now and it’s not wise to start
handling timeouts and stopping?

Thanks
Toivo




--
View this message in context: 
http://apache-nifi-developer-list.39713.n7.nabble.com/Long-running-processors-stopping-and-timeouts-tp5583.html
Sent from the Apache NiFi Developer List mailing list archive at Nabble.com.

Reply via email to