I like this idea, Toivo. The nifi guys can weigh in on pros/cons of incorporating it into the framework, but my team has implemented a TimeLimitedCallback and the corresponding Properties that do what you're suggesting. It'd be even better if there were hooks in the processor API that let you interrupt your code (soft interrupt) or killed the thread (hard interrupt).
On Thu, Dec 3, 2015 at 1:08 PM, Toivo Adams <[email protected]> wrote: > Hi, > > Sometimes long running processors cannot be easily stopped. > See nifi-users, Mon, 23 Nov, ExecuteProcess processor cannot be > stopped/started > > And often it is desirable to be able set running timeout before processor > start. > https://issues.apache.org/jira/browse/NIFI-528 Add support to specify a > timeout on ExecuteStreamCommand, ExecuteScript, and ExecuteProcess > > Such problems may be in other processors also. For example ExecuteSQL does > have timeout support but cannot be stopped in the middle of long running > query. > > Both timeouts and stopping can be maybe solved together? > There are strong similarities and some code can be reused? > And some processor internal redesign is needed for both timeouts and stops. > > More possibly related things: > https://issues.apache.org/jira/browse/NIFI-1183 > Failed to stop failing processor > > I looked ExecuteProcess code and it supports some sort stopping/cancelling > > if (!isScheduled()) { > getLogger().info("User stopped processor; will terminate > process > immediately"); > longRunningProcess.cancel(true); > return; > } > > But this works only at certain time. After processor starts streaming > stopping don’t work anymore. > > Both timeouts and prompt stopping/cancelling can be tricky. > And can be implemented in different ways. > 1. Will stop sometime in the future, we don't guarantee immediate > stop. > 2. We guarantee immediate stop, this can be sometimes very difficult. > > And we must accept some limitations. For example JDBC can be forced to > timeout/stop. But database server may continue to run query anyway for long > time after our cancel in client. > > How to move on? > Or we have other higher priority tasks now and it’s not wise to start > handling timeouts and stopping? > > Thanks > Toivo > > > > > -- > View this message in context: > http://apache-nifi-developer-list.39713.n7.nabble.com/Long-running-processors-stopping-and-timeouts-tp5583.html > Sent from the Apache NiFi Developer List mailing list archive at > Nabble.com. > -- I know what it is to be in need, and I know what it is to have plenty. I have learned the secret of being content in any and every situation, whether well fed or hungry, whether living in plenty or in want. I can do all this through him who gives me strength. *-Philippians 4:12-13*
