RE: Threading in a NiFi Processor

2015-08-13 Thread Mark Payne
Ryan,

The down side to that approach is that you're tying up that thread from the 
thread pool when you call sleep.

Typically, in a situation like this, where you want to 'pause' processing for a 
bit, you would call ProcessContext.yield() and then return. 
This causes the framework not to trigger that processor for some period of time 
(configurable in the Settings tab, default is 1 sec).

Does this work for you?

Thanks
-Mark



 Date: Thu, 13 Aug 2015 14:19:36 -0400
 Subject: Threading in a NiFi Processor
 From: rhendrickson.w...@gmail.com
 To: dev@nifi.apache.org

 I have always stayed away from doing any threading inside a NiFi processor.

 However, I recently came across a use-case where I'm calling a web service
 from within a custom Nifi Processor and I don't want to overwhelm the web
 service.

 I'd like to instrument a sleep in the onTrigger() or process() method of
 about 1 second, or a configurable amount.

 Is there a pattern established for Nifi Processors to accomplish this
 nicely?

 Thanks,
 Ryan
  

Threading in a NiFi Processor

2015-08-13 Thread Ryan Hendrickson
I have always stayed away from doing any threading inside a NiFi processor.

However, I recently came across a use-case where I'm calling a web service
from within a custom Nifi Processor and I don't want to overwhelm the web
service.

I'd like to instrument a sleep in the onTrigger() or process() method of
about 1 second, or a configurable amount.

Is there a pattern established for Nifi Processors to accomplish this
nicely?

Thanks,
Ryan


RE: Threading in a NiFi Processor

2015-08-13 Thread Mark Payne
Ryan,

When you call ProcessContext.yield(), all that does is tell the framework not 
to schedule you for a bit. 
So you can call it from anywhere. But the current thread will continue on. So 
the thread can return or can finish its job.

Thanks
-Mark


 Date: Thu, 13 Aug 2015 14:52:57 -0400
 Subject: Re: Threading in a NiFi Processor
 From: rhendrickson.w...@gmail.com
 To: dev@nifi.apache.org

 That may work.. Can you call ProcessContext.yield() anywhere with in the
 processor, or only at the end of the onTrigger()?

 flowfile -- processor -- output

 I thought if you called the ProcessContext.yield() you were intending to
 end the the processor's responsibility portion and output..


 On Thu, Aug 13, 2015 at 2:38 PM, Mark Payne marka...@hotmail.com wrote:

 Ryan,

 The down side to that approach is that you're tying up that thread from
 the thread pool when you call sleep.

 Typically, in a situation like this, where you want to 'pause' processing
 for a bit, you would call ProcessContext.yield() and then return.
 This causes the framework not to trigger that processor for some period of
 time (configurable in the Settings tab, default is 1 sec).

 Does this work for you?

 Thanks
 -Mark


 
 Date: Thu, 13 Aug 2015 14:19:36 -0400
 Subject: Threading in a NiFi Processor
 From: rhendrickson.w...@gmail.com
 To: dev@nifi.apache.org

 I have always stayed away from doing any threading inside a NiFi
 processor.

 However, I recently came across a use-case where I'm calling a web
 service
 from within a custom Nifi Processor and I don't want to overwhelm the web
 service.

 I'd like to instrument a sleep in the onTrigger() or process() method of
 about 1 second, or a configurable amount.

 Is there a pattern established for Nifi Processors to accomplish this
 nicely?

 Thanks,
 Ryan