Alex, Generally the threading in NiFi is handled by the concurrent tasks configured for the processor, meaning more threads able to call the onTrigger method. For some source/sink type processors, which sounds like your processor, it can be acceptable to create a controlled number of threads. There are a couple of processors that take the approach of starting a background thread in an @OnScheduled method to put messages onto a queue that are then pulled off for processing in the onTrigger method. A good example to look at might be the GetTwitter processor which has a BlockingQueue of messages. You will definitely want to set a capacity on the queue to prevent out-of-memory scenarios if a high volume of messages are coming through.
Let us know if this helps, or if there is any more information we can provide. -Bryan On Thu, Nov 5, 2015 at 9:49 AM, McCullough, Alex < [email protected]> wrote: > Hey There, > > This is a bit of a generic question, and I am not sure if this is the > right place to start but I am new to developing custom processors in Nifi > but have been trying to write an asynchronous message consumer to consume > from IBM MQ. > > I have read all of the developer documentation and have been testing a > processor I built, but I think where I am struggling is conceptualizing > exactly how I should set this up within the framework and how the threading > within Nifi works exactly. > > Typically I am used to registering a listener and using the onMessage > method to add messages that are received to local queue to then be > processed. What would this look like within the Nifi framework? Are there > any guiding principals you can point me towards or any source code that may > be doing something somewhat similar? > > Sorry this is a pretty vague question and I’m sure there are many ways to > skin this particular cat, but hoping you can help point me generally down > the right path. > > Thanks a ton and please let me know if you want me to provide anymore > detail! > > Thanks, > Alex > > ________________________________________________________ > > The information contained in this e-mail is confidential and/or > proprietary to Capital One and/or its affiliates and may only be used > solely in performance of work or services for Capital One. The information > transmitted herewith is intended only for use by the individual or entity > to which it is addressed. If the reader of this message is not the intended > recipient, you are hereby notified that any review, retransmission, > dissemination, distribution, copying or other use of, or taking of any > action in reliance upon this information is strictly prohibited. If you > have received this communication in error, please contact the sender and > delete the material from your computer. >
