Is it possible to consume messages from IBM WESB SIB topic using Apache Nifi ConsumeJMS component?

2018-03-06 Thread Tian TD Deng
Dear All, I was wondering if it's possible to consume messages from IBM WESB SIB topic using Apache Nifi ConsumeJMS component? I have tried to consume from WMQ topic and it worked, just curious if Nifi supports to consume from IBM WESB SIB topic. Really appreciate for your help. Thanks & Best Regar

NiFi retry capabilities

2018-03-06 Thread Boris Tyukin
Just found this Jira https://issues.apache.org/jira/browse/NIFI-90 I am surprised it has not got any traction after 3 years...Having used Apache Airflow for a while, I am looking to retry capabilities in NiFi and it seems it comes down to "build your own" flow approach, that would handle retries i

Re: NiFi retry capabilities

2018-03-06 Thread Mark Payne
Hey Boris, Using the UpdateAttribute and RouteOnAttribute approach is only necessary when you want to retry N number of times (or for some time period) and after that elapses to treat the data differently. Most of the time, though, what is used is to simply loop the 'failure' relationship back

Re: NiFi retry capabilities

2018-03-06 Thread Boris Tyukin
Hi Mark, thanks for your response! especially because I saw your name in that Jira :) I think it makes sense to "keep trying until you're successful". I am a bit confused by "yield" and "penalize" parameters. Can you give me an example how they are used? Let's say, I use ExecuteSQL processor and

Re: NiFi retry capabilities

2018-03-06 Thread Bryan Bende
Boris, "Penalty Duration" is per flow file, and "Yield" is for the processor. If the processor penalizes a flow file and transfers it to a queue, whatever is processing that queue won't take that flow file from the queue until the penalty duration has passed. If a processor yields, then the fram

Re: NiFi retry capabilities

2018-03-06 Thread Boris Tyukin
Bryan, you managed to explain it in three sentences! :) now I get it! thanks a bunch On Tue, Mar 6, 2018 at 4:23 PM, Bryan Bende wrote: > Boris, > > "Penalty Duration" is per flow file, and "Yield" is for the processor. > > If the processor penalizes a flow file and transfers it to a queue, > w

NiPyApi v0.8.0 released

2018-03-06 Thread Daniel Chaffelson
I'm pleased to announce that NiPyApi, a Python Client SDK for Apache NiFi has a new version out. Highlights include supporting secured environments, flow versioning import/export, better templates, better documentation, more demos, and NiFi version backtesting. There are extensive release notes i