Re: Back Pressure Object threshold not honored

2017-04-27 Thread Juan Sequeiros
Joe, Thanks for clarifying since I had seen the same thing. And chalked it up to research further if object meant same thing as FlowFile I just mention it because it's first instance that I notice when object is implied as FlowFile and it is confusing. Thx On Thu, Apr 27, 2017 at 8:16 PM Ke

RE: Back Pressure Object threshold not honored

2017-04-27 Thread Kevin Verhoeven
Andrew, I will look at using the ControlRate processor as that might be a better alternative to throttling my workflow because I really need the outflow to be predictable. Also, thanks Joe, I will be happy to test the change to UpdateAttribute – I have a test VM ready to go. Thanks everyone fo

Re: Back Pressure Object threshold not honored

2017-04-27 Thread Andrew Grande
I would also suggest the ControlRate processor in front of a sensitive step. There's a chance data may accumulate and sent in burst after downtime, so it ensures a predictable outflow. Andrew On Thu, Apr 27, 2017, 8:03 PM Joe Percivall wrote: > Hello Kevin, > > I believe there are two things at

Re: Back Pressure Object threshold not honored

2017-04-27 Thread Joe Percivall
Hello Kevin, I believe there are two things at play here. The first is the processor being very fast and processing the FlowFiles before back pressure gets applied. The second is that in the current distribution, UpdateAttribute uses an old style of getting higher performance and grabs batches of

RE: Back Pressure Object threshold not honored

2017-04-27 Thread Kevin Verhoeven
Thank you for your help Andy. I think you are correct, the flowfiles are very small and the previous Processor is very fast – this might explain what is happening. I’ve enclosed screenshots of the connection properties and the workflow. In the screenshot I see 400 flowfiles were allowed through

Re: Back Pressure Object threshold not honored

2017-04-27 Thread Andy LoPresto
Hi Kevin, Sorry to hear you are having this issue. Can you please provide a screenshot of the connection properties in the configuration dialog? How quickly do those flowfiles get enqueued? I think there’s a chance if they are very small & the previous processor is very fast (i.e. RouteOnAttrib

Back Pressure Object threshold not honored

2017-04-27 Thread Kevin Verhoeven
I have an odd problem. I set the Back Pressure Object threshold on a link between two Processors to 1, but 200 flowfiles are passed to the queue before back pressure is honored. I need the back pressure to be set to a small number of flowfiles to keep the source from flooding the destination. Ha

Re: Issues with debugging

2017-04-27 Thread Tim Zimmerman
Pierre, Thanks, yes I did make sure I download the corresponding source. And even tried to match up the stacktrace with the source and it all seemed to tie together. I just responded to Bryan that this morning all is working as expected. The issue may have been with NetBeans. I waited qu

Re: Issues with debugging

2017-04-27 Thread Joe Witt
no worries tim. this is a good place to ask and I guarantee this thread will help someone else later. On Thu, Apr 27, 2017 at 10:30 AM, Tim Zimmerman wrote: > Bryan, > > Thanks for the quick response. Yes I thought it quite strange myself. I > was trying to debug a processor rather than any bo

Re: Issues with debugging

2017-04-27 Thread Tim Zimmerman
Bryan, Thanks for the quick response. Yes I thought it quite strange myself. I was trying to debug a processor rather than any bootstrap process. I am pretty sure I was connected to the correct process. I did not see any additional java processes (other than my IDE) and when I disconnected I

Re: Issues with debugging

2017-04-27 Thread Pierre Villard
Hi, Dummy question but... are you sure the code of the running NiFi is exactly matching the code where you specified breakpoints? If this is not matching, that could be the reason.

Re: Issues with debugging

2017-04-27 Thread Bryan Rosander
Hey Tim, That's pretty strange that your breakpoints aren't being hit. In the past when things like that have happened to me, it's usually due to me successfully connecting but to the wrong JVM. If you're trying to debug code related to the bootstrap process itself (RunNiFi or anything it calls)

Issues with debugging

2017-04-27 Thread Tim Zimmerman
Not sure if this is a bug or misunderstanding on my part. I was trying to enable debugging so that I could troubleshoot a problem. I modified bootstrap.conf to enable debugging. I simply uncommented the java.arg.debug line and changed the port to 8187. I was able to attach to the process at 8

Re: NPE in ListenSyslog processor

2017-04-27 Thread Conrad Crampton
F5 was sending zero length messages, now it’s not and not seeing NPE any more. Despite this, excellent response to the issue from the community yet again – less than 12 hours to from report to fix! Thank you From: Andrew Grande Reply-To: "users@nifi.apache.org" Date: Tuesday, 25 April 2017 at 2