Re: InvokeHTTP hangs after several successful calls

I am having this same issue and do not have access to log files. Same pattern as the others. I need to terminate the invokeHttp processor to release threads. Is this a JDK issue or an HTTP2 issue? I am not able to find any solutions that work. -- Sent from: http://apache-nifi-users-list.2

Re: InvokeHTTP hangs after several successful calls

Jeanne, This is not a JDK issue. It is specific to HTTP. You can try to apply the fix for https://issues.apache.org/jira/browse/NIFI-8181 as a patch to your current version. HTH, Vijay > On Apr 14, 2021, at 8:20 AM, jeanne-herndon > wrot

Re: Nifi throws an error when reading a large csv file

For splitting large files, it's often recommended to use a multi-stage approach. For example, if your file contains 1_000_000 records and you want to split it into 1 record per FlowFile, it would be better to split into batches of, say 1_000 records using SplitRecord and then split each of those F

Re: Nifi throws an error when reading a large csv file

What is the complete stack trace of that exception? > On Apr 14, 2021, at 02:36, Vibhath Ileperuma > wrote: > > Requested array size exceeds VM limit

Re: Release from holding queue after timeout

Are you able to penalize the flow file for 20s within the update retry queue? Penalizing without yielding would cause a flow file to sit in the queue for 20s before the next processor will try acting on it. Paul On Wed, Apr 14, 2021 at 5:05 AM Jeremy Pemberton-Pigott < fuzzych...@gmail.com> wrot

Re: Release from holding queue after timeout

Yes it can. Is that an update attribute which can set that on the flow file? Regards, Jeremy On 15 Apr 2021, at 00:16, Paul Kelly wrote:  Are you able to penalize the flow file for 20s within the update retry queue? Penalizing without yielding would cause a flow file to sit in the queue fo

Re: Release from holding queue after timeout

Whatever processor is putting it into the queue could set a penalty on failed flow files that enter your retry queue. It's one of the properties on the General tab. If you can't set it there for whatever reason, you could put an ExecuteScript processor within the retry loop with the following Gro

Re: Nifi throws an error when reading a large csv file

Hi Chris, As you have mentioned, I am trying to split the large csv file in multiple stages. But this error is thrown at the first stage even without creating a single flow file. It seems like the issue is not with the processor, but with the CSV record reader. This error is thrown while reading t

Re: Nifi throws an error when reading a large csv file

How large is each line expected to be? You could have a massive line or much larger than thought of. Or you could be creating far more flowfiles than intended. If you cut the file in size does it work better? Will need more data to help narrow in but obviously we're all very interested to know

Re: Nifi throws an error when reading a large csv file

It would be good to get the stack trace, or rather a better one to see where the array is being created. How many columns does the file have? How are you doing the schema? Which csv parser have you configured? Can you reproduce with a smaller file size ( see split command on linux )? > On Apr 1

Re: InvokeHTTP hangs after several successful calls

Upgrading Nifi from 1.11.4 to 1.13.2 resolved this problem for us. -- Sent from: http://apache-nifi-users-list.2361937.n4.nabble.com/

Re: Nifi throws an error when reading a large csv file

I could be totally barking up the wrong tree, but I think this is our clue: Requested array size exceeds VM limit That means that something is causing the reader to try to allocate an array with a number of entries greater than the VM allows. Without seeing the schema, a sample of the CSV and a s

Re: Release from holding queue after timeout

Thanks Paul, that was an excellent idea and the line is added to the Groovy code that was already in the flow. Works now :) Jeremy On Thu, Apr 15, 2021 at 12:44 AM Paul Kelly wrote: > Whatever processor is putting it into the queue could set a penalty on > failed flow files that enter your ret