Does 10k happen to be your swap threshold in nifi.properties by any chance
(it defaults to 20k I believe)?
I suspect the behavior you are seeing could be due to the way swapping
works, but Mark or others could probably confirm.
I found this thread where Mark explained how swapping works with a
background thread, and I believe it still works this way:
On Tue, Sep 20, 2016 at 10:22 AM, Peter Wicks (pwicks) <pwi...@micron.com>
> I’m using JSONToSQL, followed by PutSQL. I’m using Teradata, which
> supports a special JDBC mode called FastLoad, designed for a minimum of
> 100,000 rows of data per batch.
> What I’m finding is that when PutSQL requests a new batch of FlowFiles
> from the queue, which has over 1 million rows in it, with a batch size of
> 1000000, it always returns a maximum of 10k. How can I get my obscenely
> sized batch request to return all the FlowFile’s I’m asking for?