Does 10k happen to be your swap threshold in by any chance
(it defaults to 20k I believe)?

I suspect the behavior you are seeing could be due to the way swapping
works, but Mark or others could probably confirm.

I found this thread where Mark explained how swapping works with a
background thread, and I believe it still works this way:


On Tue, Sep 20, 2016 at 10:22 AM, Peter Wicks (pwicks) <>

> I’m using JSONToSQL, followed by PutSQL.  I’m using Teradata, which
> supports a special JDBC mode called FastLoad, designed for a minimum of
> 100,000 rows of data per batch.
> What I’m finding is that when PutSQL requests a new batch of FlowFiles
> from the queue, which has over 1 million rows in it, with a batch size of
> 1000000, it always returns a maximum of 10k.  How can I get my obscenely
> sized batch request to return all the FlowFile’s I’m asking for?
> Thanks,
>   Peter

Reply via email to