[
https://issues.apache.org/jira/browse/NIFI-8605?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vibhath Arunapriya Ileperuma reassigned NIFI-8605:
--------------------------------------------------
Assignee: Vibhath Arunapriya Ileperuma
> ExecuteSQLRecord processor consumes a large heap volume when use with
> PostgreSQL JDBC driver
> --------------------------------------------------------------------------------------------
>
> Key: NIFI-8605
> URL: https://issues.apache.org/jira/browse/NIFI-8605
> Project: Apache NiFi
> Issue Type: Bug
> Affects Versions: 1.13.2
> Reporter: Vibhath Arunapriya Ileperuma
> Assignee: Vibhath Arunapriya Ileperuma
> Priority: Major
> Labels: Beginner, beginner
> Attachments: GC.LOG
>
>
> I'm using ExecuteSQLRecord processor to query from PostgreSQL. A 'select'
> query I'm using can return more than 60 million rows. I have configured the
> fetch size to 1000 to avoid fetching all the data into memory at once.
> But when the processor is started, heap starts to grow very fast. I have
> configured to NIFI to have 50GB heap size and even that amount is filled
> within minutes. When the heap is filled Garbage collector tries to clean the
> heap blocking other threads.
> It seems like NIFI loads all the data to memory even though fetch size is set
> to 1000. I have attached the NIFI's GC log here with this ticket for
> reference.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
