[
https://issues.apache.org/jira/browse/NIFI-7964?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17229554#comment-17229554
]
Joey Frazee commented on NIFI-7964:
-----------------------------------
[~esecules] I forgot to mention, this was replicated on 1.9.x. I think it
likely never worked but has just been hidden by heaps larger than the file size.
> PutAzureBlobStorage OutOfMemory Exception
> -----------------------------------------
>
> Key: NIFI-7964
> URL: https://issues.apache.org/jira/browse/NIFI-7964
> Project: Apache NiFi
> Issue Type: Bug
> Affects Versions: 1.12.1
> Reporter: Eric Secules
> Assignee: Joey Frazee
> Priority: Major
> Time Spent: 10m
> Remaining Estimate: 0h
>
> As part of my flow I upload files to azure blob storage. They can be several
> hundred MB in size. I putting a 300 MB file into my flow and it choked on the
> PutAzureBlobStorage processor with the following log message.
> {code:java}
> 2020-10-28 19:34:10,717 ERROR [Timer-Driven Process Thread-6]
> o.a.n.p.a.storage.PutAzureBlobStorage
> PutAzureBlobStorage[id=74b80a47-016d-3430-fd74-ece7653158d5]
> PutAzureBlobStorage[id=74b80a47-016d-3430-fd74-ece7653158d5] failed to
> process session due to java.lang.OutOfMemoryError: Java heap space; Processor
> Administratively Yielded for 1 sec: java.lang.OutOfMemoryError: Java heap
> space
> java.lang.OutOfMemoryError: Java heap space
> 2020-10-28 19:34:10,717 WARN [Timer-Driven Process Thread-6]
> o.a.n.controller.tasks.ConnectableTask Administratively Yielding
> PutAzureBlobStorage[id=74b80a47-016d-3430-fd74-ece7653158d5] due to uncaught
> Exception: java.lang.OutOfMemoryError: Java heap space
> java.lang.OutOfMemoryError: Java heap space
> {code}
> I did not expect this to happen because I think the PutAzureBlob processor
> should be streaming the flowfile from disk directly to blob. But this
> behaviour suggests to me that it's getting read into memory in its entirety.
> My JVM heap size is set to 512 MB, which shouldn't be a problem if streaming
> was used to upload to blob storage in chunks.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
