Hi Joe, Thanks for confirming, I logged this bug https://issues.apache.org/jira/browse/NIFI-7964
-Eric On Wed, Oct 28, 2020 at 2:10 PM Joe Witt <[email protected]> wrote: > Eric > > Your assumption is definitely what it should be. This will need someone > to investigate. Please file a jira with as much detail as you can. > > Thanks > > On Wed, Oct 28, 2020 at 12:40 PM Eric Secules <[email protected]> wrote: > >> Hello everyone, >> >> I am trying to upload a 300 MB file to azure blob storage using >> PutAzureBlobStorage and the processor is failing due to an >> OutOfMemoryError. My JVM heap size is set to 512 MB, but I wouldn't expect >> this to be an issue because the PutAzureBlobStorage processor should be >> using streaming to send the file to azure in chunks rather than reading it >> into memory in its entirety and then sending it out. >> >> I am using NiFi version 1.12.1 >> >> Here's the error from the NiFi logs: >> >>> 2020-10-28 19:34:10,717 ERROR [Timer-Driven Process Thread-6] >>> o.a.n.p.a.storage.PutAzureBlobStorage >>> PutAzureBlobStorage[id=74b80a47-016d-3430-fd74-ece7653158d5] >>> PutAzureBlobStorage[id=74b80a47-016d-3430-fd74-ece7653158d5] failed to >>> process session due to java.lang.OutOfMemoryError: Java heap space; >>> Processor Administratively Yielded for 1 sec: java.lang.OutOfMemoryError: >>> Java heap space >>> java.lang.OutOfMemoryError: Java heap space >>> 2020-10-28 19:34:10,717 WARN [Timer-Driven Process Thread-6] >>> o.a.n.controller.tasks.ConnectableTask Administratively Yielding >>> PutAzureBlobStorage[id=74b80a47-016d-3430-fd74-ece7653158d5] due to >>> uncaught Exception: java.lang.OutOfMemoryError: Java heap space >>> java.lang.OutOfMemoryError: Java heap space >>> >> >> What's the recommendation for using NiFi to upload files of this size to >> blob storage? >> >> Thanks, >> Eric >> >
