Hi,
You are putting the entire file into memory by running
byteSource.runFold(ByteString.empty) { case (acc, i) => acc ++ i
}.map(s => s.utf8String)
This means the body is first completely read into memory as a ByteString
and then transformed into a String.
To stream the byte chunks directly to a file you would do something like
this:
val futureWriteResult: Future[IOResult] =
byteSource.runWith(FileIO.toPath(uploadFile))
--
Johan
Akka Team
On Fri, Aug 4, 2017 at 7:01 AM, wrote:
> Does Akka directive "fileUpload" copies the entire input entity file into
> memory? Is there a way to do multipart upload of large file as 1 GB without
> consuming 1 GB of memory?
>
> My below code to upload 300MB is consuming 300MB of memory. Is this
> expected behavior with "fileUpload" directive?
>
>
> curl -k -i -X PUT --header --form "csv=@myfile300MB.txt"
> https://abc/uploadContent
>
> val uploadFile = File.createTempFile("uploadFile", ".txt")
> extractRequestContext { ctx =>
> implicit val materializer = ctx.materializer
> implicit val ec = ctx.executionContext
> fileUpload("csv") {
> case (metadata, byteSource) =>
> val sumF = byteSource.runFold(ByteString.empty) { case (acc, i) =>
> acc ++ i }.map(s => s.utf8String)
>
> onSuccess(sumF) { sum =>
> Files.write(Paths.get(uploadFile.getAbsolutePath),
> sum.toString.getBytes)
> logger.info(StatusCodes.OK + "Successfully completed fileUpload ")
> complete(s"Successfully completed fileUpload") }}
> }
>
> Thanks *johanandren *for your below response:
>
> Hi @mahadev-khapali , we use the
> issue tracker for bugs and feature requests, please use the mailing list
> https://groups.google.com/forum/#!forum/akka-user or the gitter chat
> https://gitter.im/akka/akka for questions like this, thanks!
>
> (And yes, if you use fold like that you are collecting the entire upload
> into memory, you should instead stream it directly to a file sink if you
> want to avoid that)
>
>
> Can you somebody suggest how do I directly stream to a file sink so that I
> dont upload into memroy?
>
> --
> >> Read the docs: http://akka.io/docs/
> >> Check the FAQ: http://doc.akka.io/docs/akka/
> current/additional/faq.html
> >> Search the archives: https://groups.google.com/group/akka-user
> ---
> You received this message because you are subscribed to the Google Groups
> "Akka User List" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to akka-user+unsubscr...@googlegroups.com.
> To post to this group, send email to akka-user@googlegroups.com.
> Visit this group at https://groups.google.com/group/akka-user.
> For more options, visit https://groups.google.com/d/optout.
--
>> Read the docs: http://akka.io/docs/
>> Check the FAQ:
>> http://doc.akka.io/docs/akka/current/additional/faq.html
>> Search the archives: https://groups.google.com/group/akka-user
---
You received this message because you are subscribed to the Google Groups "Akka
User List" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to akka-user+unsubscr...@googlegroups.com.
To post to this group, send email to akka-user@googlegroups.com.
Visit this group at https://groups.google.com/group/akka-user.
For more options, visit https://groups.google.com/d/optout.
