[
https://issues.apache.org/jira/browse/JCLOUDS-769?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14377758#comment-14377758
]
Andrew Gaul commented on JCLOUDS-769:
-------------------------------------
This is tricky to implement since jclouds needs to satisfy Amazon's requirement
for a Content-length per part. Depending on the chosen part size, this can
require 5 MB to 5 GB of memory per part and per simultaneous upload or spooling
the stream to a temporary file on-disk, as s3distcp does. Alternatively, we
could expose the uploadPart method to the users who can do their own buffering
and size computation.
> Upload blob from stream
> -----------------------
>
> Key: JCLOUDS-769
> URL: https://issues.apache.org/jira/browse/JCLOUDS-769
> Project: jclouds
> Issue Type: New Feature
> Components: jclouds-blobstore
> Affects Versions: 1.8.1
> Reporter: Akos Hajnal
> Labels: multipart, s3
>
> Dear Developers,
> It was not easy, but using S3 API, it was possible to upload a large blob
> from stream - without knowing its size in advance (and storing all the data
> locally). I found solutions using jclouds' aws-s3 specific API (some async
> interface), but I really miss this feature from jclouds' general API.
> My dream is to have a method like:
> blob.getOutputStream() into which I can write as many data as I want,
> which pushes data to the storage simultaneously until I close the stream.
> (When I used S3, I created a wrapper class extending OutputStream, which
> initiates multipart upload, buffers data written to the output stream, writes
> a part when the buffer is full, and finalizes multipart upload on stream
> close.)
> I don't know it is possible for all providers, but I really miss it...
> Thank you,
> Akos Hajnal
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
