[
https://issues.apache.org/jira/browse/JCLOUDS-769?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14377775#comment-14377775
]
Akos Hajnal commented on JCLOUDS-769:
-------------------------------------
As far as I know, in S3, you can use part sizes less than 5MB as well (e.g.,
the last, "fraction" part uploaded can easily be even 1 byte-long), maybe 5MB
is a good tradeoff. The content-length must be specified for the part only (its
size) at uploading (because of PUT), and even uploaded parts may be of
different lengths (I didn't try though). Moreover, in the case of output stream
"emulation", you don't have to do simultaneous uploads, you have the option to
block output stream writes until the upload of the most recent part (buffer
full) is done (if you want to save memory).
> Upload blob from stream
> -----------------------
>
> Key: JCLOUDS-769
> URL: https://issues.apache.org/jira/browse/JCLOUDS-769
> Project: jclouds
> Issue Type: New Feature
> Components: jclouds-blobstore
> Affects Versions: 1.8.1
> Reporter: Akos Hajnal
> Labels: multipart, s3
>
> Dear Developers,
> It was not easy, but using S3 API, it was possible to upload a large blob
> from stream - without knowing its size in advance (and storing all the data
> locally). I found solutions using jclouds' aws-s3 specific API (some async
> interface), but I really miss this feature from jclouds' general API.
> My dream is to have a method like:
> blob.getOutputStream() into which I can write as many data as I want,
> which pushes data to the storage simultaneously until I close the stream.
> (When I used S3, I created a wrapper class extending OutputStream, which
> initiates multipart upload, buffers data written to the output stream, writes
> a part when the buffer is full, and finalizes multipart upload on stream
> close.)
> I don't know it is possible for all providers, but I really miss it...
> Thank you,
> Akos Hajnal
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
