Add support for uploading storage objects using an iterator even though driver
might not support chunked encoding
-----------------------------------------------------------------------------------------------------------------
Key: LIBCLOUD-101
URL: https://issues.apache.org/jira/browse/LIBCLOUD-101
Project: Libcloud
Issue Type: Improvement
Components: Storage
Affects Versions: 0.5.0
Reporter: Birk Nilson
Priority: Minor
Fix For: 0.5.2
Currently there are two options for uploading objects to any given storage;
StorageDriver.upload_object() and StorageDriver.upload_object_via_stream(). The
former strictly requires the object to be written to disk prior to upload since
it takes the file path as argument. The latter does support upload given an
iterator, but does not work for Amazon S3 since they do not support chunked
encoding. After discussing this with Tomaž he and I came to the conclusion that
a third method is in order which will take an iterator as argument while not
requiring the provider to support chunked encoding.
The solution is to support an iterator as argument and generate a request by
iterating through it directly. However, these will require the object to be
kept in memory and depending on the size and available resources it might lead
to exhaustion. So it should be noted in the methods documentation that the
implementation is responsible for preventing such outcomes; which can be done
by preventing the allowed upload size for example.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
