I've filed a pull request: https://github.com/apache/libcloud/pull/568
On 22.07.2015 09:30, Torf wrote: > I've filed an issue: > > https://issues.apache.org/jira/browse/LIBCLOUD-725 > > > On 21.07.2015 13:40, Torf wrote: >> Great, will do that. >> >> >> On 21.07.2015 08:21, Tomaz Muraus wrote: >>> It would be great if you can submit a pull request with a fix and test >>> cases upstream :) >>> >>> https://libcloud.readthedocs.org/en/latest/development.html#contributing >>> >>> On Tue, Jul 21, 2015 at 2:11 PM, Torf <m...@torf.cc> wrote: >>> >>>> Hi everyone, >>>> >>>> Thanks for your input, Chris! So it seems to be indeed a problem with >>>> LocalStorageDriver, not with my understanding :) >>>> >>>> What can I do to help getting this fixed in the official distribution? >>>> I'd be happy to provide a patch, test cases, etc. >>>> >>>> >>>> Regards, >>>> Torf >>>> >>>> >>>> On 21.07.2015 00:01, Chris Richards wrote: >>>>> Torf, >>>>> >>>>> I stumbled across the same thing and modified my local repository copy. >>>> I >>>>> don't know if this was the correct way for libcloud proper, but it moved >>>> me >>>>> past the point where I was stuck. (There may be other places that need >>>> it, >>>>> but for the APIs I'm using they're sufficient.) You'll also need to do "$ >>>>> pip3 install lockfile" to use the LocalStorage driver. >>>>> >>>>> diff -r a61cf0eec1a7 -r ad3deb3173ce libcloud/storage/drivers/local.py >>>>> --- a/libcloud/storage/drivers/local.py Wed Jun 25 18:59:43 2014 -0500 >>>>> +++ b/libcloud/storage/drivers/local.py Wed Jun 25 19:03:33 2014 -0500 >>>>> @@ -406,7 +406,7 @@ >>>>> >>>>> path = self.get_object_cdn_url(obj) >>>>> >>>>> - with open(path) as obj_file: >>>>> + with open(path, 'rb') as obj_file: >>>>> for data in read_in_chunks(obj_file, chunk_size=chunk_size): >>>>> yield data >>>>> >>>>> @@ -490,7 +490,7 @@ >>>>> self._make_path(base_path) >>>>> >>>>> with LockLocalStorage(obj_path): >>>>> - obj_file = open(obj_path, 'w') >>>>> + obj_file = open(obj_path, 'wb') >>>>> for data in iterator: >>>>> obj_file.write(data) >>>>> >>>>> As for the upload, here's how I call it: >>>>> >>>>> obj = driver.upload_object_via_stream ( >>>>> BytesIO (data), container, objname, >>>>> extra={'content_type': 'application/octet-stream'}) >>>>> >>>>> Again, I don't know if this is libcloud proper or not; it works for me! >>>>> (Ship it!) >>>>> >>>>> HTH, >>>>> Chris >>>>> >>>>> On Mon, Jul 20, 2015 at 4:43 PM, Torf <m...@torf.cc> wrote: >>>>> >>>>>> Hi, >>>>>> >>>>>> I'm trying to store binary data in Python 3 to a ``LocalStorageDriver`` >>>>>> instance. If I upload data via ``upload_object`` from a file on disk and >>>>>> then use ``download_object_as_stream`` I get a stream of bytes, as >>>>>> expected. However, uploading a stream of bytes manually using >>>>>> ``upload_object_via_stream`` fails: >>>>>> >>>>>> >>>>>>>>> container.upload_object_via_stream(io.BytesIO(b'foobar'), 'key') >>>>>> Traceback (most recent call last): >>>>>> File "<stdin>", line 1, in <module> >>>>>> File >>>>>> >>>>>> >>>> "/home/torf/projects/coba/venv/lib/python3.4/site-packages/libcloud/storage/base.py", >>>>>> line 157, in upload_object_via_stream >>>>>> iterator, self, object_name, extra=extra, **kwargs) >>>>>> File >>>>>> >>>>>> >>>> "/home/torf/projects/coba/venv/lib/python3.4/site-packages/libcloud/storage/drivers/local.py", >>>>>> line 497, in upload_object_via_stream >>>>>> obj_file.close() >>>>>> File >>>>>> >>>>>> >>>> "/home/torf/projects/coba/venv/lib/python3.4/site-packages/libcloud/storage/drivers/local.py", >>>>>> line 69, in __exit__ >>>>>> raise value >>>>>> File >>>>>> >>>>>> >>>> "/home/torf/projects/coba/venv/lib/python3.4/site-packages/libcloud/storage/drivers/local.py", >>>>>> line 495, in upload_object_via_stream >>>>>> obj_file.write(data) >>>>>> TypeError: must be str, not bytes >>>>>> >>>>>> >>>>>> The problem seems to be that ``LocalStorageDriver`` opens the target >>>>>> file for writing in text mode ('w') instead of binary mode ('wb'). >>>>>> >>>>>> I couldn't find any information on whether keys and values are supposed >>>>>> to be (Unicode) strings or bytes (or both), so I am not sure whether >>>>>> this is a bug in ``LocalStorageDriver`` or a misunderstanding on my >>>> side. >>>>>> >>>>>> This is on Python 3.4 and LibCloud 0.17.0. >>>>>> >>>>>> >>>>>> Regards, >>>>>> Torf >>>>>> >>>>> >>>> >>>> >>> >> >> >> > > >
