It would be great if you can submit a pull request with a fix and test cases upstream :)
https://libcloud.readthedocs.org/en/latest/development.html#contributing On Tue, Jul 21, 2015 at 2:11 PM, Torf <m...@torf.cc> wrote: > Hi everyone, > > Thanks for your input, Chris! So it seems to be indeed a problem with > LocalStorageDriver, not with my understanding :) > > What can I do to help getting this fixed in the official distribution? > I'd be happy to provide a patch, test cases, etc. > > > Regards, > Torf > > > On 21.07.2015 00:01, Chris Richards wrote: > > Torf, > > > > I stumbled across the same thing and modified my local repository copy. > I > > don't know if this was the correct way for libcloud proper, but it moved > me > > past the point where I was stuck. (There may be other places that need > it, > > but for the APIs I'm using they're sufficient.) You'll also need to do "$ > > pip3 install lockfile" to use the LocalStorage driver. > > > > diff -r a61cf0eec1a7 -r ad3deb3173ce libcloud/storage/drivers/local.py > > --- a/libcloud/storage/drivers/local.py Wed Jun 25 18:59:43 2014 -0500 > > +++ b/libcloud/storage/drivers/local.py Wed Jun 25 19:03:33 2014 -0500 > > @@ -406,7 +406,7 @@ > > > > path = self.get_object_cdn_url(obj) > > > > - with open(path) as obj_file: > > + with open(path, 'rb') as obj_file: > > for data in read_in_chunks(obj_file, chunk_size=chunk_size): > > yield data > > > > @@ -490,7 +490,7 @@ > > self._make_path(base_path) > > > > with LockLocalStorage(obj_path): > > - obj_file = open(obj_path, 'w') > > + obj_file = open(obj_path, 'wb') > > for data in iterator: > > obj_file.write(data) > > > > As for the upload, here's how I call it: > > > > obj = driver.upload_object_via_stream ( > > BytesIO (data), container, objname, > > extra={'content_type': 'application/octet-stream'}) > > > > Again, I don't know if this is libcloud proper or not; it works for me! > > (Ship it!) > > > > HTH, > > Chris > > > > On Mon, Jul 20, 2015 at 4:43 PM, Torf <m...@torf.cc> wrote: > > > >> Hi, > >> > >> I'm trying to store binary data in Python 3 to a ``LocalStorageDriver`` > >> instance. If I upload data via ``upload_object`` from a file on disk and > >> then use ``download_object_as_stream`` I get a stream of bytes, as > >> expected. However, uploading a stream of bytes manually using > >> ``upload_object_via_stream`` fails: > >> > >> > >>>>> container.upload_object_via_stream(io.BytesIO(b'foobar'), 'key') > >> Traceback (most recent call last): > >> File "<stdin>", line 1, in <module> > >> File > >> > >> > "/home/torf/projects/coba/venv/lib/python3.4/site-packages/libcloud/storage/base.py", > >> line 157, in upload_object_via_stream > >> iterator, self, object_name, extra=extra, **kwargs) > >> File > >> > >> > "/home/torf/projects/coba/venv/lib/python3.4/site-packages/libcloud/storage/drivers/local.py", > >> line 497, in upload_object_via_stream > >> obj_file.close() > >> File > >> > >> > "/home/torf/projects/coba/venv/lib/python3.4/site-packages/libcloud/storage/drivers/local.py", > >> line 69, in __exit__ > >> raise value > >> File > >> > >> > "/home/torf/projects/coba/venv/lib/python3.4/site-packages/libcloud/storage/drivers/local.py", > >> line 495, in upload_object_via_stream > >> obj_file.write(data) > >> TypeError: must be str, not bytes > >> > >> > >> The problem seems to be that ``LocalStorageDriver`` opens the target > >> file for writing in text mode ('w') instead of binary mode ('wb'). > >> > >> I couldn't find any information on whether keys and values are supposed > >> to be (Unicode) strings or bytes (or both), so I am not sure whether > >> this is a bug in ``LocalStorageDriver`` or a misunderstanding on my > side. > >> > >> This is on Python 3.4 and LibCloud 0.17.0. > >> > >> > >> Regards, > >> Torf > >> > > > >
