[GitHub] libcloud pull request: Reset file's current position in read_in_ch...

[kun--hust]

GitHub user kun--hust opened a pull request:

    https://github.com/apache/libcloud/pull/509

    Reset file's current position in read_in_chunks()

    If a cloud storage driver's parameter  supports_chunked_encoding is False,  
the function read_in_chunks will be called twice during the 
upload_object_via_stream progress. 
    
    **read_in_chunks()  be called in _upload_object() at first (line. 629  
libcloud.storage.base)**
    ```ruby
    ……
    if iterator:
                if self.supports_chunked_encoding:
                    headers['Transfer-Encoding'] = 'chunked'
                    upload_func_kwargs['chunked'] = True
                else:
                    # Chunked transfer encoding is not supported. Need to buffer
                    # all the data in memory so we can determine file size.
                    iterator = libcloud.utils.files.read_in_chunks(
                        iterator=iterator)
                    data = 
libcloud.utils.files.exhaust_iterator(iterator=iterator)
    
                    file_size = len(data)
                    upload_func_kwargs['data'] = data
            else:
                file_size = os.path.getsize(file_path)
                upload_func_kwargs['chunked'] = False
    ……
    ```
    
    **Then, read_in_chunks()  be called in _stream_data() (line. 734  
libcloud.storage.base)**
    ```ruby
    ……
    generator = libcloud.utils.files.read_in_chunks(iterator, chunk_size)
    
    bytes_transferred = 0
            try:
                chunk = next(generator)
            except StopIteration:
    ……
    ```
    
    But in the second time, the current position of the file to be uploaded is 
the end, so the StopIteration exception will be raised and upload a empty file. 
So, I make this change in read_in_chunks.

You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/kun--hust/libcloud patch-2

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/libcloud/pull/509.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #509
    
----
commit 011e2351c0ab387eb296f03e3a4706a4b9ebace5
Author: kun--hust <yichuan0...@hust.edu.cn>
Date:   2015-04-23T06:29:15Z

    Reset file's current position in read_in_chunks()
    
    If a cloud storage driver's parameter  supports_chunked_encoding is False,  
the function read_in_chunks will be called twice during the 
upload_object_via_stream progress. But in the second time, the current position 
of the file to be uploaded is the end, so the StopIteration exception will be 
raised and upload a empty file.

----


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---