triplepoint opened a new issue #1529:
URL: https://github.com/apache/libcloud/issues/1529
## Summary
We're in the process of upgrading to the latest libcloud release (and Python
3) from a relatively old version 2.0.0 (and Python 2). During testing, we
discovered a change in behavior when uploading objects to AWS S3 when they had
a double leading slash in the object path, after the bucket name, for example
`s3://some-production-bucket//some-path/some-file.txt`. Previously, this
would've resulted in a folder-like object named `/` in the root of the path to
the file, where now it does not, and `some-path` is the top-most folder-like
object in the bucket.
While we realize this is bad practice, the leading object named `/` is valid
by AWS S3's documentation, and for legacy reasons we cannot immediately migrate
from this behavior. We need to be able to access these double-leading-slash S3
paths as before.
## Detailed Information
In trying to trace this problem, I discovered this call path:
- `libcloud/storage/drivers/s3.py` : `upload_object()`
- `libcloud/storage/drivers/s3.py` : `self._put_object()`
- `libcloud/storage/base.py`: `_upload_object()` invokes
`self.connection.request()`
- `libcloud/common/base.py`: `request()`
- `libcloud/common/base.py`: `self.morph_action_hook(action)`
It looks like the root of this behavior issue is in `morph_action_hook` on
line 655, which attempts to strip out any double slashes:
```
def morph_action_hook(self, action):
url = urlparse.urljoin(self.request_path.lstrip('/').rstrip('/') +
'/', action.lstrip('/'))
if not url.startswith('/'):
return '/' + url
else:
return url
```
and its this that prevents our valid use of the top-level folder-like object
named `/`.
I verified this by modifying `morph_action_hook()` to just `return action`,
which works in our case because request_path is empty. Uploading files with
this change in place returns us to the previous behavior. We've manually
patched libcloud in our production deploy, but we'd very much like for that to
be a temporary fix.
I don't entirely understand the goal of the current `morph_action_hook`
implementation (except as maybe an attempt to protect users from their own
mistakes?), but whatever it was, it was clearly added since our original
application of this library at version 2.0.0.
Is there something I'm missing? Is there a provision for disabling this
corrective code that I'm not seeing?
Thanks in advance for any help you can give.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
