Maksim Hadalau created JCLOUDS-1633:
---------------------------------------
Summary: Google cloud storage handles content-encoding incorrectly
Key: JCLOUDS-1633
URL: https://issues.apache.org/jira/browse/JCLOUDS-1633
Project: jclouds
Issue Type: Bug
Components: jclouds-blobstore
Affects Versions: 2.5.0
Reporter: Maksim Hadalau
Steps to reproduce:
1. upload blob with Content-Encoding: gzip (content must be compressed on user
side)
2. verify that both getBlobMetadata and GCS UI shows correct object (stored
size == compressed size) and metadata returns Content-Encoding: gzip
3. Download blob using getBlob() and try to decompress payload
As result - decompression - failed because payload is not GZIP
Problem location
GoogleCloudStorageBlobStore.java
{code:java}
@Override
public Blob getBlob(String container, String name, GetOptions options) {
GoogleCloudStorageObject gcsObject = api.getObjectApi().getObject(container,
Strings2.urlEncode(name));
// here metadata contains a correct value for Content-Encoding: gzip
if (gcsObject == null) {
return null;
}
org.jclouds.http.options.GetOptions httpOptions =
blob2ObjectGetOptions.apply(options);
MutableBlobMetadata metadata = objectToBlobMetadata.apply(gcsObject);
Blob blob = new BlobImpl(metadata);
// TODO: Does getObject not get the payload?!
// Payload metadata returns Content-Encoding: null, due to google cloud
storage API, by default they do auto-decompression. If user want to download
compressed content - we should provide Accept-Encoding: gzip header. Which is
not possible right now (see GetOptions.class)
Payload payload = api.getObjectApi().download(container,
Strings2.urlEncode(name), httpOptions).getPayload();
// replacing payload metadata with object metadata is incorrect
// metadata contains Content-Encoding:gzip, but without providing
Accept-Encoding header payload metadata returned in decompressed form
(Content-Encoding: null)
payload.setContentMetadata(metadata.getContentMetadata()); // Doing this
first retains it on setPayload.
blob.setPayload(payload);
return blob;
} {code}
How to fix?
We should not replace payload metadata with object metadata(content encoding at
least) and extend GetOptions class with ability to provide
headers(Accept-Encoding in my case), or write a logic that will add
Accept-Encoding header if object metadata contains Content-Encoding header.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
