Andrew Gaul created JCLOUDS-1368:
------------------------------------
Summary: GCS cannot upload multipart file larger than 1 GB
Key: JCLOUDS-1368
URL: https://issues.apache.org/jira/browse/JCLOUDS-1368
Project: jclouds
Issue Type: Bug
Components: jclouds-blobstore
Affects Versions: 2.0.3
Reporter: Andrew Gaul
Assignee: Andrew Gaul
When uploading a file:
{code:java}
long size = 33L * (32L * 1024L * 1024L);
ByteSource byteSource = TestUtils.randomByteSource().slice(0, size);
blobStore.createContainerInLocation(null, containerName);
Blob blob = blobStore.blobBuilder(blobName)
.payload(byteSource)
.contentLength(size)
.build();
blobStore.putBlob(containerName, blob, PutOptions.Builder.multipart());
{code}
I see an error:
{noformat}
org.jclouds.http.HttpResponseException: command: POST
https://www.googleapis.com/storage/v1/b/gaul-blobstore0/o/large-blob/compose
HTTP/1.1 failed with response: HTTP/1.1 400 Bad Request; content: [{
"error": {
"errors": [
{
"domain": "global",
"reason": "invalid",
"message": "The number of source components provided (33) exceeds the
maximum (32)"
}
],
"code": 400,
"message": "The number of source components provided (33) exceeds the maximum
(32)"
}
}
]
{noformat}
This is due to the wonky {{MultipartUploadSlicingAlgorithm}} slicing into (31)
32 MB parts then creating a remainder 64 MB part that
{{BaseBlobStore.putMultipartBlob}} does not account for. This interface seems
stupid and better to create (32) 33 MB parts. Originally reported at:
https://stackoverflow.com/questions/47048264/the-number-of-source-components-provided-78-exceeds-the-maximum-32
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
