[
https://issues.apache.org/jira/browse/JCLOUDS-1368?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16319146#comment-16319146
]
ASF subversion and git services commented on JCLOUDS-1368:
----------------------------------------------------------
Commit 382c9201e37eb5ddfe5e9ab06a6ec73e1fb2cba5 in jclouds's branch
refs/heads/2.0.x from [~gaul]
[ https://git-wip-us.apache.org/repos/asf?p=jclouds.git;h=382c920 ]
JCLOUDS-1368: Correct use of slicing algorithm
MultipartUploadSlicingAlgorithm creates multiple equal-sized parts and
a remaining amount. BaseBlobStore.putMultipartBlob used this
interface incorrectly, which could create more parts than intended
since the remaining size could be larger than the part size. This
manifested with Google Cloud Storage which only allows 32 parts.
> GCS cannot upload multipart file larger than 1 GB
> -------------------------------------------------
>
> Key: JCLOUDS-1368
> URL: https://issues.apache.org/jira/browse/JCLOUDS-1368
> Project: jclouds
> Issue Type: Bug
> Components: jclouds-blobstore
> Affects Versions: 2.0.3
> Reporter: Andrew Gaul
> Assignee: Andrew Gaul
> Labels: google-cloud-storage
>
> When uploading a file:
> {code:java}
> long size = 33L * (32L * 1024L * 1024L);
> ByteSource byteSource = TestUtils.randomByteSource().slice(0, size);
> blobStore.createContainerInLocation(null, containerName);
> Blob blob = blobStore.blobBuilder(blobName)
> .payload(byteSource)
> .contentLength(size)
> .build();
> blobStore.putBlob(containerName, blob, PutOptions.Builder.multipart());
> {code}
> I see an error:
> {noformat}
> org.jclouds.http.HttpResponseException: command: POST
> https://www.googleapis.com/storage/v1/b/gaul-blobstore0/o/large-blob/compose
> HTTP/1.1 failed with response: HTTP/1.1 400 Bad Request; content: [{
> "error": {
> "errors": [
> {
> "domain": "global",
> "reason": "invalid",
> "message": "The number of source components provided (33) exceeds the
> maximum (32)"
> }
> ],
> "code": 400,
> "message": "The number of source components provided (33) exceeds the
> maximum (32)"
> }
> }
> ]
> {noformat}
> This is due to the wonky {{MultipartUploadSlicingAlgorithm}} slicing into
> (31) 32 MB parts then creating a remainder 64 MB part that
> {{BaseBlobStore.putMultipartBlob}} does not account for. This interface
> seems stupid and better to create (32) 33 MB parts. Originally reported at:
> https://stackoverflow.com/questions/47048264/the-number-of-source-components-provided-78-exceeds-the-maximum-32
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
