Mat Mannion created JCLOUDS-1264:
------------------------------------
Summary: Multipart SLO uploads fail if key contains multibyte
Unicode characters
Key: JCLOUDS-1264
URL: https://issues.apache.org/jira/browse/JCLOUDS-1264
Project: jclouds
Issue Type: Bug
Components: jclouds-blobstore
Affects Versions: 2.0.1
Environment: Devstack on Ubuntu 14.04
Reporter: Mat Mannion
If a large object is PUT to a key that contains multibyte Unicode characters,
the manifest upload fails because the {{Content-Length}} header sent is
incorrect.
A small project that reproduces the issue against the latest Devstack Swift can
be found here: https://github.com/UniversityofWarwick/jclouds-highbyte-slo-bug
In short:
{noformat}
>> "[{"path":"slo-test/Ni[0xc5][0x9f]an.rar/slo/1491214092.299000/38547913/0/00000001","etag":"58f06dd588d8ffb3beb46ada6309436b","size_bytes":33554432},{"path":"slo-test/Ni[0xc5][0x9f]an.rar/slo/1491214092.299000/38547913/0/00000002","etag":"2b4b81733d0a2e4abe89516639627408","size_bytes":4993481}]"
>> PUT
>> http://137.205.194.8:8080/v1/AUTH_c7a3e66567ec442080a360d6d23f2dbe/slo-test/Ni%C5%9Fan.rar?multipart-manifest=put
>> HTTP/1.1
>> Content-Length: 272
14657 ERROR org.jclouds.http.internal.JavaUrlHttpCommandExecutorService error
after writing 0/272 bytes to
http://137.205.194.8:8080/v1/AUTH_c7a3e66567ec442080a360d6d23f2dbe/slo-test/Ni%C5%9Fan.rar?multipart-manifest=put
java.io.IOException: too many bytes written
at
sun.net.www.protocol.http.HttpURLConnection$StreamingOutputStream.write(HttpURLConnection.java:3505)
at
com.google.common.io.CountingOutputStream.write(CountingOutputStream.java:53)
at com.google.common.io.ByteStreams.copy(ByteStreams.java:179)
at
org.jclouds.http.internal.JavaUrlHttpCommandExecutorService.writePayloadToConnection(JavaUrlHttpCommandExecutorService.java:298)
at
org.jclouds.http.internal.JavaUrlHttpCommandExecutorService.convert(JavaUrlHttpCommandExecutorService.java:171)
at
org.jclouds.http.internal.JavaUrlHttpCommandExecutorService.convert(JavaUrlHttpCommandExecutorService.java:65)
at
org.jclouds.http.internal.BaseHttpCommandExecutorService.invoke(BaseHttpCommandExecutorService.java:99)
at
org.jclouds.rest.internal.InvokeHttpMethod.invoke(InvokeHttpMethod.java:90)
at
org.jclouds.rest.internal.InvokeHttpMethod.apply(InvokeHttpMethod.java:73)
at
org.jclouds.rest.internal.InvokeHttpMethod.apply(InvokeHttpMethod.java:44)
at
org.jclouds.reflect.FunctionalReflection$FunctionalInvocationHandler.handleInvocation(FunctionalReflection.java:117)
at
com.google.common.reflect.AbstractInvocationHandler.invoke(AbstractInvocationHandler.java:87)
at com.sun.proxy.$Proxy77.replaceManifest(Unknown Source)
at
org.jclouds.openstack.swift.v1.blobstore.RegionScopedSwiftBlobStore.completeMultipartUpload(RegionScopedSwiftBlobStore.java:522)
at
uk.ac.warwick.slo.AbstractJCloudsSLOTest.putSLO(AbstractJCloudsSLOTest.java:70)
at
uk.ac.warwick.slo.AbstractJCloudsSLOTest.assertCanPutAndGetSLO(AbstractJCloudsSLOTest.java:74)
at
uk.ac.warwick.slo.AbstractJCloudsSLOTest.sloWithHighByteChars(AbstractJCloudsSLOTest.java:91)
at
uk.ac.warwick.slo.SwiftJCloudsSLOTest.sloWithHighByteChars(SwiftJCloudsSLOTest.java:7)
{noformat}
The cause of this is
{{org.jclouds.openstack.swift.v1.binders.BindManifestToJsonPayload:61}}, where
the ContentLength is set to the length of the JSON String in Java, which is not
the same as the String byte length. This is actually unnecessary, as
{{request.setPayload(String)}} creates a {{StringPayload}}, and the constructor
here sets the content length to the correct byte length - so I think this code
is left-over from a previous version where this isn't the case.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
