Hi
I have written a sample code for multipart upload using jClouds-2.0
Properties overrides = new Properties();
BlobStoreContext context = ContextBuilder.newBuilder("openstack-swift")
.endpoint("http://x.xxx.xx.xx:5000/v2.0";)
.credentials("xx:xx", "xx")
.overrides(overrides)
.modules(modules)
.buildView(BlobStoreContext.class);
BlobStore blobStore = context.getBlobStore();
blobStore.createContainerInLocation(null, CONTAINER_NAME);
Path path = Paths.get("test2");
File f = new File("test2");
byte []byteArray = Files.readAllBytes(path);
Payload payload = newByteSourcePayload(wrap(byteArray));
PutOptions opt = new PutOptions();
opt.multipart();
Blob blob = blobStore.blobBuilder(OBJECT_NAME)
.payload(payload).contentLength(f.length())
.build();
String etag = blobStore.putBlob(CONTAINER_NAME, blob, opt);
test2 is the file I am trying to upload which is of size 36MB and I am getting
the following exception
10:21:52.355 [main] DEBUG o.j.h.i.JavaUrlHttpCommandExecutorService - Sending
request 1344471693: PUT
http://x.x.x.x:8091/v1/AUTH_0909ac10e7024847b1a9fe9787c7de8f/arctestMP HTTP/1.1
10:21:52.356 [main] DEBUG jclouds.headers - >> PUT
http://x.x.x.x:8091/v1/AUTH_0909ac10e7024847b1a9fe9787c7de8f/arctestMP HTTP/1.1
10:21:52.356 [main] DEBUG jclouds.headers - >> Accept: application/json
10:21:52.357 [main] DEBUG jclouds.headers - >> X-Auth-Token:
fd72b74db90c46cabcca3f317d5a09d4
10:21:53.129 [main] DEBUG o.j.h.i.JavaUrlHttpCommandExecutorService - Receiving
response 1344471693: HTTP/1.1 201 Created
10:21:53.129 [main] DEBUG jclouds.headers - << HTTP/1.1 201 Created
10:21:53.129 [main] DEBUG jclouds.headers - << Date: Fri, 03 Feb 2017 04:51:53
GMT
10:21:53.129 [main] DEBUG jclouds.headers - << X-Trans-Id:
tx83ba6249347c43c99bb41-0058940c68
10:21:53.129 [main] DEBUG jclouds.headers - << Connection: keep-alive
10:21:53.129 [main] DEBUG jclouds.headers - << Content-Type: text/html;
charset=UTF-8
10:21:53.129 [main] DEBUG jclouds.headers - << Content-Length: 0
-------> Container Creation Successful
10:21:53.373 [user thread 1] DEBUG o.j.rest.internal.InvokeHttpMethod - >>
invoking object:put
10:21:53.373 [user thread 0] DEBUG o.j.rest.internal.InvokeHttpMethod - >>
invoking object:put
10:21:53.374 [user thread 1] DEBUG o.j.h.i.JavaUrlHttpCommandExecutorService -
Sending request 823625484: PUT
http://x.x.x.x:8091/v1/AUTH_0909ac10e7024847b1a9fe9787c7de8f/arctestMP/arc/slo/1486097513.327000/0/33554432/00000001
HTTP/1.1
10:21:53.376 [user thread 0] DEBUG o.j.h.i.JavaUrlHttpCommandExecutorService -
Sending request -1220101806: PUT
http://x.x.x.x:8091/v1/AUTH_0909ac10e7024847b1a9fe9787c7de8f/arctestMP/arc/slo/1486097513.327000/0/33554432/00000000
HTTP/1.1
10:21:53.396 [user thread 1] DEBUG org.jclouds.http.internal.HttpWire - over
limit 3145728/262144: wrote temp file
10:21:53.552 [user thread 0] DEBUG org.jclouds.http.internal.HttpWire - over
limit 33554432/262144: wrote temp fileException in thread "main"
com.google.common.util.concurrent.ExecutionError: java.lang.OutOfMemoryError:
Java heap space
at
com.google.common.util.concurrent.Futures.wrapAndThrowUnchecked(Futures.java:1380)
at com.google.common.util.concurrent.Futures.getUnchecked(Futures.java:1373)
at
org.jclouds.openstack.swift.v1.blobstore.RegionScopedSwiftBlobStore.putMultipartBlob(RegionScopedSwiftBlobStore.java:650)
at
org.jclouds.openstack.swift.v1.blobstore.RegionScopedSwiftBlobStore.putMultipartBlob(RegionScopedSwiftBlobStore.java:628)
at
org.jclouds.openstack.swift.v1.blobstore.RegionScopedSwiftBlobStore.putBlob(RegionScopedSwiftBlobStore.java:274)
at jclouds20.App.main(App.java:83)
Caused by: java.lang.OutOfMemoryError: Java heap space
at java.lang.StringBuilder.ensureCapacityImpl(StringBuilder.java:342)
at java.lang.StringBuilder.append(StringBuilder.java:208)
at org.jclouds.logging.internal.Wire.wire(Wire.java:68)
at org.jclouds.logging.internal.Wire.copy(Wire.java:99)
at org.jclouds.logging.internal.Wire.output(Wire.java:176)
at org.jclouds.logging.internal.Wire.output(Wire.java:143)
at org.jclouds.http.HttpUtils.wirePayloadIfEnabled(HttpUtils.java:296)
at
org.jclouds.http.internal.BaseHttpCommandExecutorService.invoke(BaseHttpCommandExecutorService.java:97)
at
org.jclouds.rest.internal.InvokeHttpMethod.invoke(InvokeHttpMethod.java:90)
at
org.jclouds.rest.internal.InvokeHttpMethod.apply(InvokeHttpMethod.java:73)
at
org.jclouds.rest.internal.InvokeHttpMethod.apply(InvokeHttpMethod.java:44)
at
org.jclouds.reflect.FunctionalReflection$FunctionalInvocationHandler.handleInvocation(FunctionalReflection.java:117)
at
com.google.common.reflect.AbstractInvocationHandler.invoke(AbstractInvocationHandler.java:87)
at com.sun.proxy.$Proxy68.put(Unknown Source)
at
org.jclouds.openstack.swift.v1.blobstore.RegionScopedSwiftBlobStore.uploadMultipartPart(RegionScopedSwiftBlobStore.java:529)
at
org.jclouds.openstack.swift.v1.blobstore.RegionScopedSwiftBlobStore$BlobUploader.call(RegionScopedSwiftBlobStore.java:666)
at
org.jclouds.openstack.swift.v1.blobstore.RegionScopedSwiftBlobStore$BlobUploader.call(RegionScopedSwiftBlobStore.java:653)
at java.util.concurrent.FutureTask.run(FutureTask.java:274)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1157)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:627)
at java.lang.Thread.run(Thread.java:798)
at java.lang.Thread.getStackTraceImpl(Native Method)
at java.lang.Thread.getStackTrace(Thread.java:1133)
at
org.jclouds.openstack.swift.v1.blobstore.RegionScopedSwiftBlobStore.putMultipartBlob(RegionScopedSwiftBlobStore.java:647)
... 3 more
Feb 03, 2017 10:22:03 AM
com.google.common.util.concurrent.Futures$CombinedFuture setExceptionAndMaybeLog
SEVERE: input future failed.
java.lang.OutOfMemoryError: Java heap space
at java.lang.StringBuilder.ensureCapacityImpl(StringBuilder.java:342)
at java.lang.StringBuilder.append(StringBuilder.java:208)
at org.jclouds.logging.internal.Wire.wire(Wire.java:68)
at org.jclouds.logging.internal.Wire.copy(Wire.java:99)
at org.jclouds.logging.internal.Wire.output(Wire.java:176)
at org.jclouds.logging.internal.Wire.output(Wire.java:143)
at org.jclouds.http.HttpUtils.wirePayloadIfEnabled(HttpUtils.java:296)
at
org.jclouds.http.internal.BaseHttpCommandExecutorService.invoke(BaseHttpCommandExecutorService.java:97)
at
org.jclouds.rest.internal.InvokeHttpMethod.invoke(InvokeHttpMethod.java:90)
at
org.jclouds.rest.internal.InvokeHttpMethod.apply(InvokeHttpMethod.java:73)
at
org.jclouds.rest.internal.InvokeHttpMethod.apply(InvokeHttpMethod.java:44)
at
org.jclouds.reflect.FunctionalReflection$FunctionalInvocationHandler.handleInvocation(FunctionalReflection.java:117)
at
com.google.common.reflect.AbstractInvocationHandler.invoke(AbstractInvocationHandler.java:87)
at com.sun.proxy.$Proxy68.put(Unknown Source)
at
org.jclouds.openstack.swift.v1.blobstore.RegionScopedSwiftBlobStore.uploadMultipartPart(RegionScopedSwiftBlobStore.java:529)
at
org.jclouds.openstack.swift.v1.blobstore.RegionScopedSwiftBlobStore$BlobUploader.call(RegionScopedSwiftBlobStore.java:666)
at
org.jclouds.openstack.swift.v1.blobstore.RegionScopedSwiftBlobStore$BlobUploader.call(RegionScopedSwiftBlobStore.java:653)
at java.util.concurrent.FutureTask.run(FutureTask.java:274)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1157)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:627)
at java.lang.Thread.run(Thread.java:798)
at java.lang.Thread.getStackTraceImpl(Native Method)
at java.lang.Thread.getStackTrace(Thread.java:1133)
at
org.jclouds.openstack.swift.v1.blobstore.RegionScopedSwiftBlobStore.putMultipartBlob(RegionScopedSwiftBlobStore.java:647)
at
org.jclouds.openstack.swift.v1.blobstore.RegionScopedSwiftBlobStore.putMultipartBlob(RegionScopedSwiftBlobStore.java:628)
at
org.jclouds.openstack.swift.v1.blobstore.RegionScopedSwiftBlobStore.putBlob(RegionScopedSwiftBlobStore.java:274)
at jclouds20.App.main(App.java:83)
Should putBlob with multipart enabled in PutOptions not be used for uploading
files greater than 32MB from jClouds-2.0 ?
Is it mandatory to use Swift API's initiate, upload and complete multipart from
jClouds-2.0 ?
RegardsArchana
