A PR would be much appreciated :) It is also easier to discuss impl/design things there. Thanks!
On 11 March 2015 at 22:47, Veit Guna <veit.g...@gmx.de> wrote: > Hi Ignasi. > > In fact, in the "patch" I provided for the other issue (S3 upload > performance), I already > removed the hashing lines and simply took the provided MD5 hash from the > ContentMetadata. > And that worked. But I wasn't sure about the intention of the additional > hashing, > so I thought I better ask for more insight here ;). > > So no changes were needed for [1] and [2]. If it makes sense, I can > create a PR so someone > can review the changes. > > Thanks, > Veit > > > Am 11.03.2015 um 22:23 schrieb Ignasi Barrera: >> Looking at the code I would say it is an issue, but at some point that >> should be intentional. The Apache HC driver explicitly excludes the >> Content-MD5 header when converting the jclouds request to its request >> object (see [1]). >> >> By looking at the other drivers (the default Java one and OkHttp), >> both use that method to populate the Content-MD5 header (here you can >> see what it does [2]) which does pretty much the same than what the >> ApacheHC does *after*, by computing the hash again. >> >> Could you try removing the lines that rehash the stream and populate >> the Content-MD5 header from the Apache HC driver, and add the >> Content-MD5 header to the list of "allowed headers" in the first link? >> I'm pretty sure that would work. And if that works, mind opening a >> pull request with the fix? >> >> >> HTH! >> >> I. >> >> >> [1] >> https://github.com/jclouds/jclouds/blob/master/drivers/apachehc/src/main/java/org/jclouds/http/apachehc/ApacheHCUtils.java#L155-L162 >> [2] >> https://github.com/jclouds/jclouds/blob/master/core/src/main/java/org/jclouds/io/ContentMetadataCodec.java#L98-L99 >> >> On 11 March 2015 at 17:22, Veit Guna <veit.g...@gmx.de> wrote: >>> Hi. >>> >>> I have a problem with the ApacheHC driver using jclouds 1.8.1. I'm using it >>> to upload files >>> to an S3 bucket. For the payload I'm using an InputStream that is not >>> resettable nor "rereadable". >>> When I try to send the file via: >>> >>> PayloadBlobBuilder blobBuilder = blobStore.blobBuilder("test") >>> .payload(inputStream) >>> .contentDisposition("attachment; filename=test") >>> .contentType("application/octet-stream") >>> .contentMD5(hash) >>> .contentLength(file.length() >>> ); >>> >>> It dies with the exception below. >>> >>> A quick look at ApacheHCHttpCommandExecutorService shows: >>> >>> >>> @Override >>> protected HttpUriRequest convert(HttpRequest request) throws IOException >>> { >>> HttpUriRequest returnVal = >>> apacheHCUtils.convertToApacheRequest(request); >>> if (request.getPayload() != null && >>> request.getPayload().getContentMetadata().getContentMD5() != null) { >>> String md5 = >>> base64().encode(ByteStreams2.hashAndClose(request.getPayload().openStream(), >>> md5()).asBytes()); >>> returnVal.addHeader("Content-MD5", md5); >>> } >>> return returnVal; >>> } >>> >>> That means, that if an MD5 is specified, it reads the whole(!) stream to >>> create his own MD5. Why is that? The "client" already >>> supplied an MD5. So why hashing it again? >>> >>> Does that mean, using Md5 together with ApacheHC and non "rereadable" >>> streams isn't working at all? Is that by design? >>> The normal JavaUrlHttp driver works like a charm though. >>> >>> Maybe someone could shed some light on this :)? >>> >>> Thanks! >>> Veit >>> >>> >>> >>> Exception in thread "main" org.jclouds.http.HttpResponseException: null >>> connecting to PUT https://mybucket.s3-eu-west-1.amazonaws.com/test HTTP/1.1 >>> at >>> org.jclouds.http.internal.BaseHttpCommandExecutorService.invoke(BaseHttpCommandExecutorService.java:110) >>> at >>> org.jclouds.rest.internal.InvokeHttpMethod.invoke(InvokeHttpMethod.java:90) >>> at >>> org.jclouds.rest.internal.InvokeHttpMethod.apply(InvokeHttpMethod.java:73) >>> at >>> org.jclouds.rest.internal.InvokeHttpMethod.apply(InvokeHttpMethod.java:44) >>> at >>> org.jclouds.rest.internal.DelegatesToInvocationFunction.handle(DelegatesToInvocationFunction.java:156) >>> at >>> org.jclouds.rest.internal.DelegatesToInvocationFunction.invoke(DelegatesToInvocationFunction.java:123) >>> at com.sun.proxy.$Proxy42.putObject(Unknown Source) >>> at org.jclouds.s3.blobstore.S3BlobStore.putBlob(S3BlobStore.java:235) >>> at >>> org.jclouds.aws.s3.blobstore.AWSS3BlobStore.putBlob(AWSS3BlobStore.java:95) >>> at org.jclouds.s3.blobstore.S3BlobStore.putBlob(S3BlobStore.java:213) >>> at org.jclouds.test.S3UploadTest.performUpload(S3UploadTest.java:71) >>> at org.jclouds.test.S3UploadTest.main(S3UploadTest.java:36) >>> Caused by: org.apache.http.client.ClientProtocolException >>> at >>> org.apache.http.impl.client.AbstractHttpClient.doExecute(AbstractHttpClient.java:867) >>> at >>> org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:115) >>> at >>> org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:57) >>> at >>> org.jclouds.http.apachehc.ApacheHCHttpCommandExecutorService.executeRequest(ApacheHCHttpCommandExecutorService.java:109) >>> at >>> org.jclouds.http.apachehc.ApacheHCHttpCommandExecutorService.invoke(ApacheHCHttpCommandExecutorService.java:78) >>> at >>> org.jclouds.http.apachehc.ApacheHCHttpCommandExecutorService.invoke(ApacheHCHttpCommandExecutorService.java:52) >>> at >>> org.jclouds.http.internal.BaseHttpCommandExecutorService.invoke(BaseHttpCommandExecutorService.java:89) >>> ... 11 more >>> Caused by: org.apache.http.client.NonRepeatableRequestException: Cannot >>> retry request with a non-repeatable request entity. The cause lists the >>> reason the original request failed. >>> at >>> org.apache.http.impl.client.DefaultRequestDirector.tryExecute(DefaultRequestDirector.java:659) >>> at >>> org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:487) >>> at >>> org.apache.http.impl.client.AbstractHttpClient.doExecute(AbstractHttpClient.java:863) >>> ... 17 more >>> Caused by: java.io.IOException: Stream Closed >>> at java.io.FileInputStream.readBytes(Native Method) >>> at java.io.FileInputStream.read(FileInputStream.java:272) >>> at >>> org.apache.http.entity.InputStreamEntity.writeTo(InputStreamEntity.java:138) >>> at >>> org.apache.http.entity.HttpEntityWrapper.writeTo(HttpEntityWrapper.java:89) >>> at >>> org.apache.http.impl.client.EntityEnclosingRequestWrapper$EntityWrapper.writeTo(EntityEnclosingRequestWrapper.java:108) >>> at >>> org.apache.http.impl.entity.EntitySerializer.serialize(EntitySerializer.java:117) >>> at >>> org.apache.http.impl.AbstractHttpClientConnection.sendRequestEntity(AbstractHttpClientConnection.java:265) >>> at >>> org.apache.http.impl.conn.AbstractClientConnAdapter.sendRequestEntity(AbstractClientConnAdapter.java:231) >>> at >>> org.apache.http.protocol.HttpRequestExecutor.doSendRequest(HttpRequestExecutor.java:236) >>> at >>> org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:121) >>> at >>> org.apache.http.impl.client.DefaultRequestDirector.tryExecute(DefaultRequestDirector.java:685) >>> ... 19 more >>> >>> >>> >>> > >
