Hi Ignasi. In fact, in the "patch" I provided for the other issue (S3 upload performance), I already removed the hashing lines and simply took the provided MD5 hash from the ContentMetadata. And that worked. But I wasn't sure about the intention of the additional hashing, so I thought I better ask for more insight here ;).
So no changes were needed for [1] and [2]. If it makes sense, I can create a PR so someone can review the changes. Thanks, Veit Am 11.03.2015 um 22:23 schrieb Ignasi Barrera: > Looking at the code I would say it is an issue, but at some point that > should be intentional. The Apache HC driver explicitly excludes the > Content-MD5 header when converting the jclouds request to its request > object (see [1]). > > By looking at the other drivers (the default Java one and OkHttp), > both use that method to populate the Content-MD5 header (here you can > see what it does [2]) which does pretty much the same than what the > ApacheHC does *after*, by computing the hash again. > > Could you try removing the lines that rehash the stream and populate > the Content-MD5 header from the Apache HC driver, and add the > Content-MD5 header to the list of "allowed headers" in the first link? > I'm pretty sure that would work. And if that works, mind opening a > pull request with the fix? > > > HTH! > > I. > > > [1] > https://github.com/jclouds/jclouds/blob/master/drivers/apachehc/src/main/java/org/jclouds/http/apachehc/ApacheHCUtils.java#L155-L162 > [2] > https://github.com/jclouds/jclouds/blob/master/core/src/main/java/org/jclouds/io/ContentMetadataCodec.java#L98-L99 > > On 11 March 2015 at 17:22, Veit Guna <veit.g...@gmx.de> wrote: >> Hi. >> >> I have a problem with the ApacheHC driver using jclouds 1.8.1. I'm using it >> to upload files >> to an S3 bucket. For the payload I'm using an InputStream that is not >> resettable nor "rereadable". >> When I try to send the file via: >> >> PayloadBlobBuilder blobBuilder = blobStore.blobBuilder("test") >> .payload(inputStream) >> .contentDisposition("attachment; filename=test") >> .contentType("application/octet-stream") >> .contentMD5(hash) >> .contentLength(file.length() >> ); >> >> It dies with the exception below. >> >> A quick look at ApacheHCHttpCommandExecutorService shows: >> >> >> @Override >> protected HttpUriRequest convert(HttpRequest request) throws IOException { >> HttpUriRequest returnVal = >> apacheHCUtils.convertToApacheRequest(request); >> if (request.getPayload() != null && >> request.getPayload().getContentMetadata().getContentMD5() != null) { >> String md5 = >> base64().encode(ByteStreams2.hashAndClose(request.getPayload().openStream(), >> md5()).asBytes()); >> returnVal.addHeader("Content-MD5", md5); >> } >> return returnVal; >> } >> >> That means, that if an MD5 is specified, it reads the whole(!) stream to >> create his own MD5. Why is that? The "client" already >> supplied an MD5. So why hashing it again? >> >> Does that mean, using Md5 together with ApacheHC and non "rereadable" >> streams isn't working at all? Is that by design? >> The normal JavaUrlHttp driver works like a charm though. >> >> Maybe someone could shed some light on this :)? >> >> Thanks! >> Veit >> >> >> >> Exception in thread "main" org.jclouds.http.HttpResponseException: null >> connecting to PUT https://mybucket.s3-eu-west-1.amazonaws.com/test HTTP/1.1 >> at >> org.jclouds.http.internal.BaseHttpCommandExecutorService.invoke(BaseHttpCommandExecutorService.java:110) >> at >> org.jclouds.rest.internal.InvokeHttpMethod.invoke(InvokeHttpMethod.java:90) >> at >> org.jclouds.rest.internal.InvokeHttpMethod.apply(InvokeHttpMethod.java:73) >> at >> org.jclouds.rest.internal.InvokeHttpMethod.apply(InvokeHttpMethod.java:44) >> at >> org.jclouds.rest.internal.DelegatesToInvocationFunction.handle(DelegatesToInvocationFunction.java:156) >> at >> org.jclouds.rest.internal.DelegatesToInvocationFunction.invoke(DelegatesToInvocationFunction.java:123) >> at com.sun.proxy.$Proxy42.putObject(Unknown Source) >> at org.jclouds.s3.blobstore.S3BlobStore.putBlob(S3BlobStore.java:235) >> at >> org.jclouds.aws.s3.blobstore.AWSS3BlobStore.putBlob(AWSS3BlobStore.java:95) >> at org.jclouds.s3.blobstore.S3BlobStore.putBlob(S3BlobStore.java:213) >> at org.jclouds.test.S3UploadTest.performUpload(S3UploadTest.java:71) >> at org.jclouds.test.S3UploadTest.main(S3UploadTest.java:36) >> Caused by: org.apache.http.client.ClientProtocolException >> at >> org.apache.http.impl.client.AbstractHttpClient.doExecute(AbstractHttpClient.java:867) >> at >> org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:115) >> at >> org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:57) >> at >> org.jclouds.http.apachehc.ApacheHCHttpCommandExecutorService.executeRequest(ApacheHCHttpCommandExecutorService.java:109) >> at >> org.jclouds.http.apachehc.ApacheHCHttpCommandExecutorService.invoke(ApacheHCHttpCommandExecutorService.java:78) >> at >> org.jclouds.http.apachehc.ApacheHCHttpCommandExecutorService.invoke(ApacheHCHttpCommandExecutorService.java:52) >> at >> org.jclouds.http.internal.BaseHttpCommandExecutorService.invoke(BaseHttpCommandExecutorService.java:89) >> ... 11 more >> Caused by: org.apache.http.client.NonRepeatableRequestException: Cannot >> retry request with a non-repeatable request entity. The cause lists the >> reason the original request failed. >> at >> org.apache.http.impl.client.DefaultRequestDirector.tryExecute(DefaultRequestDirector.java:659) >> at >> org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:487) >> at >> org.apache.http.impl.client.AbstractHttpClient.doExecute(AbstractHttpClient.java:863) >> ... 17 more >> Caused by: java.io.IOException: Stream Closed >> at java.io.FileInputStream.readBytes(Native Method) >> at java.io.FileInputStream.read(FileInputStream.java:272) >> at >> org.apache.http.entity.InputStreamEntity.writeTo(InputStreamEntity.java:138) >> at >> org.apache.http.entity.HttpEntityWrapper.writeTo(HttpEntityWrapper.java:89) >> at >> org.apache.http.impl.client.EntityEnclosingRequestWrapper$EntityWrapper.writeTo(EntityEnclosingRequestWrapper.java:108) >> at >> org.apache.http.impl.entity.EntitySerializer.serialize(EntitySerializer.java:117) >> at >> org.apache.http.impl.AbstractHttpClientConnection.sendRequestEntity(AbstractHttpClientConnection.java:265) >> at >> org.apache.http.impl.conn.AbstractClientConnAdapter.sendRequestEntity(AbstractClientConnAdapter.java:231) >> at >> org.apache.http.protocol.HttpRequestExecutor.doSendRequest(HttpRequestExecutor.java:236) >> at >> org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:121) >> at >> org.apache.http.impl.client.DefaultRequestDirector.tryExecute(DefaultRequestDirector.java:685) >> ... 19 more >> >> >> >>
