[jira] [Updated] (JCLOUDS-1520) JClouds is not using the JDK's KeepAliveCache when UntrustedSSLContextSupplier is used
[
https://issues.apache.org/jira/browse/JCLOUDS-1520?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Roded Bahat updated JCLOUDS-1520:
-
Description:
It seems like the fact that {{UntrustedSSLContextSupplier}} returns a new
{{SSLContext}} on every {{get()}} call causes a consistent cache miss on the
JVM's {{sun.net.www.http.KeepAliveCache}} which causes JClouds to not reuse
existing TLS connections even though it could.
The cache miss happens at {{sun.net.www.protocol.https.HttpsClient}} line 329
(openjdk version "1.8.0_222"):
{noformat}
/* see if one's already around */
ret = (HttpsClient) kac.get(url, sf);
{noformat}
To reproduce, consider the following main:
{noformat}
public static void main(String[] args) {
Properties overrides = new Properties();
overrides.setProperty(org.jclouds.Constants.PROPERTY_TRUST_ALL_CERTS,
"true");
BlobStoreContext blobStoreContext = ContextBuilder.newBuilder("aws-s3")
.endpoint("https://s3.amazonaws.com;)
.credentials("...", "...")
.overrides(overrides)
.buildView(BlobStoreContext.class);
BlobStore blobStore = blobStoreContext.getBlobStore();
blobStore.getBlob("roded-data", "blobname");
blobStore.getBlob("roded-data", "blobname");
blobStore.getBlob("roded-data", "blobname");
blobStoreContext.close();
System.exit(0);
}
{noformat}
If run using a JUL logging.properties with the following logger set to FINEST:
{noformat}
sun.net.www.protocol.http.level=FINEST
{noformat}
The following log is produced:
{noformat}
2019-10-10 18:15:19.668 FINE[org.jclouds.rest.internal.InvokeHttpMethod ]
>> invoking GetBucketLocation
2019-10-10 18:15:19.733 FINE
[org.jclouds.http.internal.JavaUrlHttpCommandExecutorService ] Sending request
-1721710788: GET https://s3.amazonaws.com/roded-data?location HTTP/1.1
2019-10-10 18:15:19.893 FINEST [sun.net.www.protocol.http.HttpURLConnection ]
Looking for HttpClient for URL https://s3.amazonaws.com/roded-data?location and
proxy value of DIRECT
2019-10-10 18:15:19.893 FINEST [sun.net.www.protocol.http.HttpURLConnection ]
Creating new HttpsClient with url:https://s3.amazonaws.com/roded-data?location
and proxy:DIRECT with connect timeout:6
2019-10-10 18:15:20.837 FINE[sun.net.www.protocol.http.HttpURLConnection ]
sun.net.www.MessageHeader@537b32ef8 pairs: {GET /roded-data?location HTTP/1.1:
null}{x-amz-content-sha256:
e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855}{X-Amz-Date:
20191010T151519Z}{Authorization: AWS4-HMAC-SHA256
Credential=AKIAJO5RLGWKFW5ASG3A/20191010/us-east-1/s3/aws4_request,
SignedHeaders=host;x-amz-content-sha256;x-amz-date,
Signature=896e11ddd9efac465b6ff2506d1688d454a50b3f73ac68d557ad036b1826e591}{User-Agent:
jclouds/2019.224.2 java/1.8.0_222}{Host: s3.amazonaws.com}{Accept: text/html,
image/gif, image/jpeg, *; q=.2, */*; q=.2}{Connection: keep-alive}
2019-10-10 18:15:21.169 FINE[sun.net.www.protocol.http.HttpURLConnection ]
sun.net.www.MessageHeader@6f815e7f7 pairs: {null: HTTP/1.1 200 OK}{x-amz-id-2:
1VVlx4h/fBOFe3n/7IxvpWN0RoVcE2rSpnnxMjvAQ93lJ6tHJAS+3IlXAx++/ZMEblp7kjJT4eQ=}{x-amz-request-id:
AE0779131201B495}{Date: Thu, 10 Oct 2019 15:15:21 GMT}{Content-Type:
application/xml}{Transfer-Encoding: chunked}{Server: AmazonS3}
2019-10-10 18:15:21.185 FINE
[org.jclouds.http.internal.JavaUrlHttpCommandExecutorService ] Receiving
response -1721710788: HTTP/1.1 200 OK
2019-10-10 18:15:21.500 FINE[org.jclouds.rest.internal.InvokeHttpMethod ]
>> invoking GetObject
2019-10-10 18:15:21.514 FINE
[org.jclouds.http.internal.JavaUrlHttpCommandExecutorService ] Sending request
-1332190413: GET https://roded-data.s3-eu-central-1.amazonaws.com/blobname
HTTP/1.1
2019-10-10 18:15:21.517 FINEST [sun.net.www.protocol.http.HttpURLConnection ]
Looking for HttpClient for URL
https://roded-data.s3-eu-central-1.amazonaws.com/blobname and proxy value of
DIRECT
2019-10-10 18:15:21.519 FINEST [sun.net.www.protocol.http.HttpURLConnection ]
Creating new HttpsClient with
url:https://roded-data.s3-eu-central-1.amazonaws.com/blobname and proxy:DIRECT
with connect timeout:6
2019-10-10 18:15:22.319 FINE[sun.net.www.protocol.http.HttpURLConnection ]
sun.net.www.MessageHeader@6a933be28 pairs: {GET /blobname HTTP/1.1:
null}{x-amz-content-sha256:
e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855}{X-Amz-Date:
20191010T151519Z}{Authorization: AWS4-HMAC-SHA256
Credential=AKIAJO5RLGWKFW5ASG3A/20191010/eu-central-1/s3/aws4_request,
SignedHeaders=host;x-amz-content-sha256;x-amz-date,
Signature=52fadcda579bb56e62b9b0489d7a0a90080103213021b0ea26d63f9e9620f4cc}{User-Agent:
jclouds/2019.224.2 java/1.8.0_222}{Host:
roded-data.s3-eu-central-1.ama
[jira] [Updated] (JCLOUDS-1520) JClouds is not using the JDK's KeepAliveCache when UntrustedSSLContextSupplier is used
[
https://issues.apache.org/jira/browse/JCLOUDS-1520?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Roded Bahat updated JCLOUDS-1520:
-
Description:
It seems like the fact that {{UntrustedSSLContextSupplier}} returns a new
{{SSLContext}} on every {{get()}} call causes a consistent cache miss on the
JVM's {{sun.net.www.http.KeepAliveCache}} which causes JClouds to not reuse
existing TLS connections even though it could.
The cache miss happens at {{sun.net.www.protocol.https.HttpsClient}} line 329
(openjdk version "1.8.0_222"):
{noformat}
/* see if one's already around */
ret = (HttpsClient) kac.get(url, sf);
{noformat}
To reproduce, consider the following main:
{noformat}
public static void main(String[] args) {
Properties overrides = new Properties();
overrides.setProperty(org.jclouds.Constants.PROPERTY_TRUST_ALL_CERTS,
"true");
BlobStoreContext blobStoreContext = ContextBuilder.newBuilder("aws-s3")
.endpoint("https://s3.amazonaws.com;)
.credentials("...", "...")
.overrides(overrides)
.buildView(BlobStoreContext.class);
BlobStore blobStore = blobStoreContext.getBlobStore();
blobStore.getBlob("roded-data", "blobname");
blobStore.getBlob("roded-data", "blobname");
blobStore.getBlob("roded-data", "blobname");
blobStoreContext.close();
System.exit(0);
}
{noformat}
If run using a JUL logging.properties with the following logger set to FINEST:
{noformat}
sun.net.www.protocol.http.level=FINEST
{noformat}
The following log is produced:
{noformat}
2019-10-10 18:15:19.668 FINE[org.jclouds.rest.internal.InvokeHttpMethod ]
>> invoking GetBucketLocation
2019-10-10 18:15:19.733 FINE
[org.jclouds.http.internal.JavaUrlHttpCommandExecutorService ] Sending request
-1721710788: GET https://s3.amazonaws.com/roded-data?location HTTP/1.1
2019-10-10 18:15:19.893 FINEST [sun.net.www.protocol.http.HttpURLConnection ]
Looking for HttpClient for URL https://s3.amazonaws.com/roded-data?location and
proxy value of DIRECT
2019-10-10 18:15:19.893 FINEST [sun.net.www.protocol.http.HttpURLConnection ]
Creating new HttpsClient with url:https://s3.amazonaws.com/roded-data?location
and proxy:DIRECT with connect timeout:6
2019-10-10 18:15:20.837 FINE[sun.net.www.protocol.http.HttpURLConnection ]
sun.net.www.MessageHeader@537b32ef8 pairs: {GET /roded-data?location HTTP/1.1:
null}{x-amz-content-sha256:
e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855}{X-Amz-Date:
20191010T151519Z}{Authorization: AWS4-HMAC-SHA256
Credential=AKIAJO5RLGWKFW5ASG3A/20191010/us-east-1/s3/aws4_request,
SignedHeaders=host;x-amz-content-sha256;x-amz-date,
Signature=896e11ddd9efac465b6ff2506d1688d454a50b3f73ac68d557ad036b1826e591}{User-Agent:
jclouds/2019.224.2 java/1.8.0_222}{Host: s3.amazonaws.com}{Accept: text/html,
image/gif, image/jpeg, *; q=.2, */*; q=.2}{Connection: keep-alive}
2019-10-10 18:15:21.169 FINE[sun.net.www.protocol.http.HttpURLConnection ]
sun.net.www.MessageHeader@6f815e7f7 pairs: {null: HTTP/1.1 200 OK}{x-amz-id-2:
1VVlx4h/fBOFe3n/7IxvpWN0RoVcE2rSpnnxMjvAQ93lJ6tHJAS+3IlXAx++/ZMEblp7kjJT4eQ=}{x-amz-request-id:
AE0779131201B495}{Date: Thu, 10 Oct 2019 15:15:21 GMT}{Content-Type:
application/xml}{Transfer-Encoding: chunked}{Server: AmazonS3}
2019-10-10 18:15:21.185 FINE
[org.jclouds.http.internal.JavaUrlHttpCommandExecutorService ] Receiving
response -1721710788: HTTP/1.1 200 OK
2019-10-10 18:15:21.500 FINE[org.jclouds.rest.internal.InvokeHttpMethod ]
>> invoking GetObject
2019-10-10 18:15:21.514 FINE
[org.jclouds.http.internal.JavaUrlHttpCommandExecutorService ] Sending request
-1332190413: GET https://roded-data.s3-eu-central-1.amazonaws.com/blobname
HTTP/1.1
2019-10-10 18:15:21.517 FINEST [sun.net.www.protocol.http.HttpURLConnection ]
Looking for HttpClient for URL
https://roded-data.s3-eu-central-1.amazonaws.com/blobname and proxy value of
DIRECT
2019-10-10 18:15:21.519 FINEST [sun.net.www.protocol.http.HttpURLConnection ]
Creating new HttpsClient with
url:https://roded-data.s3-eu-central-1.amazonaws.com/blobname and proxy:DIRECT
with connect timeout:6
2019-10-10 18:15:22.319 FINE[sun.net.www.protocol.http.HttpURLConnection ]
sun.net.www.MessageHeader@6a933be28 pairs: {GET /blobname HTTP/1.1:
null}{x-amz-content-sha256:
e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855}{X-Amz-Date:
20191010T151519Z}{Authorization: AWS4-HMAC-SHA256
Credential=AKIAJO5RLGWKFW5ASG3A/20191010/eu-central-1/s3/aws4_request,
SignedHeaders=host;x-amz-content-sha256;x-amz-date,
Signature=52fadcda579bb56e62b9b0489d7a0a90080103213021b0ea26d63f9e9620f4cc}{User-Agent:
jclouds/2019.224.2 java/1.8.0_222}{Host:
roded-data.s3-eu-central-1.ama
[jira] [Updated] (JCLOUDS-1520) JClouds is not using the JDK's KeepAliveCache when UntrustedSSLContextSupplier is used
[
https://issues.apache.org/jira/browse/JCLOUDS-1520?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Roded Bahat updated JCLOUDS-1520:
-
Description:
It seems like the fact that {{UntrustedSSLContextSupplier}} returns a new
{{SSLContext}} on every {{get()}} call causes a consistent cache miss on the
JVM's {{sun.net.www.http.KeepAliveCache}} which causes JClouds to not reusing
existing TLS connections even though it could.
The cache miss happens at {{sun.net.www.protocol.https.HttpsClient}} line 329
(openjdk version "1.8.0_222"):
{noformat}
/* see if one's already around */
ret = (HttpsClient) kac.get(url, sf);
{noformat}
To reproduce, consider the following main:
{noformat}
public static void main(String[] args) {
Properties overrides = new Properties();
overrides.setProperty(org.jclouds.Constants.PROPERTY_TRUST_ALL_CERTS,
"true");
BlobStoreContext blobStoreContext = ContextBuilder.newBuilder("aws-s3")
.endpoint("https://s3.amazonaws.com;)
.credentials("...", "...")
.overrides(overrides)
.buildView(BlobStoreContext.class);
BlobStore blobStore = blobStoreContext.getBlobStore();
blobStore.getBlob("roded-data", "blobname");
blobStore.getBlob("roded-data", "blobname");
blobStore.getBlob("roded-data", "blobname");
blobStoreContext.close();
System.exit(0);
}
{noformat}
If run using a JUL logging.properties with the following logger set to FINEST:
{noformat}
sun.net.www.protocol.http.level=FINEST
{noformat}
The following log is produced:
{noformat}
2019-10-10 18:15:19.668 FINE[org.jclouds.rest.internal.InvokeHttpMethod ]
>> invoking GetBucketLocation
2019-10-10 18:15:19.733 FINE
[org.jclouds.http.internal.JavaUrlHttpCommandExecutorService ] Sending request
-1721710788: GET https://s3.amazonaws.com/roded-data?location HTTP/1.1
2019-10-10 18:15:19.893 FINEST [sun.net.www.protocol.http.HttpURLConnection ]
Looking for HttpClient for URL https://s3.amazonaws.com/roded-data?location and
proxy value of DIRECT
2019-10-10 18:15:19.893 FINEST [sun.net.www.protocol.http.HttpURLConnection ]
Creating new HttpsClient with url:https://s3.amazonaws.com/roded-data?location
and proxy:DIRECT with connect timeout:6
2019-10-10 18:15:20.837 FINE[sun.net.www.protocol.http.HttpURLConnection ]
sun.net.www.MessageHeader@537b32ef8 pairs: {GET /roded-data?location HTTP/1.1:
null}{x-amz-content-sha256:
e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855}{X-Amz-Date:
20191010T151519Z}{Authorization: AWS4-HMAC-SHA256
Credential=AKIAJO5RLGWKFW5ASG3A/20191010/us-east-1/s3/aws4_request,
SignedHeaders=host;x-amz-content-sha256;x-amz-date,
Signature=896e11ddd9efac465b6ff2506d1688d454a50b3f73ac68d557ad036b1826e591}{User-Agent:
jclouds/2019.224.2 java/1.8.0_222}{Host: s3.amazonaws.com}{Accept: text/html,
image/gif, image/jpeg, *; q=.2, */*; q=.2}{Connection: keep-alive}
2019-10-10 18:15:21.169 FINE[sun.net.www.protocol.http.HttpURLConnection ]
sun.net.www.MessageHeader@6f815e7f7 pairs: {null: HTTP/1.1 200 OK}{x-amz-id-2:
1VVlx4h/fBOFe3n/7IxvpWN0RoVcE2rSpnnxMjvAQ93lJ6tHJAS+3IlXAx++/ZMEblp7kjJT4eQ=}{x-amz-request-id:
AE0779131201B495}{Date: Thu, 10 Oct 2019 15:15:21 GMT}{Content-Type:
application/xml}{Transfer-Encoding: chunked}{Server: AmazonS3}
2019-10-10 18:15:21.185 FINE
[org.jclouds.http.internal.JavaUrlHttpCommandExecutorService ] Receiving
response -1721710788: HTTP/1.1 200 OK
2019-10-10 18:15:21.500 FINE[org.jclouds.rest.internal.InvokeHttpMethod ]
>> invoking GetObject
2019-10-10 18:15:21.514 FINE
[org.jclouds.http.internal.JavaUrlHttpCommandExecutorService ] Sending request
-1332190413: GET https://roded-data.s3-eu-central-1.amazonaws.com/blobname
HTTP/1.1
2019-10-10 18:15:21.517 FINEST [sun.net.www.protocol.http.HttpURLConnection ]
Looking for HttpClient for URL
https://roded-data.s3-eu-central-1.amazonaws.com/blobname and proxy value of
DIRECT
2019-10-10 18:15:21.519 FINEST [sun.net.www.protocol.http.HttpURLConnection ]
Creating new HttpsClient with
url:https://roded-data.s3-eu-central-1.amazonaws.com/blobname and proxy:DIRECT
with connect timeout:6
2019-10-10 18:15:22.319 FINE[sun.net.www.protocol.http.HttpURLConnection ]
sun.net.www.MessageHeader@6a933be28 pairs: {GET /blobname HTTP/1.1:
null}{x-amz-content-sha256:
e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855}{X-Amz-Date:
20191010T151519Z}{Authorization: AWS4-HMAC-SHA256
Credential=AKIAJO5RLGWKFW5ASG3A/20191010/eu-central-1/s3/aws4_request,
SignedHeaders=host;x-amz-content-sha256;x-amz-date,
Signature=52fadcda579bb56e62b9b0489d7a0a90080103213021b0ea26d63f9e9620f4cc}{User-Agent:
jclouds/2019.224.2 java/1.8.0_222}{Host:
roded-data.s3-eu-central-1.ama
[jira] [Updated] (JCLOUDS-1520) JClouds is not using the JDK's KeepAliveCache when UntrustedSSLContextSupplier is used
[
https://issues.apache.org/jira/browse/JCLOUDS-1520?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Roded Bahat updated JCLOUDS-1520:
-
Description:
It seems like the fact that {{UntrustedSSLContextSupplier}} returns a new
{{SSLContext}} on every {{get()}} call causes a consistent cache miss on the
JVM's {{sun.net.www.http.KeepAliveCache}} Which causes JClouds to not reusing
existing TLS connections even though it could.
The cache miss happens at {{sun.net.www.protocol.https.HttpsClient}} line 329
(openjdk version "1.8.0_222"):
{noformat}
/* see if one's already around */
ret = (HttpsClient) kac.get(url, sf);
{noformat}
To reproduce, consider the following main:
{noformat}
public static void main(String[] args) {
Properties overrides = new Properties();
overrides.setProperty(org.jclouds.Constants.PROPERTY_TRUST_ALL_CERTS,
"true");
BlobStoreContext blobStoreContext = ContextBuilder.newBuilder("aws-s3")
.endpoint("https://s3.amazonaws.com;)
.credentials("...", "...")
.overrides(overrides)
.buildView(BlobStoreContext.class);
BlobStore blobStore = blobStoreContext.getBlobStore();
blobStore.getBlob("roded-data", "blobname");
blobStore.getBlob("roded-data", "blobname");
blobStore.getBlob("roded-data", "blobname");
blobStoreContext.close();
System.exit(0);
}
{noformat}
If run using a JUL logging.properties with the following logger set to FINEST:
{noformat}
sun.net.www.protocol.http.level=FINEST
{noformat}
The following log is produced:
{noformat}
2019-10-10 18:15:19.668 FINE[org.jclouds.rest.internal.InvokeHttpMethod ]
>> invoking GetBucketLocation
2019-10-10 18:15:19.733 FINE
[org.jclouds.http.internal.JavaUrlHttpCommandExecutorService ] Sending request
-1721710788: GET https://s3.amazonaws.com/roded-data?location HTTP/1.1
2019-10-10 18:15:19.893 FINEST [sun.net.www.protocol.http.HttpURLConnection ]
Looking for HttpClient for URL https://s3.amazonaws.com/roded-data?location and
proxy value of DIRECT
2019-10-10 18:15:19.893 FINEST [sun.net.www.protocol.http.HttpURLConnection ]
Creating new HttpsClient with url:https://s3.amazonaws.com/roded-data?location
and proxy:DIRECT with connect timeout:6
2019-10-10 18:15:20.837 FINE[sun.net.www.protocol.http.HttpURLConnection ]
sun.net.www.MessageHeader@537b32ef8 pairs: {GET /roded-data?location HTTP/1.1:
null}{x-amz-content-sha256:
e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855}{X-Amz-Date:
20191010T151519Z}{Authorization: AWS4-HMAC-SHA256
Credential=AKIAJO5RLGWKFW5ASG3A/20191010/us-east-1/s3/aws4_request,
SignedHeaders=host;x-amz-content-sha256;x-amz-date,
Signature=896e11ddd9efac465b6ff2506d1688d454a50b3f73ac68d557ad036b1826e591}{User-Agent:
jclouds/2019.224.2 java/1.8.0_222}{Host: s3.amazonaws.com}{Accept: text/html,
image/gif, image/jpeg, *; q=.2, */*; q=.2}{Connection: keep-alive}
2019-10-10 18:15:21.169 FINE[sun.net.www.protocol.http.HttpURLConnection ]
sun.net.www.MessageHeader@6f815e7f7 pairs: {null: HTTP/1.1 200 OK}{x-amz-id-2:
1VVlx4h/fBOFe3n/7IxvpWN0RoVcE2rSpnnxMjvAQ93lJ6tHJAS+3IlXAx++/ZMEblp7kjJT4eQ=}{x-amz-request-id:
AE0779131201B495}{Date: Thu, 10 Oct 2019 15:15:21 GMT}{Content-Type:
application/xml}{Transfer-Encoding: chunked}{Server: AmazonS3}
2019-10-10 18:15:21.185 FINE
[org.jclouds.http.internal.JavaUrlHttpCommandExecutorService ] Receiving
response -1721710788: HTTP/1.1 200 OK
2019-10-10 18:15:21.500 FINE[org.jclouds.rest.internal.InvokeHttpMethod ]
>> invoking GetObject
2019-10-10 18:15:21.514 FINE
[org.jclouds.http.internal.JavaUrlHttpCommandExecutorService ] Sending request
-1332190413: GET https://roded-data.s3-eu-central-1.amazonaws.com/blobname
HTTP/1.1
2019-10-10 18:15:21.517 FINEST [sun.net.www.protocol.http.HttpURLConnection ]
Looking for HttpClient for URL
https://roded-data.s3-eu-central-1.amazonaws.com/blobname and proxy value of
DIRECT
2019-10-10 18:15:21.519 FINEST [sun.net.www.protocol.http.HttpURLConnection ]
Creating new HttpsClient with
url:https://roded-data.s3-eu-central-1.amazonaws.com/blobname and proxy:DIRECT
with connect timeout:6
2019-10-10 18:15:22.319 FINE[sun.net.www.protocol.http.HttpURLConnection ]
sun.net.www.MessageHeader@6a933be28 pairs: {GET /blobname HTTP/1.1:
null}{x-amz-content-sha256:
e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855}{X-Amz-Date:
20191010T151519Z}{Authorization: AWS4-HMAC-SHA256
Credential=AKIAJO5RLGWKFW5ASG3A/20191010/eu-central-1/s3/aws4_request,
SignedHeaders=host;x-amz-content-sha256;x-amz-date,
Signature=52fadcda579bb56e62b9b0489d7a0a90080103213021b0ea26d63f9e9620f4cc}{User-Agent:
jclouds/2019.224.2 java/1.8.0_222}{Host:
roded-data.s3-eu-central-1.ama
[jira] [Resolved] (JCLOUDS-1505) BlobStore.blobMetadata(container, object) returns a StorageMetadata object with empty size when using org.jclouds.http.apachehc.config.ApacheHCHttpCommandExecutorServi
[
https://issues.apache.org/jira/browse/JCLOUDS-1505?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Gaul resolved JCLOUDS-1505.
--
Fix Version/s: 2.1.3
2.2.0
Assignee: Andrew Gaul
Resolution: Fixed
> BlobStore.blobMetadata(container, object) returns a StorageMetadata object
> with empty size when using
> org.jclouds.http.apachehc.config.ApacheHCHttpCommandExecutorServiceModule
> ---
>
> Key: JCLOUDS-1505
> URL: https://issues.apache.org/jira/browse/JCLOUDS-1505
> Project: jclouds
> Issue Type: Bug
> Components: jclouds-drivers
>Reporter: Xavier BOURGOUIN
>Assignee: Andrew Gaul
>Priority: Major
> Fix For: 2.2.0, 2.1.3
>
> Attachments: jclouds_null_size_reproducer.tar.gz
>
>
> Hello,
>
> When calling *blobStore.blobMetadata(container, objectName).getSize()* with a
> blobStoreContext using ApacheHCHttpCommandExecutorServiceModule, the returned
> BlobMetadata object has a size attribute which is null.
> Issue spotted while using jclouds 2.1.2 over an S3 object store (Scality, but
> seems like it is reproducible with any other S3 provider, and potentially
> others OS providers like GCS, to be confirmed).
> In essence, the minimal reproducer pseudo-code is:
> {code:java}
> BlobStoreContext context = ContextBuilder.newBuilder(provider)
> .credentials(identity, credential)
> .endpoint(endpoint)
> // usage of ApacheHCHttp module
> .modules(ImmutableList. of(new
> ApacheHCHttpCommandExecutorServiceModule(), new SLF4JLoggingModule()))
> .buildView(BlobStoreContext.class);
>
> try {
> BlobStore blobStore = context.getBlobStore();
> // output is: "File size from metadata: null"
> System.out.println("File size from metadata: " +
> blobStore.blobMetadata(DUMMY_CONTAINER, DUMMY_FILE_NAME).getSize());
> } finally {
> context.close();
> }
> {code}
>
> For convenience, I've attached a ready to run reproducer to this ticket,
> here's how to run it:
>
> {code:bash}
> > mkdir reproducer
> > tar xvzf jclouds_null_size_reproducer.tar.gz -C reproducer
> > cd reproducer
> > mvn install
> # note: pre-req from this stage is to have an S3 Object Store available, you
> can use minio one if you have docker (docker run -it -p 9000:9000 -v
> /mnt/data:/data minio/minio server /data)
> # replace with your Object Store endpoint, identity and access key
> > java -jar target/blobstore-basics-jar-with-dependencies.jar s3
> > http://10.66.33.1:9000
> File size from metadata: 8
> File size from metadata (with ApacheHCHttpCommandExecutorServiceModule): null
> {code}
> The reproducer code has jclouds-wire logs enabled (inside log folder) and
> I've also added the log files inside the attached tarball for convenience. We
> see that in the HTTP HEAD response (the 2nd one, which correspond to my
> .blobMetadata() call using ApacheHCHttpCommandExecutorServiceModule), the
> Content-* headers are missing :
> {code:bash}
> >cat log/jclouds-wire.log
> # note: first .blobMetadata() call without
> ApacheHCHttpCommandExecutorServiceModule, the response contains Content-*
> headers
> 2019-07-31 17:30:13,257 DEBUG [jclouds.headers] [main] >> HEAD
> http://10.66.33.1:9000/foo/example_for_empty_size HTTP/1.1
> 2019-07-31 17:30:13,258 DEBUG [jclouds.headers] [main] >> Date: Wed, 31 Jul
> 2019 15:30:13 GMT
> 2019-07-31 17:30:13,258 DEBUG [jclouds.headers] [main] >> Authorization: AWS
> 8A0CESVSLMU5HSSBIOB0:hvzLzlfV+ep7K6Om+tfFss5AuZE=
> 2019-07-31 17:30:13,259 DEBUG [jclouds.headers] [main] << HTTP/1.1 200 OK
> 2019-07-31 17:30:13,259 DEBUG [jclouds.headers] [main] << Accept-Ranges: bytes
> 2019-07-31 17:30:13,259 DEBUG [jclouds.headers] [main] << Server:
> MinIO/RELEASE.2019-07-24T02-02-23Z
> 2019-07-31 17:30:13,259 DEBUG [jclouds.headers] [main] << ETag:
> "35a41457219c775659b6018fb7f465a1-1"
> 2019-07-31 17:30:13,259 DEBUG [jclouds.headers] [main] << X-Xss-Protection:
> 1; mode=block
> 2019-07-31 17:30:13,259 DEBUG [jclouds.headers] [main] <<
> Content-Security-Policy: block-all-mixed-content
> 2019-07-31 17:30:13,259 DEBUG [jclouds.headers] [main] << Vary: Origin
> 2019-07-31 17:30:13,259 DEBUG [jclouds.headers] [main] << Last-Modified: Wed,
> 31 Jul 2019 15:30:13 GMT
> 2019-07-31 17:30:13,259 DEBUG [jclouds.headers] [main] << Date: Wed, 31 Jul
> 2019 15:30:13 GMT
> 2019-07-31 17:30:13,259 DEBUG [jclouds.headers] [main] << X-Amz-Request-Id:
> 15B687995977CC49
> 2019-07-31 17:30:13,259 DEBUG [jclouds.headers] [main] << Content-Type:
> application/unknown
> 2019-07-31 17:30:13,259 DEBUG [jclouds.headers] [main] << Content-Length: 8
> # note: second call with
[jira] [Commented] (JCLOUDS-1505) BlobStore.blobMetadata(container, object) returns a StorageMetadata object with empty size when using org.jclouds.http.apachehc.config.ApacheHCHttpCommandExecutorServ
[
https://issues.apache.org/jira/browse/JCLOUDS-1505?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16948782#comment-16948782
]
ASF subversion and git services commented on JCLOUDS-1505:
--
Commit 6303d9630840e99ea150e6ad192ed4c8b89b8295 in jclouds's branch
refs/heads/2.1.x from Xavier BOURGOUIN
[ https://gitbox.apache.org/repos/asf?p=jclouds.git;h=6303d96 ]
Fix null content-length header on HEAD requests
https://issues.apache.org/jira/projects/JCLOUDS/issues/JCLOUDS-1505
> BlobStore.blobMetadata(container, object) returns a StorageMetadata object
> with empty size when using
> org.jclouds.http.apachehc.config.ApacheHCHttpCommandExecutorServiceModule
> ---
>
> Key: JCLOUDS-1505
> URL: https://issues.apache.org/jira/browse/JCLOUDS-1505
> Project: jclouds
> Issue Type: Bug
> Components: jclouds-drivers
>Reporter: Xavier BOURGOUIN
>Priority: Major
> Attachments: jclouds_null_size_reproducer.tar.gz
>
>
> Hello,
>
> When calling *blobStore.blobMetadata(container, objectName).getSize()* with a
> blobStoreContext using ApacheHCHttpCommandExecutorServiceModule, the returned
> BlobMetadata object has a size attribute which is null.
> Issue spotted while using jclouds 2.1.2 over an S3 object store (Scality, but
> seems like it is reproducible with any other S3 provider, and potentially
> others OS providers like GCS, to be confirmed).
> In essence, the minimal reproducer pseudo-code is:
> {code:java}
> BlobStoreContext context = ContextBuilder.newBuilder(provider)
> .credentials(identity, credential)
> .endpoint(endpoint)
> // usage of ApacheHCHttp module
> .modules(ImmutableList. of(new
> ApacheHCHttpCommandExecutorServiceModule(), new SLF4JLoggingModule()))
> .buildView(BlobStoreContext.class);
>
> try {
> BlobStore blobStore = context.getBlobStore();
> // output is: "File size from metadata: null"
> System.out.println("File size from metadata: " +
> blobStore.blobMetadata(DUMMY_CONTAINER, DUMMY_FILE_NAME).getSize());
> } finally {
> context.close();
> }
> {code}
>
> For convenience, I've attached a ready to run reproducer to this ticket,
> here's how to run it:
>
> {code:bash}
> > mkdir reproducer
> > tar xvzf jclouds_null_size_reproducer.tar.gz -C reproducer
> > cd reproducer
> > mvn install
> # note: pre-req from this stage is to have an S3 Object Store available, you
> can use minio one if you have docker (docker run -it -p 9000:9000 -v
> /mnt/data:/data minio/minio server /data)
> # replace with your Object Store endpoint, identity and access key
> > java -jar target/blobstore-basics-jar-with-dependencies.jar s3
> > http://10.66.33.1:9000
> File size from metadata: 8
> File size from metadata (with ApacheHCHttpCommandExecutorServiceModule): null
> {code}
> The reproducer code has jclouds-wire logs enabled (inside log folder) and
> I've also added the log files inside the attached tarball for convenience. We
> see that in the HTTP HEAD response (the 2nd one, which correspond to my
> .blobMetadata() call using ApacheHCHttpCommandExecutorServiceModule), the
> Content-* headers are missing :
> {code:bash}
> >cat log/jclouds-wire.log
> # note: first .blobMetadata() call without
> ApacheHCHttpCommandExecutorServiceModule, the response contains Content-*
> headers
> 2019-07-31 17:30:13,257 DEBUG [jclouds.headers] [main] >> HEAD
> http://10.66.33.1:9000/foo/example_for_empty_size HTTP/1.1
> 2019-07-31 17:30:13,258 DEBUG [jclouds.headers] [main] >> Date: Wed, 31 Jul
> 2019 15:30:13 GMT
> 2019-07-31 17:30:13,258 DEBUG [jclouds.headers] [main] >> Authorization: AWS
> 8A0CESVSLMU5HSSBIOB0:hvzLzlfV+ep7K6Om+tfFss5AuZE=
> 2019-07-31 17:30:13,259 DEBUG [jclouds.headers] [main] << HTTP/1.1 200 OK
> 2019-07-31 17:30:13,259 DEBUG [jclouds.headers] [main] << Accept-Ranges: bytes
> 2019-07-31 17:30:13,259 DEBUG [jclouds.headers] [main] << Server:
> MinIO/RELEASE.2019-07-24T02-02-23Z
> 2019-07-31 17:30:13,259 DEBUG [jclouds.headers] [main] << ETag:
> "35a41457219c775659b6018fb7f465a1-1"
> 2019-07-31 17:30:13,259 DEBUG [jclouds.headers] [main] << X-Xss-Protection:
> 1; mode=block
> 2019-07-31 17:30:13,259 DEBUG [jclouds.headers] [main] <<
> Content-Security-Policy: block-all-mixed-content
> 2019-07-31 17:30:13,259 DEBUG [jclouds.headers] [main] << Vary: Origin
> 2019-07-31 17:30:13,259 DEBUG [jclouds.headers] [main] << Last-Modified: Wed,
> 31 Jul 2019 15:30:13 GMT
> 2019-07-31 17:30:13,259 DEBUG [jclouds.headers] [main] << Date: Wed, 31 Jul
> 2019 15:30:13 GMT
> 2019-07-31 17:30:13,259 DEBUG [jclouds.headers] [main] << X-Amz-Request-Id:
> 15B687995977CC49
> 2019-07-31 17:30:13,259
[GitHub] [jclouds] gaul commented on issue #48: Fix null content-length header on HEAD requests
gaul commented on issue #48: Fix null content-length header on HEAD requests URL: https://github.com/apache/jclouds/pull/48#issuecomment-540679758 Also pushed to 2.1.x as 6303d9630840e99ea150e6ad192ed4c8b89b8295. Thank you for your contribution @xavierb-amadeus! This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (JCLOUDS-1505) BlobStore.blobMetadata(container, object) returns a StorageMetadata object with empty size when using org.jclouds.http.apachehc.config.ApacheHCHttpCommandExecutorServ
[
https://issues.apache.org/jira/browse/JCLOUDS-1505?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16948778#comment-16948778
]
ASF subversion and git services commented on JCLOUDS-1505:
--
Commit c445547ea80a184a328591b40c182a208c6670d5 in jclouds's branch
refs/heads/master from Xavier BOURGOUIN
[ https://gitbox.apache.org/repos/asf?p=jclouds.git;h=c445547 ]
Fix null content-length header on HEAD requests
https://issues.apache.org/jira/projects/JCLOUDS/issues/JCLOUDS-1505
> BlobStore.blobMetadata(container, object) returns a StorageMetadata object
> with empty size when using
> org.jclouds.http.apachehc.config.ApacheHCHttpCommandExecutorServiceModule
> ---
>
> Key: JCLOUDS-1505
> URL: https://issues.apache.org/jira/browse/JCLOUDS-1505
> Project: jclouds
> Issue Type: Bug
> Components: jclouds-drivers
>Reporter: Xavier BOURGOUIN
>Priority: Major
> Attachments: jclouds_null_size_reproducer.tar.gz
>
>
> Hello,
>
> When calling *blobStore.blobMetadata(container, objectName).getSize()* with a
> blobStoreContext using ApacheHCHttpCommandExecutorServiceModule, the returned
> BlobMetadata object has a size attribute which is null.
> Issue spotted while using jclouds 2.1.2 over an S3 object store (Scality, but
> seems like it is reproducible with any other S3 provider, and potentially
> others OS providers like GCS, to be confirmed).
> In essence, the minimal reproducer pseudo-code is:
> {code:java}
> BlobStoreContext context = ContextBuilder.newBuilder(provider)
> .credentials(identity, credential)
> .endpoint(endpoint)
> // usage of ApacheHCHttp module
> .modules(ImmutableList. of(new
> ApacheHCHttpCommandExecutorServiceModule(), new SLF4JLoggingModule()))
> .buildView(BlobStoreContext.class);
>
> try {
> BlobStore blobStore = context.getBlobStore();
> // output is: "File size from metadata: null"
> System.out.println("File size from metadata: " +
> blobStore.blobMetadata(DUMMY_CONTAINER, DUMMY_FILE_NAME).getSize());
> } finally {
> context.close();
> }
> {code}
>
> For convenience, I've attached a ready to run reproducer to this ticket,
> here's how to run it:
>
> {code:bash}
> > mkdir reproducer
> > tar xvzf jclouds_null_size_reproducer.tar.gz -C reproducer
> > cd reproducer
> > mvn install
> # note: pre-req from this stage is to have an S3 Object Store available, you
> can use minio one if you have docker (docker run -it -p 9000:9000 -v
> /mnt/data:/data minio/minio server /data)
> # replace with your Object Store endpoint, identity and access key
> > java -jar target/blobstore-basics-jar-with-dependencies.jar s3
> > http://10.66.33.1:9000
> File size from metadata: 8
> File size from metadata (with ApacheHCHttpCommandExecutorServiceModule): null
> {code}
> The reproducer code has jclouds-wire logs enabled (inside log folder) and
> I've also added the log files inside the attached tarball for convenience. We
> see that in the HTTP HEAD response (the 2nd one, which correspond to my
> .blobMetadata() call using ApacheHCHttpCommandExecutorServiceModule), the
> Content-* headers are missing :
> {code:bash}
> >cat log/jclouds-wire.log
> # note: first .blobMetadata() call without
> ApacheHCHttpCommandExecutorServiceModule, the response contains Content-*
> headers
> 2019-07-31 17:30:13,257 DEBUG [jclouds.headers] [main] >> HEAD
> http://10.66.33.1:9000/foo/example_for_empty_size HTTP/1.1
> 2019-07-31 17:30:13,258 DEBUG [jclouds.headers] [main] >> Date: Wed, 31 Jul
> 2019 15:30:13 GMT
> 2019-07-31 17:30:13,258 DEBUG [jclouds.headers] [main] >> Authorization: AWS
> 8A0CESVSLMU5HSSBIOB0:hvzLzlfV+ep7K6Om+tfFss5AuZE=
> 2019-07-31 17:30:13,259 DEBUG [jclouds.headers] [main] << HTTP/1.1 200 OK
> 2019-07-31 17:30:13,259 DEBUG [jclouds.headers] [main] << Accept-Ranges: bytes
> 2019-07-31 17:30:13,259 DEBUG [jclouds.headers] [main] << Server:
> MinIO/RELEASE.2019-07-24T02-02-23Z
> 2019-07-31 17:30:13,259 DEBUG [jclouds.headers] [main] << ETag:
> "35a41457219c775659b6018fb7f465a1-1"
> 2019-07-31 17:30:13,259 DEBUG [jclouds.headers] [main] << X-Xss-Protection:
> 1; mode=block
> 2019-07-31 17:30:13,259 DEBUG [jclouds.headers] [main] <<
> Content-Security-Policy: block-all-mixed-content
> 2019-07-31 17:30:13,259 DEBUG [jclouds.headers] [main] << Vary: Origin
> 2019-07-31 17:30:13,259 DEBUG [jclouds.headers] [main] << Last-Modified: Wed,
> 31 Jul 2019 15:30:13 GMT
> 2019-07-31 17:30:13,259 DEBUG [jclouds.headers] [main] << Date: Wed, 31 Jul
> 2019 15:30:13 GMT
> 2019-07-31 17:30:13,259 DEBUG [jclouds.headers] [main] << X-Amz-Request-Id:
> 15B687995977CC49
> 2019-07-31
[GitHub] [jclouds] gaul merged pull request #48: Fix null content-length header on HEAD requests
gaul merged pull request #48: Fix null content-length header on HEAD requests URL: https://github.com/apache/jclouds/pull/48 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (JCLOUDS-1505) BlobStore.blobMetadata(container, object) returns a StorageMetadata object with empty size when using org.jclouds.http.apachehc.config.ApacheHCHttpCommandExecutorServ
[
https://issues.apache.org/jira/browse/JCLOUDS-1505?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16948716#comment-16948716
]
Xavier BOURGOUIN commented on JCLOUDS-1505:
---
I've made a proposal fix for this issue:
https://github.com/apache/jclouds/pull/48
Could you please review it when time permits ?
Thanks!
Brgds,
Xavier
> BlobStore.blobMetadata(container, object) returns a StorageMetadata object
> with empty size when using
> org.jclouds.http.apachehc.config.ApacheHCHttpCommandExecutorServiceModule
> ---
>
> Key: JCLOUDS-1505
> URL: https://issues.apache.org/jira/browse/JCLOUDS-1505
> Project: jclouds
> Issue Type: Bug
> Components: jclouds-drivers
>Reporter: Xavier BOURGOUIN
>Priority: Major
> Attachments: jclouds_null_size_reproducer.tar.gz
>
>
> Hello,
>
> When calling *blobStore.blobMetadata(container, objectName).getSize()* with a
> blobStoreContext using ApacheHCHttpCommandExecutorServiceModule, the returned
> BlobMetadata object has a size attribute which is null.
> Issue spotted while using jclouds 2.1.2 over an S3 object store (Scality, but
> seems like it is reproducible with any other S3 provider, and potentially
> others OS providers like GCS, to be confirmed).
> In essence, the minimal reproducer pseudo-code is:
> {code:java}
> BlobStoreContext context = ContextBuilder.newBuilder(provider)
> .credentials(identity, credential)
> .endpoint(endpoint)
> // usage of ApacheHCHttp module
> .modules(ImmutableList. of(new
> ApacheHCHttpCommandExecutorServiceModule(), new SLF4JLoggingModule()))
> .buildView(BlobStoreContext.class);
>
> try {
> BlobStore blobStore = context.getBlobStore();
> // output is: "File size from metadata: null"
> System.out.println("File size from metadata: " +
> blobStore.blobMetadata(DUMMY_CONTAINER, DUMMY_FILE_NAME).getSize());
> } finally {
> context.close();
> }
> {code}
>
> For convenience, I've attached a ready to run reproducer to this ticket,
> here's how to run it:
>
> {code:bash}
> > mkdir reproducer
> > tar xvzf jclouds_null_size_reproducer.tar.gz -C reproducer
> > cd reproducer
> > mvn install
> # note: pre-req from this stage is to have an S3 Object Store available, you
> can use minio one if you have docker (docker run -it -p 9000:9000 -v
> /mnt/data:/data minio/minio server /data)
> # replace with your Object Store endpoint, identity and access key
> > java -jar target/blobstore-basics-jar-with-dependencies.jar s3
> > http://10.66.33.1:9000
> File size from metadata: 8
> File size from metadata (with ApacheHCHttpCommandExecutorServiceModule): null
> {code}
> The reproducer code has jclouds-wire logs enabled (inside log folder) and
> I've also added the log files inside the attached tarball for convenience. We
> see that in the HTTP HEAD response (the 2nd one, which correspond to my
> .blobMetadata() call using ApacheHCHttpCommandExecutorServiceModule), the
> Content-* headers are missing :
> {code:bash}
> >cat log/jclouds-wire.log
> # note: first .blobMetadata() call without
> ApacheHCHttpCommandExecutorServiceModule, the response contains Content-*
> headers
> 2019-07-31 17:30:13,257 DEBUG [jclouds.headers] [main] >> HEAD
> http://10.66.33.1:9000/foo/example_for_empty_size HTTP/1.1
> 2019-07-31 17:30:13,258 DEBUG [jclouds.headers] [main] >> Date: Wed, 31 Jul
> 2019 15:30:13 GMT
> 2019-07-31 17:30:13,258 DEBUG [jclouds.headers] [main] >> Authorization: AWS
> 8A0CESVSLMU5HSSBIOB0:hvzLzlfV+ep7K6Om+tfFss5AuZE=
> 2019-07-31 17:30:13,259 DEBUG [jclouds.headers] [main] << HTTP/1.1 200 OK
> 2019-07-31 17:30:13,259 DEBUG [jclouds.headers] [main] << Accept-Ranges: bytes
> 2019-07-31 17:30:13,259 DEBUG [jclouds.headers] [main] << Server:
> MinIO/RELEASE.2019-07-24T02-02-23Z
> 2019-07-31 17:30:13,259 DEBUG [jclouds.headers] [main] << ETag:
> "35a41457219c775659b6018fb7f465a1-1"
> 2019-07-31 17:30:13,259 DEBUG [jclouds.headers] [main] << X-Xss-Protection:
> 1; mode=block
> 2019-07-31 17:30:13,259 DEBUG [jclouds.headers] [main] <<
> Content-Security-Policy: block-all-mixed-content
> 2019-07-31 17:30:13,259 DEBUG [jclouds.headers] [main] << Vary: Origin
> 2019-07-31 17:30:13,259 DEBUG [jclouds.headers] [main] << Last-Modified: Wed,
> 31 Jul 2019 15:30:13 GMT
> 2019-07-31 17:30:13,259 DEBUG [jclouds.headers] [main] << Date: Wed, 31 Jul
> 2019 15:30:13 GMT
> 2019-07-31 17:30:13,259 DEBUG [jclouds.headers] [main] << X-Amz-Request-Id:
> 15B687995977CC49
> 2019-07-31 17:30:13,259 DEBUG [jclouds.headers] [main] << Content-Type:
> application/unknown
> 2019-07-31 17:30:13,259 DEBUG [jclouds.headers] [main] << Content-Length: 8
> # note: second
[GitHub] [jclouds] xavierb-amadeus opened a new pull request #48: Fix null content-length header on HEAD requests
xavierb-amadeus opened a new pull request #48: Fix null content-length header on HEAD requests URL: https://github.com/apache/jclouds/pull/48 proposal fix for issue reported in https://issues.apache.org/jira/projects/JCLOUDS/issues/JCLOUDS-1505 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Updated] (JCLOUDS-1520) JClouds is not using the JDK's KeepAliveCache when UntrustedSSLContextSupplier is used
[
https://issues.apache.org/jira/browse/JCLOUDS-1520?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Roded Bahat updated JCLOUDS-1520:
-
Description:
It seems like the fact that {{UntrustedSSLContextSupplier}} returns a new
{{SSLContext}} on every {{get()}} call causes a consistent cache miss on the
JVM's {{sun.net.www.http.KeepAliveCache}} Which causes JClouds to not reusing
existing TLS connections even though it could.
The cache miss happens at {{sun.net.www.protocol.https.HttpsClient}} line 329:
{noformat}
/* see if one's already around */
ret = (HttpsClient) kac.get(url, sf);
{noformat}
To reproduce, consider the following main:
{noformat}
public static void main(String[] args) {
Properties overrides = new Properties();
overrides.setProperty(org.jclouds.Constants.PROPERTY_TRUST_ALL_CERTS,
"true");
BlobStoreContext blobStoreContext = ContextBuilder.newBuilder("aws-s3")
.endpoint("https://s3.amazonaws.com;)
.credentials("...", "...")
.overrides(overrides)
.buildView(BlobStoreContext.class);
BlobStore blobStore = blobStoreContext.getBlobStore();
blobStore.getBlob("roded-data", "blobname");
blobStore.getBlob("roded-data", "blobname");
blobStore.getBlob("roded-data", "blobname");
blobStoreContext.close();
System.exit(0);
}
{noformat}
If run using a JUL logging.properties with the following logger set to FINEST:
{noformat}
sun.net.www.protocol.http.level=FINEST
{noformat}
The following log is produced:
{noformat}
2019-10-10 18:15:19.668 FINE[org.jclouds.rest.internal.InvokeHttpMethod ]
>> invoking GetBucketLocation
2019-10-10 18:15:19.733 FINE
[org.jclouds.http.internal.JavaUrlHttpCommandExecutorService ] Sending request
-1721710788: GET https://s3.amazonaws.com/roded-data?location HTTP/1.1
2019-10-10 18:15:19.893 FINEST [sun.net.www.protocol.http.HttpURLConnection ]
Looking for HttpClient for URL https://s3.amazonaws.com/roded-data?location and
proxy value of DIRECT
2019-10-10 18:15:19.893 FINEST [sun.net.www.protocol.http.HttpURLConnection ]
Creating new HttpsClient with url:https://s3.amazonaws.com/roded-data?location
and proxy:DIRECT with connect timeout:6
2019-10-10 18:15:20.837 FINE[sun.net.www.protocol.http.HttpURLConnection ]
sun.net.www.MessageHeader@537b32ef8 pairs: {GET /roded-data?location HTTP/1.1:
null}{x-amz-content-sha256:
e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855}{X-Amz-Date:
20191010T151519Z}{Authorization: AWS4-HMAC-SHA256
Credential=AKIAJO5RLGWKFW5ASG3A/20191010/us-east-1/s3/aws4_request,
SignedHeaders=host;x-amz-content-sha256;x-amz-date,
Signature=896e11ddd9efac465b6ff2506d1688d454a50b3f73ac68d557ad036b1826e591}{User-Agent:
jclouds/2019.224.2 java/1.8.0_222}{Host: s3.amazonaws.com}{Accept: text/html,
image/gif, image/jpeg, *; q=.2, */*; q=.2}{Connection: keep-alive}
2019-10-10 18:15:21.169 FINE[sun.net.www.protocol.http.HttpURLConnection ]
sun.net.www.MessageHeader@6f815e7f7 pairs: {null: HTTP/1.1 200 OK}{x-amz-id-2:
1VVlx4h/fBOFe3n/7IxvpWN0RoVcE2rSpnnxMjvAQ93lJ6tHJAS+3IlXAx++/ZMEblp7kjJT4eQ=}{x-amz-request-id:
AE0779131201B495}{Date: Thu, 10 Oct 2019 15:15:21 GMT}{Content-Type:
application/xml}{Transfer-Encoding: chunked}{Server: AmazonS3}
2019-10-10 18:15:21.185 FINE
[org.jclouds.http.internal.JavaUrlHttpCommandExecutorService ] Receiving
response -1721710788: HTTP/1.1 200 OK
2019-10-10 18:15:21.500 FINE[org.jclouds.rest.internal.InvokeHttpMethod ]
>> invoking GetObject
2019-10-10 18:15:21.514 FINE
[org.jclouds.http.internal.JavaUrlHttpCommandExecutorService ] Sending request
-1332190413: GET https://roded-data.s3-eu-central-1.amazonaws.com/blobname
HTTP/1.1
2019-10-10 18:15:21.517 FINEST [sun.net.www.protocol.http.HttpURLConnection ]
Looking for HttpClient for URL
https://roded-data.s3-eu-central-1.amazonaws.com/blobname and proxy value of
DIRECT
2019-10-10 18:15:21.519 FINEST [sun.net.www.protocol.http.HttpURLConnection ]
Creating new HttpsClient with
url:https://roded-data.s3-eu-central-1.amazonaws.com/blobname and proxy:DIRECT
with connect timeout:6
2019-10-10 18:15:22.319 FINE[sun.net.www.protocol.http.HttpURLConnection ]
sun.net.www.MessageHeader@6a933be28 pairs: {GET /blobname HTTP/1.1:
null}{x-amz-content-sha256:
e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855}{X-Amz-Date:
20191010T151519Z}{Authorization: AWS4-HMAC-SHA256
Credential=AKIAJO5RLGWKFW5ASG3A/20191010/eu-central-1/s3/aws4_request,
SignedHeaders=host;x-amz-content-sha256;x-amz-date,
Signature=52fadcda579bb56e62b9b0489d7a0a90080103213021b0ea26d63f9e9620f4cc}{User-Agent:
jclouds/2019.224.2 java/1.8.0_222}{Host:
roded-data.s3-eu-central-1.amazonaws.com
[jira] [Updated] (JCLOUDS-1520) JClouds is not using the JDK's KeepAliveCache when UntrustedSSLContextSupplier is used
[
https://issues.apache.org/jira/browse/JCLOUDS-1520?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Roded Bahat updated JCLOUDS-1520:
-
Description:
It seems like the fact that {{UntrustedSSLContextSupplier}} returns a new
{{SSLContext}} on every {{get()}} call causes a consistent cache miss on the
JVM's {{sun.net.www.http.KeepAliveCache}} Which causes JClouds to not reusing
existing TLS connections even though it could.
The cache miss happens at {{sun.net.www.protocol.https.HttpsClient}} line 329:
{noformat}
/* see if one's already around */
ret = (HttpsClient) kac.get(url, sf);
{noformat}
To reproduce, consider the following main:
{noformat}
public static void main(String[] args) {
Properties overrides = new Properties();
overrides.setProperty(org.jclouds.Constants.PROPERTY_TRUST_ALL_CERTS,
"true");
BlobStoreContext blobStoreContext = ContextBuilder.newBuilder("aws-s3")
.endpoint("https://s3.amazonaws.com;)
.credentials("...", "...")
.overrides(overrides)
.buildView(BlobStoreContext.class);
BlobStore blobStore = blobStoreContext.getBlobStore();
blobStore.getBlob("roded-data", "blobname");
blobStore.getBlob("roded-data", "blobname");
blobStore.getBlob("roded-data", "blobname");
blobStoreContext.close();
System.exit(0);
}
{noformat}
If run using a JUL logging.properties with the following logger set to FINEST:
{noformat}
sun.net.www.protocol.http.level=FINEST
{noformat}
The following log is produced:
{noformat}
2019-10-10 18:15:19.668 FINE[org.jclouds.rest.internal.InvokeHttpMethod ]
>> invoking GetBucketLocation
2019-10-10 18:15:19.733 FINE
[org.jclouds.http.internal.JavaUrlHttpCommandExecutorService ] Sending request
-1721710788: GET https://s3.amazonaws.com/roded-data?location HTTP/1.1
2019-10-10 18:15:19.893 FINEST [sun.net.www.protocol.http.HttpURLConnection ]
Looking for HttpClient for URL https://s3.amazonaws.com/roded-data?location and
proxy value of DIRECT
2019-10-10 18:15:19.893 FINEST [sun.net.www.protocol.http.HttpURLConnection ]
Creating new HttpsClient with url:https://s3.amazonaws.com/roded-data?location
and proxy:DIRECT with connect timeout:6
2019-10-10 18:15:20.837 FINE[sun.net.www.protocol.http.HttpURLConnection ]
sun.net.www.MessageHeader@537b32ef8 pairs: {GET /roded-data?location HTTP/1.1:
null}{x-amz-content-sha256:
e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855}{X-Amz-Date:
20191010T151519Z}{Authorization: AWS4-HMAC-SHA256
Credential=AKIAJO5RLGWKFW5ASG3A/20191010/us-east-1/s3/aws4_request,
SignedHeaders=host;x-amz-content-sha256;x-amz-date,
Signature=896e11ddd9efac465b6ff2506d1688d454a50b3f73ac68d557ad036b1826e591}{User-Agent:
jclouds/2019.224.2 java/1.8.0_222}{Host: s3.amazonaws.com}{Accept: text/html,
image/gif, image/jpeg, *; q=.2, */*; q=.2}{Connection: keep-alive}
2019-10-10 18:15:21.169 FINE[sun.net.www.protocol.http.HttpURLConnection ]
sun.net.www.MessageHeader@6f815e7f7 pairs: {null: HTTP/1.1 200 OK}{x-amz-id-2:
1VVlx4h/fBOFe3n/7IxvpWN0RoVcE2rSpnnxMjvAQ93lJ6tHJAS+3IlXAx++/ZMEblp7kjJT4eQ=}{x-amz-request-id:
AE0779131201B495}{Date: Thu, 10 Oct 2019 15:15:21 GMT}{Content-Type:
application/xml}{Transfer-Encoding: chunked}{Server: AmazonS3}
2019-10-10 18:15:21.185 FINE
[org.jclouds.http.internal.JavaUrlHttpCommandExecutorService ] Receiving
response -1721710788: HTTP/1.1 200 OK
2019-10-10 18:15:21.500 FINE[org.jclouds.rest.internal.InvokeHttpMethod ]
>> invoking GetObject
2019-10-10 18:15:21.514 FINE
[org.jclouds.http.internal.JavaUrlHttpCommandExecutorService ] Sending request
-1332190413: GET https://roded-data.s3-eu-central-1.amazonaws.com/blobname
HTTP/1.1
2019-10-10 18:15:21.517 FINEST [sun.net.www.protocol.http.HttpURLConnection ]
Looking for HttpClient for URL
https://roded-data.s3-eu-central-1.amazonaws.com/blobname and proxy value of
DIRECT
2019-10-10 18:15:21.519 FINEST [sun.net.www.protocol.http.HttpURLConnection ]
Creating new HttpsClient with
url:https://roded-data.s3-eu-central-1.amazonaws.com/blobname and proxy:DIRECT
with connect timeout:6
2019-10-10 18:15:22.319 FINE[sun.net.www.protocol.http.HttpURLConnection ]
sun.net.www.MessageHeader@6a933be28 pairs: {GET /blobname HTTP/1.1:
null}{x-amz-content-sha256:
e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855}{X-Amz-Date:
20191010T151519Z}{Authorization: AWS4-HMAC-SHA256
Credential=AKIAJO5RLGWKFW5ASG3A/20191010/eu-central-1/s3/aws4_request,
SignedHeaders=host;x-amz-content-sha256;x-amz-date,
Signature=52fadcda579bb56e62b9b0489d7a0a90080103213021b0ea26d63f9e9620f4cc}{User-Agent:
jclouds/2019.224.2 java/1.8.0_222}{Host:
roded-data.s3-eu-central-1.amazonaws.com
[jira] [Created] (JCLOUDS-1520) JClouds is not using the JDK's KeepAliveCache when UntrustedSSLContextSupplier is used
Roded Bahat created JCLOUDS-1520:
Summary: JClouds is not using the JDK's KeepAliveCache when
UntrustedSSLContextSupplier is used
Key: JCLOUDS-1520
URL: https://issues.apache.org/jira/browse/JCLOUDS-1520
Project: jclouds
Issue Type: Bug
Components: jclouds-core
Affects Versions: 2.1.0
Reporter: Roded Bahat
It seems like the fact that {{UntrustedSSLContextSupplier}} returns a new
{{SSLContext}} on every {{get()}} call causes a consistent cache miss on the
JVM's {{sun.net.www.http.KeepAliveCache}} Which causes JClouds to not reusing
existing TLS connections even though it could.
The cache miss happens at {{sun.net.www.protocol.https.HttpsClient}} line 329:
{noformat}
/* see if one's already around */
ret = (HttpsClient) kac.get(url, sf);
{noformat}
To reproduce, consider the following main:
{noformat}
public static void main(String[] args) {
Properties overrides = new Properties();
overrides.setProperty(org.jclouds.Constants.PROPERTY_TRUST_ALL_CERTS,
"true");
BlobStoreContext blobStoreContext = ContextBuilder.newBuilder("aws-s3")
.endpoint("https://s3.amazonaws.com;)
.credentials("...", "...")
.overrides(overrides)
.buildView(BlobStoreContext.class);
BlobStore blobStore = blobStoreContext.getBlobStore();
blobStore.getBlob("roded-data", "blobname");
blobStore.getBlob("roded-data", "blobname");
blobStore.getBlob("roded-data", "blobname");
blobStoreContext.close();
System.exit(0);
}
{noformat}
If run using a JUL logging.properties with the following logger set to FINEST:
{noformat}
sun.net.www.protocol.http.level=FINEST
{noformat}
The following log is produced:
{noformat}
2019-10-10 18:15:19.668 FINE[org.jclouds.rest.internal.InvokeHttpMethod ]
>> invoking GetBucketLocation
2019-10-10 18:15:19.733 FINE
[org.jclouds.http.internal.JavaUrlHttpCommandExecutorService ] Sending request
-1721710788: GET https://s3.amazonaws.com/roded-data?location HTTP/1.1
2019-10-10 18:15:19.893 FINEST [sun.net.www.protocol.http.HttpURLConnection ]
Looking for HttpClient for URL https://s3.amazonaws.com/roded-data?location and
proxy value of DIRECT
2019-10-10 18:15:19.893 FINEST [sun.net.www.protocol.http.HttpURLConnection ]
Creating new HttpsClient with url:https://s3.amazonaws.com/roded-data?location
and proxy:DIRECT with connect timeout:6
2019-10-10 18:15:20.837 FINE[sun.net.www.protocol.http.HttpURLConnection ]
sun.net.www.MessageHeader@537b32ef8 pairs: {GET /roded-data?location HTTP/1.1:
null}{x-amz-content-sha256:
e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855}{X-Amz-Date:
20191010T151519Z}{Authorization: AWS4-HMAC-SHA256
Credential=AKIAJO5RLGWKFW5ASG3A/20191010/us-east-1/s3/aws4_request,
SignedHeaders=host;x-amz-content-sha256;x-amz-date,
Signature=896e11ddd9efac465b6ff2506d1688d454a50b3f73ac68d557ad036b1826e591}{User-Agent:
jclouds/2019.224.2 java/1.8.0_222}{Host: s3.amazonaws.com}{Accept: text/html,
image/gif, image/jpeg, *; q=.2, */*; q=.2}{Connection: keep-alive}
2019-10-10 18:15:21.169 FINE[sun.net.www.protocol.http.HttpURLConnection ]
sun.net.www.MessageHeader@6f815e7f7 pairs: {null: HTTP/1.1 200 OK}{x-amz-id-2:
1VVlx4h/fBOFe3n/7IxvpWN0RoVcE2rSpnnxMjvAQ93lJ6tHJAS+3IlXAx++/ZMEblp7kjJT4eQ=}{x-amz-request-id:
AE0779131201B495}{Date: Thu, 10 Oct 2019 15:15:21 GMT}{Content-Type:
application/xml}{Transfer-Encoding: chunked}{Server: AmazonS3}
2019-10-10 18:15:21.185 FINE
[org.jclouds.http.internal.JavaUrlHttpCommandExecutorService ] Receiving
response -1721710788: HTTP/1.1 200 OK
2019-10-10 18:15:21.500 FINE[org.jclouds.rest.internal.InvokeHttpMethod ]
>> invoking GetObject
2019-10-10 18:15:21.514 FINE
[org.jclouds.http.internal.JavaUrlHttpCommandExecutorService ] Sending request
-1332190413: GET https://roded-data.s3-eu-central-1.amazonaws.com/blobname
HTTP/1.1
2019-10-10 18:15:21.517 FINEST [sun.net.www.protocol.http.HttpURLConnection ]
Looking for HttpClient for URL
https://roded-data.s3-eu-central-1.amazonaws.com/blobname and proxy value of
DIRECT
2019-10-10 18:15:21.519 FINEST [sun.net.www.protocol.http.HttpURLConnection ]
Creating new HttpsClient with
url:https://roded-data.s3-eu-central-1.amazonaws.com/blobname and proxy:DIRECT
with connect timeout:6
2019-10-10 18:15:22.319 FINE[sun.net.www.protocol.http.HttpURLConnection ]
sun.net.www.MessageHeader@6a933be28 pairs: {GET /blobname HTTP/1.1:
null}{x-amz-content-sha256:
e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855}{X-Amz-Date:
20191010T151519Z}{Authorization: AWS4-HMAC-SHA256
Credential=AKIAJO5RLGWKFW5ASG3A/20191010/eu-central-1/s3/aws4_request,
SignedHeaders=host;x-a
[GitHub] [jclouds] didixith commented on issue #47: J clouds 1516
didixith commented on issue #47: J clouds 1516 URL: https://github.com/apache/jclouds/pull/47#issuecomment-540608524 Let me know, your inputs on this. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (JCLOUDS-1518) Next jclouds release
[ https://issues.apache.org/jira/browse/JCLOUDS-1518?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16948571#comment-16948571 ] Vasil Bozhurski commented on JCLOUDS-1518: -- Any update on when you plan to release the new version? > Next jclouds release > > > Key: JCLOUDS-1518 > URL: https://issues.apache.org/jira/browse/JCLOUDS-1518 > Project: jclouds > Issue Type: Task > Components: jclouds-core >Reporter: Vasil Bozhurski >Priority: Major > > For Azure Blobstore we at SAP require authentication with SAS token which I > saw was completed in JCLOUDS-1428 and marked for 2.1.3/2.2.0 but it is yet to > be released. This is currently a blocker for us so when can we expect the > next jclouds release? -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [jclouds] didixith commented on issue #47: J clouds 1516
didixith commented on issue #47: J clouds 1516 URL: https://github.com/apache/jclouds/pull/47#issuecomment-540507399 I am not seeing required headers in the request.. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [jclouds] didixith commented on issue #47: J clouds 1516
didixith commented on issue #47: J clouds 1516
URL: https://github.com/apache/jclouds/pull/47#issuecomment-540507240
public void testCreateBucketNonDefaultRegion() throws IOException,
InterruptedException {
MockWebServer server = new MockWebServer();
server.enqueue(new
MockResponse().setBody("").addHeader("x-amz-bucket-region",
"sa-east-1").addHeader("Host", "testbucketsaeasttest-sa-east-1"));
server.play();
S3Client client = getS3Client(server.getUrl("/"));
assertTrue(client.putBucketInRegion(Region.US_EAST_1,
"testbucketsaeasttest"));
RecordedRequest request = server.takeRequest();
server.shutdown();
}
I have written this and output shows as below:
Oct 10, 2019 3:55:59 PM com.squareup.okhttp.mockwebserver.MockWebServer$2
execute
INFO: MockWebServer[51389] starting to accept connections
Oct 10, 2019 3:56:00 PM com.squareup.okhttp.mockwebserver.MockWebServer$3
processOneRequest
INFO: MockWebServer[51389] received request: PUT /testbucketsaeasttest
HTTP/1.1 and responded: HTTP/1.1 200 OK
PUT /testbucketsaeasttest HTTP/1.1
Accept: */*
User-Agent: jclouds/2.0.1 java/1.8.0_181
Date: Thu, 10 Oct 2019 10:26:00 GMT
Authorization: AWS AKIAIGKQ7V52FQQJFYJQ:KnOIQRd7WPASMopLKYMFjh2hB2M=
Content-Length: 0
Host: localhost:51389
Connection: Keep-Alive
Accept-Encoding: gzip
Oct 10, 2019 3:56:01 PM com.squareup.okhttp.mockwebserver.MockWebServer$2
acceptConnections
INFO: MockWebServer[51389] done accepting connections: Socket closed
I want to know ideally : PUT /testbucketsaeasttest HTTP/1.1 should have
region also appended with it.
I want your inputs here, is this right flow or need changes?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [jclouds] nacx commented on issue #47: J clouds 1516
nacx commented on issue #47: J clouds 1516 URL: https://github.com/apache/jclouds/pull/47#issuecomment-540481010 You just need to mock the responses. Mockwebserver has a queue (FIFO) of mocked requests it serves upon request. You just need to configure it with mock responses, then use the S3 client normally to call the method you modified. Mockwebserver will record all requests it has received, so you can verify that the HTTP request that was generated by invoking your method has the expected values. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
