[jira] [Updated] (JCLOUDS-1520) JClouds is not using the JDK's KeepAliveCache when UntrustedSSLContextSupplier is used
[ https://issues.apache.org/jira/browse/JCLOUDS-1520?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Gaul updated JCLOUDS-1520: - Fix Version/s: (was: 2.2.1) (was: 2.3.0) > JClouds is not using the JDK's KeepAliveCache when > UntrustedSSLContextSupplier is used > -- > > Key: JCLOUDS-1520 > URL: https://issues.apache.org/jira/browse/JCLOUDS-1520 > Project: jclouds > Issue Type: Bug > Components: jclouds-core >Affects Versions: 2.1.0 >Reporter: Roded Bahat >Priority: Major > Attachments: screenshot-1.png > > Time Spent: 4h 20m > Remaining Estimate: 0h > > It seems like the fact that {{UntrustedSSLContextSupplier}} returns a new > {{SSLContext}} on every {{get()}} call causes a consistent cache miss on the > JVM's {{sun.net.www.http.KeepAliveCache}} which causes JClouds to not reuse > existing TLS connections even though it could. > The cache miss happens at {{sun.net.www.protocol.https.HttpsClient}} line 329 > (openjdk version "1.8.0_222"): > {noformat} > /* see if one's already around */ > ret = (HttpsClient) kac.get(url, sf); > {noformat} > To reproduce, consider the following main: > {noformat} > public static void main(String[] args) { > Properties overrides = new Properties(); > overrides.setProperty(org.jclouds.Constants.PROPERTY_TRUST_ALL_CERTS, > "true"); > BlobStoreContext blobStoreContext = > ContextBuilder.newBuilder("aws-s3") > .endpoint("https://s3.amazonaws.com;) > .credentials("...", "...") > .overrides(overrides) > .buildView(BlobStoreContext.class); > BlobStore blobStore = blobStoreContext.getBlobStore(); > blobStore.getBlob("roded-data", "blobname"); > blobStore.getBlob("roded-data", "blobname"); > blobStore.getBlob("roded-data", "blobname"); > blobStoreContext.close(); > System.exit(0); > } > {noformat} > If run using a JUL logging.properties with the following logger set to FINEST: > {noformat} > sun.net.www.protocol.http.level=FINEST > {noformat} > The following log is produced: > {noformat} > 2019-10-10 18:15:19.668 FINE[org.jclouds.rest.internal.InvokeHttpMethod ] > >> invoking GetBucketLocation > 2019-10-10 18:15:19.733 FINE > [org.jclouds.http.internal.JavaUrlHttpCommandExecutorService ] Sending > request -1721710788: GET https://s3.amazonaws.com/roded-data?location HTTP/1.1 > 2019-10-10 18:15:19.893 FINEST [sun.net.www.protocol.http.HttpURLConnection > ] Looking for HttpClient for URL https://s3.amazonaws.com/roded-data?location > and proxy value of DIRECT > 2019-10-10 18:15:19.893 FINEST [sun.net.www.protocol.http.HttpURLConnection > ] Creating new HttpsClient with > url:https://s3.amazonaws.com/roded-data?location and proxy:DIRECT with > connect timeout:6 > 2019-10-10 18:15:20.837 FINE[sun.net.www.protocol.http.HttpURLConnection > ] sun.net.www.MessageHeader@537b32ef8 pairs: {GET /roded-data?location > HTTP/1.1: null}{x-amz-content-sha256: > e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855}{X-Amz-Date: > 20191010T151519Z}{Authorization: AWS4-HMAC-SHA256 > Credential=AKIAJO5RLGWKFW5ASG3A/20191010/us-east-1/s3/aws4_request, > SignedHeaders=host;x-amz-content-sha256;x-amz-date, > Signature=896e11ddd9efac465b6ff2506d1688d454a50b3f73ac68d557ad036b1826e591}{User-Agent: > jclouds/2019.224.2 java/1.8.0_222}{Host: s3.amazonaws.com}{Accept: > text/html, image/gif, image/jpeg, *; q=.2, */*; q=.2}{Connection: keep-alive} > 2019-10-10 18:15:21.169 FINE[sun.net.www.protocol.http.HttpURLConnection > ] sun.net.www.MessageHeader@6f815e7f7 pairs: {null: HTTP/1.1 200 > OK}{x-amz-id-2: > 1VVlx4h/fBOFe3n/7IxvpWN0RoVcE2rSpnnxMjvAQ93lJ6tHJAS+3IlXAx++/ZMEblp7kjJT4eQ=}{x-amz-request-id: > AE0779131201B495}{Date: Thu, 10 Oct 2019 15:15:21 GMT}{Content-Type: > application/xml}{Transfer-Encoding: chunked}{Server: AmazonS3} > 2019-10-10 18:15:21.185 FINE > [org.jclouds.http.internal.JavaUrlHttpCommandExecutorService ] Receiving > response -1721710788: HTTP/1.1 200 OK > 2019-10-10 18:15:21.500 FINE[org.jclouds.rest.internal.InvokeHttpMethod ] > >> invoking GetObject > 2019-10-10 18:15:21.514 FINE > [org.jclouds.http.internal.JavaUrlHttpCommandExecutorService ] Sending > request -1332190413: GET > https://roded-data.s3-eu-central-1.amazonaws.com/blobname HTTP/1.1 > 2019-10-10 18:15:21.517 FINEST [sun.net.www.protocol.http.HttpURLConnection > ] Looking for HttpClient for URL > https://roded-data.s3-eu-central-1.amazonaws.com/blobname and proxy value of > DIRECT > 2019-10-10 18:15:21.519 FINEST [sun.net.www.protocol.http.HttpURLConnection > ] Creating new HttpsClient with >
[jira] [Updated] (JCLOUDS-1520) JClouds is not using the JDK's KeepAliveCache when UntrustedSSLContextSupplier is used
[ https://issues.apache.org/jira/browse/JCLOUDS-1520?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Roded Bahat updated JCLOUDS-1520: - Attachment: screenshot-1.png > JClouds is not using the JDK's KeepAliveCache when > UntrustedSSLContextSupplier is used > -- > > Key: JCLOUDS-1520 > URL: https://issues.apache.org/jira/browse/JCLOUDS-1520 > Project: jclouds > Issue Type: Bug > Components: jclouds-core >Affects Versions: 2.1.0 >Reporter: Roded Bahat >Priority: Major > Attachments: screenshot-1.png > > > It seems like the fact that {{UntrustedSSLContextSupplier}} returns a new > {{SSLContext}} on every {{get()}} call causes a consistent cache miss on the > JVM's {{sun.net.www.http.KeepAliveCache}} which causes JClouds to not reuse > existing TLS connections even though it could. > The cache miss happens at {{sun.net.www.protocol.https.HttpsClient}} line 329 > (openjdk version "1.8.0_222"): > {noformat} > /* see if one's already around */ > ret = (HttpsClient) kac.get(url, sf); > {noformat} > To reproduce, consider the following main: > {noformat} > public static void main(String[] args) { > Properties overrides = new Properties(); > overrides.setProperty(org.jclouds.Constants.PROPERTY_TRUST_ALL_CERTS, > "true"); > BlobStoreContext blobStoreContext = > ContextBuilder.newBuilder("aws-s3") > .endpoint("https://s3.amazonaws.com;) > .credentials("...", "...") > .overrides(overrides) > .buildView(BlobStoreContext.class); > BlobStore blobStore = blobStoreContext.getBlobStore(); > blobStore.getBlob("roded-data", "blobname"); > blobStore.getBlob("roded-data", "blobname"); > blobStore.getBlob("roded-data", "blobname"); > blobStoreContext.close(); > System.exit(0); > } > {noformat} > If run using a JUL logging.properties with the following logger set to FINEST: > {noformat} > sun.net.www.protocol.http.level=FINEST > {noformat} > The following log is produced: > {noformat} > 2019-10-10 18:15:19.668 FINE[org.jclouds.rest.internal.InvokeHttpMethod ] > >> invoking GetBucketLocation > 2019-10-10 18:15:19.733 FINE > [org.jclouds.http.internal.JavaUrlHttpCommandExecutorService ] Sending > request -1721710788: GET https://s3.amazonaws.com/roded-data?location HTTP/1.1 > 2019-10-10 18:15:19.893 FINEST [sun.net.www.protocol.http.HttpURLConnection > ] Looking for HttpClient for URL https://s3.amazonaws.com/roded-data?location > and proxy value of DIRECT > 2019-10-10 18:15:19.893 FINEST [sun.net.www.protocol.http.HttpURLConnection > ] Creating new HttpsClient with > url:https://s3.amazonaws.com/roded-data?location and proxy:DIRECT with > connect timeout:6 > 2019-10-10 18:15:20.837 FINE[sun.net.www.protocol.http.HttpURLConnection > ] sun.net.www.MessageHeader@537b32ef8 pairs: {GET /roded-data?location > HTTP/1.1: null}{x-amz-content-sha256: > e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855}{X-Amz-Date: > 20191010T151519Z}{Authorization: AWS4-HMAC-SHA256 > Credential=AKIAJO5RLGWKFW5ASG3A/20191010/us-east-1/s3/aws4_request, > SignedHeaders=host;x-amz-content-sha256;x-amz-date, > Signature=896e11ddd9efac465b6ff2506d1688d454a50b3f73ac68d557ad036b1826e591}{User-Agent: > jclouds/2019.224.2 java/1.8.0_222}{Host: s3.amazonaws.com}{Accept: > text/html, image/gif, image/jpeg, *; q=.2, */*; q=.2}{Connection: keep-alive} > 2019-10-10 18:15:21.169 FINE[sun.net.www.protocol.http.HttpURLConnection > ] sun.net.www.MessageHeader@6f815e7f7 pairs: {null: HTTP/1.1 200 > OK}{x-amz-id-2: > 1VVlx4h/fBOFe3n/7IxvpWN0RoVcE2rSpnnxMjvAQ93lJ6tHJAS+3IlXAx++/ZMEblp7kjJT4eQ=}{x-amz-request-id: > AE0779131201B495}{Date: Thu, 10 Oct 2019 15:15:21 GMT}{Content-Type: > application/xml}{Transfer-Encoding: chunked}{Server: AmazonS3} > 2019-10-10 18:15:21.185 FINE > [org.jclouds.http.internal.JavaUrlHttpCommandExecutorService ] Receiving > response -1721710788: HTTP/1.1 200 OK > 2019-10-10 18:15:21.500 FINE[org.jclouds.rest.internal.InvokeHttpMethod ] > >> invoking GetObject > 2019-10-10 18:15:21.514 FINE > [org.jclouds.http.internal.JavaUrlHttpCommandExecutorService ] Sending > request -1332190413: GET > https://roded-data.s3-eu-central-1.amazonaws.com/blobname HTTP/1.1 > 2019-10-10 18:15:21.517 FINEST [sun.net.www.protocol.http.HttpURLConnection > ] Looking for HttpClient for URL > https://roded-data.s3-eu-central-1.amazonaws.com/blobname and proxy value of > DIRECT > 2019-10-10 18:15:21.519 FINEST [sun.net.www.protocol.http.HttpURLConnection > ] Creating new HttpsClient with > url:https://roded-data.s3-eu-central-1.amazonaws.com/blobname and > proxy:DIRECT with connect timeout:6 > 2019-10-10
[jira] [Updated] (JCLOUDS-1520) JClouds is not using the JDK's KeepAliveCache when UntrustedSSLContextSupplier is used
[ https://issues.apache.org/jira/browse/JCLOUDS-1520?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Roded Bahat updated JCLOUDS-1520: - Description: It seems like the fact that {{UntrustedSSLContextSupplier}} returns a new {{SSLContext}} on every {{get()}} call causes a consistent cache miss on the JVM's {{sun.net.www.http.KeepAliveCache}} which causes JClouds to not reuse existing TLS connections even though it could. The cache miss happens at {{sun.net.www.protocol.https.HttpsClient}} line 329 (openjdk version "1.8.0_222"): {noformat} /* see if one's already around */ ret = (HttpsClient) kac.get(url, sf); {noformat} To reproduce, consider the following main: {noformat} public static void main(String[] args) { Properties overrides = new Properties(); overrides.setProperty(org.jclouds.Constants.PROPERTY_TRUST_ALL_CERTS, "true"); BlobStoreContext blobStoreContext = ContextBuilder.newBuilder("aws-s3") .endpoint("https://s3.amazonaws.com;) .credentials("...", "...") .overrides(overrides) .buildView(BlobStoreContext.class); BlobStore blobStore = blobStoreContext.getBlobStore(); blobStore.getBlob("roded-data", "blobname"); blobStore.getBlob("roded-data", "blobname"); blobStore.getBlob("roded-data", "blobname"); blobStoreContext.close(); System.exit(0); } {noformat} If run using a JUL logging.properties with the following logger set to FINEST: {noformat} sun.net.www.protocol.http.level=FINEST {noformat} The following log is produced: {noformat} 2019-10-10 18:15:19.668 FINE[org.jclouds.rest.internal.InvokeHttpMethod ] >> invoking GetBucketLocation 2019-10-10 18:15:19.733 FINE [org.jclouds.http.internal.JavaUrlHttpCommandExecutorService ] Sending request -1721710788: GET https://s3.amazonaws.com/roded-data?location HTTP/1.1 2019-10-10 18:15:19.893 FINEST [sun.net.www.protocol.http.HttpURLConnection ] Looking for HttpClient for URL https://s3.amazonaws.com/roded-data?location and proxy value of DIRECT 2019-10-10 18:15:19.893 FINEST [sun.net.www.protocol.http.HttpURLConnection ] Creating new HttpsClient with url:https://s3.amazonaws.com/roded-data?location and proxy:DIRECT with connect timeout:6 2019-10-10 18:15:20.837 FINE[sun.net.www.protocol.http.HttpURLConnection ] sun.net.www.MessageHeader@537b32ef8 pairs: {GET /roded-data?location HTTP/1.1: null}{x-amz-content-sha256: e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855}{X-Amz-Date: 20191010T151519Z}{Authorization: AWS4-HMAC-SHA256 Credential=AKIAJO5RLGWKFW5ASG3A/20191010/us-east-1/s3/aws4_request, SignedHeaders=host;x-amz-content-sha256;x-amz-date, Signature=896e11ddd9efac465b6ff2506d1688d454a50b3f73ac68d557ad036b1826e591}{User-Agent: jclouds/2019.224.2 java/1.8.0_222}{Host: s3.amazonaws.com}{Accept: text/html, image/gif, image/jpeg, *; q=.2, */*; q=.2}{Connection: keep-alive} 2019-10-10 18:15:21.169 FINE[sun.net.www.protocol.http.HttpURLConnection ] sun.net.www.MessageHeader@6f815e7f7 pairs: {null: HTTP/1.1 200 OK}{x-amz-id-2: 1VVlx4h/fBOFe3n/7IxvpWN0RoVcE2rSpnnxMjvAQ93lJ6tHJAS+3IlXAx++/ZMEblp7kjJT4eQ=}{x-amz-request-id: AE0779131201B495}{Date: Thu, 10 Oct 2019 15:15:21 GMT}{Content-Type: application/xml}{Transfer-Encoding: chunked}{Server: AmazonS3} 2019-10-10 18:15:21.185 FINE [org.jclouds.http.internal.JavaUrlHttpCommandExecutorService ] Receiving response -1721710788: HTTP/1.1 200 OK 2019-10-10 18:15:21.500 FINE[org.jclouds.rest.internal.InvokeHttpMethod ] >> invoking GetObject 2019-10-10 18:15:21.514 FINE [org.jclouds.http.internal.JavaUrlHttpCommandExecutorService ] Sending request -1332190413: GET https://roded-data.s3-eu-central-1.amazonaws.com/blobname HTTP/1.1 2019-10-10 18:15:21.517 FINEST [sun.net.www.protocol.http.HttpURLConnection ] Looking for HttpClient for URL https://roded-data.s3-eu-central-1.amazonaws.com/blobname and proxy value of DIRECT 2019-10-10 18:15:21.519 FINEST [sun.net.www.protocol.http.HttpURLConnection ] Creating new HttpsClient with url:https://roded-data.s3-eu-central-1.amazonaws.com/blobname and proxy:DIRECT with connect timeout:6 2019-10-10 18:15:22.319 FINE[sun.net.www.protocol.http.HttpURLConnection ] sun.net.www.MessageHeader@6a933be28 pairs: {GET /blobname HTTP/1.1: null}{x-amz-content-sha256: e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855}{X-Amz-Date: 20191010T151519Z}{Authorization: AWS4-HMAC-SHA256 Credential=AKIAJO5RLGWKFW5ASG3A/20191010/eu-central-1/s3/aws4_request, SignedHeaders=host;x-amz-content-sha256;x-amz-date, Signature=52fadcda579bb56e62b9b0489d7a0a90080103213021b0ea26d63f9e9620f4cc}{User-Agent: jclouds/2019.224.2 java/1.8.0_222}{Host: roded-data.s3-eu-central-1.amazonaws.com}{Accept: text/html, image/gif, image/jpeg, *; q=.2, */*; q=.2}{Connection: keep-alive} 2019-10-10 18:15:22.414 FINE
[jira] [Updated] (JCLOUDS-1520) JClouds is not using the JDK's KeepAliveCache when UntrustedSSLContextSupplier is used
[ https://issues.apache.org/jira/browse/JCLOUDS-1520?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Roded Bahat updated JCLOUDS-1520: - Description: It seems like the fact that {{UntrustedSSLContextSupplier}} returns a new {{SSLContext}} on every {{get()}} call causes a consistent cache miss on the JVM's {{sun.net.www.http.KeepAliveCache}} which causes JClouds to not reuse existing TLS connections even though it could. The cache miss happens at {{sun.net.www.protocol.https.HttpsClient}} line 329 (openjdk version "1.8.0_222"): {noformat} /* see if one's already around */ ret = (HttpsClient) kac.get(url, sf); {noformat} To reproduce, consider the following main: {noformat} public static void main(String[] args) { Properties overrides = new Properties(); overrides.setProperty(org.jclouds.Constants.PROPERTY_TRUST_ALL_CERTS, "true"); BlobStoreContext blobStoreContext = ContextBuilder.newBuilder("aws-s3") .endpoint("https://s3.amazonaws.com;) .credentials("...", "...") .overrides(overrides) .buildView(BlobStoreContext.class); BlobStore blobStore = blobStoreContext.getBlobStore(); blobStore.getBlob("roded-data", "blobname"); blobStore.getBlob("roded-data", "blobname"); blobStore.getBlob("roded-data", "blobname"); blobStoreContext.close(); System.exit(0); } {noformat} If run using a JUL logging.properties with the following logger set to FINEST: {noformat} sun.net.www.protocol.http.level=FINEST {noformat} The following log is produced: {noformat} 2019-10-10 18:15:19.668 FINE[org.jclouds.rest.internal.InvokeHttpMethod ] >> invoking GetBucketLocation 2019-10-10 18:15:19.733 FINE [org.jclouds.http.internal.JavaUrlHttpCommandExecutorService ] Sending request -1721710788: GET https://s3.amazonaws.com/roded-data?location HTTP/1.1 2019-10-10 18:15:19.893 FINEST [sun.net.www.protocol.http.HttpURLConnection ] Looking for HttpClient for URL https://s3.amazonaws.com/roded-data?location and proxy value of DIRECT 2019-10-10 18:15:19.893 FINEST [sun.net.www.protocol.http.HttpURLConnection ] Creating new HttpsClient with url:https://s3.amazonaws.com/roded-data?location and proxy:DIRECT with connect timeout:6 2019-10-10 18:15:20.837 FINE[sun.net.www.protocol.http.HttpURLConnection ] sun.net.www.MessageHeader@537b32ef8 pairs: {GET /roded-data?location HTTP/1.1: null}{x-amz-content-sha256: e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855}{X-Amz-Date: 20191010T151519Z}{Authorization: AWS4-HMAC-SHA256 Credential=AKIAJO5RLGWKFW5ASG3A/20191010/us-east-1/s3/aws4_request, SignedHeaders=host;x-amz-content-sha256;x-amz-date, Signature=896e11ddd9efac465b6ff2506d1688d454a50b3f73ac68d557ad036b1826e591}{User-Agent: jclouds/2019.224.2 java/1.8.0_222}{Host: s3.amazonaws.com}{Accept: text/html, image/gif, image/jpeg, *; q=.2, */*; q=.2}{Connection: keep-alive} 2019-10-10 18:15:21.169 FINE[sun.net.www.protocol.http.HttpURLConnection ] sun.net.www.MessageHeader@6f815e7f7 pairs: {null: HTTP/1.1 200 OK}{x-amz-id-2: 1VVlx4h/fBOFe3n/7IxvpWN0RoVcE2rSpnnxMjvAQ93lJ6tHJAS+3IlXAx++/ZMEblp7kjJT4eQ=}{x-amz-request-id: AE0779131201B495}{Date: Thu, 10 Oct 2019 15:15:21 GMT}{Content-Type: application/xml}{Transfer-Encoding: chunked}{Server: AmazonS3} 2019-10-10 18:15:21.185 FINE [org.jclouds.http.internal.JavaUrlHttpCommandExecutorService ] Receiving response -1721710788: HTTP/1.1 200 OK 2019-10-10 18:15:21.500 FINE[org.jclouds.rest.internal.InvokeHttpMethod ] >> invoking GetObject 2019-10-10 18:15:21.514 FINE [org.jclouds.http.internal.JavaUrlHttpCommandExecutorService ] Sending request -1332190413: GET https://roded-data.s3-eu-central-1.amazonaws.com/blobname HTTP/1.1 2019-10-10 18:15:21.517 FINEST [sun.net.www.protocol.http.HttpURLConnection ] Looking for HttpClient for URL https://roded-data.s3-eu-central-1.amazonaws.com/blobname and proxy value of DIRECT 2019-10-10 18:15:21.519 FINEST [sun.net.www.protocol.http.HttpURLConnection ] Creating new HttpsClient with url:https://roded-data.s3-eu-central-1.amazonaws.com/blobname and proxy:DIRECT with connect timeout:6 2019-10-10 18:15:22.319 FINE[sun.net.www.protocol.http.HttpURLConnection ] sun.net.www.MessageHeader@6a933be28 pairs: {GET /blobname HTTP/1.1: null}{x-amz-content-sha256: e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855}{X-Amz-Date: 20191010T151519Z}{Authorization: AWS4-HMAC-SHA256 Credential=AKIAJO5RLGWKFW5ASG3A/20191010/eu-central-1/s3/aws4_request, SignedHeaders=host;x-amz-content-sha256;x-amz-date, Signature=52fadcda579bb56e62b9b0489d7a0a90080103213021b0ea26d63f9e9620f4cc}{User-Agent: jclouds/2019.224.2 java/1.8.0_222}{Host: roded-data.s3-eu-central-1.amazonaws.com}{Accept: text/html, image/gif, image/jpeg, *; q=.2, */*; q=.2}{Connection: keep-alive} 2019-10-10 18:15:22.414 FINE
[jira] [Updated] (JCLOUDS-1520) JClouds is not using the JDK's KeepAliveCache when UntrustedSSLContextSupplier is used
[ https://issues.apache.org/jira/browse/JCLOUDS-1520?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Roded Bahat updated JCLOUDS-1520: - Description: It seems like the fact that {{UntrustedSSLContextSupplier}} returns a new {{SSLContext}} on every {{get()}} call causes a consistent cache miss on the JVM's {{sun.net.www.http.KeepAliveCache}} which causes JClouds to not reusing existing TLS connections even though it could. The cache miss happens at {{sun.net.www.protocol.https.HttpsClient}} line 329 (openjdk version "1.8.0_222"): {noformat} /* see if one's already around */ ret = (HttpsClient) kac.get(url, sf); {noformat} To reproduce, consider the following main: {noformat} public static void main(String[] args) { Properties overrides = new Properties(); overrides.setProperty(org.jclouds.Constants.PROPERTY_TRUST_ALL_CERTS, "true"); BlobStoreContext blobStoreContext = ContextBuilder.newBuilder("aws-s3") .endpoint("https://s3.amazonaws.com;) .credentials("...", "...") .overrides(overrides) .buildView(BlobStoreContext.class); BlobStore blobStore = blobStoreContext.getBlobStore(); blobStore.getBlob("roded-data", "blobname"); blobStore.getBlob("roded-data", "blobname"); blobStore.getBlob("roded-data", "blobname"); blobStoreContext.close(); System.exit(0); } {noformat} If run using a JUL logging.properties with the following logger set to FINEST: {noformat} sun.net.www.protocol.http.level=FINEST {noformat} The following log is produced: {noformat} 2019-10-10 18:15:19.668 FINE[org.jclouds.rest.internal.InvokeHttpMethod ] >> invoking GetBucketLocation 2019-10-10 18:15:19.733 FINE [org.jclouds.http.internal.JavaUrlHttpCommandExecutorService ] Sending request -1721710788: GET https://s3.amazonaws.com/roded-data?location HTTP/1.1 2019-10-10 18:15:19.893 FINEST [sun.net.www.protocol.http.HttpURLConnection ] Looking for HttpClient for URL https://s3.amazonaws.com/roded-data?location and proxy value of DIRECT 2019-10-10 18:15:19.893 FINEST [sun.net.www.protocol.http.HttpURLConnection ] Creating new HttpsClient with url:https://s3.amazonaws.com/roded-data?location and proxy:DIRECT with connect timeout:6 2019-10-10 18:15:20.837 FINE[sun.net.www.protocol.http.HttpURLConnection ] sun.net.www.MessageHeader@537b32ef8 pairs: {GET /roded-data?location HTTP/1.1: null}{x-amz-content-sha256: e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855}{X-Amz-Date: 20191010T151519Z}{Authorization: AWS4-HMAC-SHA256 Credential=AKIAJO5RLGWKFW5ASG3A/20191010/us-east-1/s3/aws4_request, SignedHeaders=host;x-amz-content-sha256;x-amz-date, Signature=896e11ddd9efac465b6ff2506d1688d454a50b3f73ac68d557ad036b1826e591}{User-Agent: jclouds/2019.224.2 java/1.8.0_222}{Host: s3.amazonaws.com}{Accept: text/html, image/gif, image/jpeg, *; q=.2, */*; q=.2}{Connection: keep-alive} 2019-10-10 18:15:21.169 FINE[sun.net.www.protocol.http.HttpURLConnection ] sun.net.www.MessageHeader@6f815e7f7 pairs: {null: HTTP/1.1 200 OK}{x-amz-id-2: 1VVlx4h/fBOFe3n/7IxvpWN0RoVcE2rSpnnxMjvAQ93lJ6tHJAS+3IlXAx++/ZMEblp7kjJT4eQ=}{x-amz-request-id: AE0779131201B495}{Date: Thu, 10 Oct 2019 15:15:21 GMT}{Content-Type: application/xml}{Transfer-Encoding: chunked}{Server: AmazonS3} 2019-10-10 18:15:21.185 FINE [org.jclouds.http.internal.JavaUrlHttpCommandExecutorService ] Receiving response -1721710788: HTTP/1.1 200 OK 2019-10-10 18:15:21.500 FINE[org.jclouds.rest.internal.InvokeHttpMethod ] >> invoking GetObject 2019-10-10 18:15:21.514 FINE [org.jclouds.http.internal.JavaUrlHttpCommandExecutorService ] Sending request -1332190413: GET https://roded-data.s3-eu-central-1.amazonaws.com/blobname HTTP/1.1 2019-10-10 18:15:21.517 FINEST [sun.net.www.protocol.http.HttpURLConnection ] Looking for HttpClient for URL https://roded-data.s3-eu-central-1.amazonaws.com/blobname and proxy value of DIRECT 2019-10-10 18:15:21.519 FINEST [sun.net.www.protocol.http.HttpURLConnection ] Creating new HttpsClient with url:https://roded-data.s3-eu-central-1.amazonaws.com/blobname and proxy:DIRECT with connect timeout:6 2019-10-10 18:15:22.319 FINE[sun.net.www.protocol.http.HttpURLConnection ] sun.net.www.MessageHeader@6a933be28 pairs: {GET /blobname HTTP/1.1: null}{x-amz-content-sha256: e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855}{X-Amz-Date: 20191010T151519Z}{Authorization: AWS4-HMAC-SHA256 Credential=AKIAJO5RLGWKFW5ASG3A/20191010/eu-central-1/s3/aws4_request, SignedHeaders=host;x-amz-content-sha256;x-amz-date, Signature=52fadcda579bb56e62b9b0489d7a0a90080103213021b0ea26d63f9e9620f4cc}{User-Agent: jclouds/2019.224.2 java/1.8.0_222}{Host: roded-data.s3-eu-central-1.amazonaws.com}{Accept: text/html, image/gif, image/jpeg, *; q=.2, */*; q=.2}{Connection: keep-alive} 2019-10-10 18:15:22.414 FINE
[jira] [Updated] (JCLOUDS-1520) JClouds is not using the JDK's KeepAliveCache when UntrustedSSLContextSupplier is used
[ https://issues.apache.org/jira/browse/JCLOUDS-1520?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Roded Bahat updated JCLOUDS-1520: - Description: It seems like the fact that {{UntrustedSSLContextSupplier}} returns a new {{SSLContext}} on every {{get()}} call causes a consistent cache miss on the JVM's {{sun.net.www.http.KeepAliveCache}} Which causes JClouds to not reusing existing TLS connections even though it could. The cache miss happens at {{sun.net.www.protocol.https.HttpsClient}} line 329 (openjdk version "1.8.0_222"): {noformat} /* see if one's already around */ ret = (HttpsClient) kac.get(url, sf); {noformat} To reproduce, consider the following main: {noformat} public static void main(String[] args) { Properties overrides = new Properties(); overrides.setProperty(org.jclouds.Constants.PROPERTY_TRUST_ALL_CERTS, "true"); BlobStoreContext blobStoreContext = ContextBuilder.newBuilder("aws-s3") .endpoint("https://s3.amazonaws.com;) .credentials("...", "...") .overrides(overrides) .buildView(BlobStoreContext.class); BlobStore blobStore = blobStoreContext.getBlobStore(); blobStore.getBlob("roded-data", "blobname"); blobStore.getBlob("roded-data", "blobname"); blobStore.getBlob("roded-data", "blobname"); blobStoreContext.close(); System.exit(0); } {noformat} If run using a JUL logging.properties with the following logger set to FINEST: {noformat} sun.net.www.protocol.http.level=FINEST {noformat} The following log is produced: {noformat} 2019-10-10 18:15:19.668 FINE[org.jclouds.rest.internal.InvokeHttpMethod ] >> invoking GetBucketLocation 2019-10-10 18:15:19.733 FINE [org.jclouds.http.internal.JavaUrlHttpCommandExecutorService ] Sending request -1721710788: GET https://s3.amazonaws.com/roded-data?location HTTP/1.1 2019-10-10 18:15:19.893 FINEST [sun.net.www.protocol.http.HttpURLConnection ] Looking for HttpClient for URL https://s3.amazonaws.com/roded-data?location and proxy value of DIRECT 2019-10-10 18:15:19.893 FINEST [sun.net.www.protocol.http.HttpURLConnection ] Creating new HttpsClient with url:https://s3.amazonaws.com/roded-data?location and proxy:DIRECT with connect timeout:6 2019-10-10 18:15:20.837 FINE[sun.net.www.protocol.http.HttpURLConnection ] sun.net.www.MessageHeader@537b32ef8 pairs: {GET /roded-data?location HTTP/1.1: null}{x-amz-content-sha256: e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855}{X-Amz-Date: 20191010T151519Z}{Authorization: AWS4-HMAC-SHA256 Credential=AKIAJO5RLGWKFW5ASG3A/20191010/us-east-1/s3/aws4_request, SignedHeaders=host;x-amz-content-sha256;x-amz-date, Signature=896e11ddd9efac465b6ff2506d1688d454a50b3f73ac68d557ad036b1826e591}{User-Agent: jclouds/2019.224.2 java/1.8.0_222}{Host: s3.amazonaws.com}{Accept: text/html, image/gif, image/jpeg, *; q=.2, */*; q=.2}{Connection: keep-alive} 2019-10-10 18:15:21.169 FINE[sun.net.www.protocol.http.HttpURLConnection ] sun.net.www.MessageHeader@6f815e7f7 pairs: {null: HTTP/1.1 200 OK}{x-amz-id-2: 1VVlx4h/fBOFe3n/7IxvpWN0RoVcE2rSpnnxMjvAQ93lJ6tHJAS+3IlXAx++/ZMEblp7kjJT4eQ=}{x-amz-request-id: AE0779131201B495}{Date: Thu, 10 Oct 2019 15:15:21 GMT}{Content-Type: application/xml}{Transfer-Encoding: chunked}{Server: AmazonS3} 2019-10-10 18:15:21.185 FINE [org.jclouds.http.internal.JavaUrlHttpCommandExecutorService ] Receiving response -1721710788: HTTP/1.1 200 OK 2019-10-10 18:15:21.500 FINE[org.jclouds.rest.internal.InvokeHttpMethod ] >> invoking GetObject 2019-10-10 18:15:21.514 FINE [org.jclouds.http.internal.JavaUrlHttpCommandExecutorService ] Sending request -1332190413: GET https://roded-data.s3-eu-central-1.amazonaws.com/blobname HTTP/1.1 2019-10-10 18:15:21.517 FINEST [sun.net.www.protocol.http.HttpURLConnection ] Looking for HttpClient for URL https://roded-data.s3-eu-central-1.amazonaws.com/blobname and proxy value of DIRECT 2019-10-10 18:15:21.519 FINEST [sun.net.www.protocol.http.HttpURLConnection ] Creating new HttpsClient with url:https://roded-data.s3-eu-central-1.amazonaws.com/blobname and proxy:DIRECT with connect timeout:6 2019-10-10 18:15:22.319 FINE[sun.net.www.protocol.http.HttpURLConnection ] sun.net.www.MessageHeader@6a933be28 pairs: {GET /blobname HTTP/1.1: null}{x-amz-content-sha256: e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855}{X-Amz-Date: 20191010T151519Z}{Authorization: AWS4-HMAC-SHA256 Credential=AKIAJO5RLGWKFW5ASG3A/20191010/eu-central-1/s3/aws4_request, SignedHeaders=host;x-amz-content-sha256;x-amz-date, Signature=52fadcda579bb56e62b9b0489d7a0a90080103213021b0ea26d63f9e9620f4cc}{User-Agent: jclouds/2019.224.2 java/1.8.0_222}{Host: roded-data.s3-eu-central-1.amazonaws.com}{Accept: text/html, image/gif, image/jpeg, *; q=.2, */*; q=.2}{Connection: keep-alive} 2019-10-10 18:15:22.414 FINE
[jira] [Updated] (JCLOUDS-1520) JClouds is not using the JDK's KeepAliveCache when UntrustedSSLContextSupplier is used
[ https://issues.apache.org/jira/browse/JCLOUDS-1520?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Roded Bahat updated JCLOUDS-1520: - Description: It seems like the fact that {{UntrustedSSLContextSupplier}} returns a new {{SSLContext}} on every {{get()}} call causes a consistent cache miss on the JVM's {{sun.net.www.http.KeepAliveCache}} Which causes JClouds to not reusing existing TLS connections even though it could. The cache miss happens at {{sun.net.www.protocol.https.HttpsClient}} line 329: {noformat} /* see if one's already around */ ret = (HttpsClient) kac.get(url, sf); {noformat} To reproduce, consider the following main: {noformat} public static void main(String[] args) { Properties overrides = new Properties(); overrides.setProperty(org.jclouds.Constants.PROPERTY_TRUST_ALL_CERTS, "true"); BlobStoreContext blobStoreContext = ContextBuilder.newBuilder("aws-s3") .endpoint("https://s3.amazonaws.com;) .credentials("...", "...") .overrides(overrides) .buildView(BlobStoreContext.class); BlobStore blobStore = blobStoreContext.getBlobStore(); blobStore.getBlob("roded-data", "blobname"); blobStore.getBlob("roded-data", "blobname"); blobStore.getBlob("roded-data", "blobname"); blobStoreContext.close(); System.exit(0); } {noformat} If run using a JUL logging.properties with the following logger set to FINEST: {noformat} sun.net.www.protocol.http.level=FINEST {noformat} The following log is produced: {noformat} 2019-10-10 18:15:19.668 FINE[org.jclouds.rest.internal.InvokeHttpMethod ] >> invoking GetBucketLocation 2019-10-10 18:15:19.733 FINE [org.jclouds.http.internal.JavaUrlHttpCommandExecutorService ] Sending request -1721710788: GET https://s3.amazonaws.com/roded-data?location HTTP/1.1 2019-10-10 18:15:19.893 FINEST [sun.net.www.protocol.http.HttpURLConnection ] Looking for HttpClient for URL https://s3.amazonaws.com/roded-data?location and proxy value of DIRECT 2019-10-10 18:15:19.893 FINEST [sun.net.www.protocol.http.HttpURLConnection ] Creating new HttpsClient with url:https://s3.amazonaws.com/roded-data?location and proxy:DIRECT with connect timeout:6 2019-10-10 18:15:20.837 FINE[sun.net.www.protocol.http.HttpURLConnection ] sun.net.www.MessageHeader@537b32ef8 pairs: {GET /roded-data?location HTTP/1.1: null}{x-amz-content-sha256: e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855}{X-Amz-Date: 20191010T151519Z}{Authorization: AWS4-HMAC-SHA256 Credential=AKIAJO5RLGWKFW5ASG3A/20191010/us-east-1/s3/aws4_request, SignedHeaders=host;x-amz-content-sha256;x-amz-date, Signature=896e11ddd9efac465b6ff2506d1688d454a50b3f73ac68d557ad036b1826e591}{User-Agent: jclouds/2019.224.2 java/1.8.0_222}{Host: s3.amazonaws.com}{Accept: text/html, image/gif, image/jpeg, *; q=.2, */*; q=.2}{Connection: keep-alive} 2019-10-10 18:15:21.169 FINE[sun.net.www.protocol.http.HttpURLConnection ] sun.net.www.MessageHeader@6f815e7f7 pairs: {null: HTTP/1.1 200 OK}{x-amz-id-2: 1VVlx4h/fBOFe3n/7IxvpWN0RoVcE2rSpnnxMjvAQ93lJ6tHJAS+3IlXAx++/ZMEblp7kjJT4eQ=}{x-amz-request-id: AE0779131201B495}{Date: Thu, 10 Oct 2019 15:15:21 GMT}{Content-Type: application/xml}{Transfer-Encoding: chunked}{Server: AmazonS3} 2019-10-10 18:15:21.185 FINE [org.jclouds.http.internal.JavaUrlHttpCommandExecutorService ] Receiving response -1721710788: HTTP/1.1 200 OK 2019-10-10 18:15:21.500 FINE[org.jclouds.rest.internal.InvokeHttpMethod ] >> invoking GetObject 2019-10-10 18:15:21.514 FINE [org.jclouds.http.internal.JavaUrlHttpCommandExecutorService ] Sending request -1332190413: GET https://roded-data.s3-eu-central-1.amazonaws.com/blobname HTTP/1.1 2019-10-10 18:15:21.517 FINEST [sun.net.www.protocol.http.HttpURLConnection ] Looking for HttpClient for URL https://roded-data.s3-eu-central-1.amazonaws.com/blobname and proxy value of DIRECT 2019-10-10 18:15:21.519 FINEST [sun.net.www.protocol.http.HttpURLConnection ] Creating new HttpsClient with url:https://roded-data.s3-eu-central-1.amazonaws.com/blobname and proxy:DIRECT with connect timeout:6 2019-10-10 18:15:22.319 FINE[sun.net.www.protocol.http.HttpURLConnection ] sun.net.www.MessageHeader@6a933be28 pairs: {GET /blobname HTTP/1.1: null}{x-amz-content-sha256: e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855}{X-Amz-Date: 20191010T151519Z}{Authorization: AWS4-HMAC-SHA256 Credential=AKIAJO5RLGWKFW5ASG3A/20191010/eu-central-1/s3/aws4_request, SignedHeaders=host;x-amz-content-sha256;x-amz-date, Signature=52fadcda579bb56e62b9b0489d7a0a90080103213021b0ea26d63f9e9620f4cc}{User-Agent: jclouds/2019.224.2 java/1.8.0_222}{Host: roded-data.s3-eu-central-1.amazonaws.com}{Accept: text/html, image/gif, image/jpeg, *; q=.2, */*; q=.2}{Connection: keep-alive} 2019-10-10 18:15:22.414 FINE
[jira] [Updated] (JCLOUDS-1520) JClouds is not using the JDK's KeepAliveCache when UntrustedSSLContextSupplier is used
[ https://issues.apache.org/jira/browse/JCLOUDS-1520?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Roded Bahat updated JCLOUDS-1520: - Description: It seems like the fact that {{UntrustedSSLContextSupplier}} returns a new {{SSLContext}} on every {{get()}} call causes a consistent cache miss on the JVM's {{sun.net.www.http.KeepAliveCache}} Which causes JClouds to not reusing existing TLS connections even though it could. The cache miss happens at {{sun.net.www.protocol.https.HttpsClient}} line 329: {noformat} /* see if one's already around */ ret = (HttpsClient) kac.get(url, sf); {noformat} To reproduce, consider the following main: {noformat} public static void main(String[] args) { Properties overrides = new Properties(); overrides.setProperty(org.jclouds.Constants.PROPERTY_TRUST_ALL_CERTS, "true"); BlobStoreContext blobStoreContext = ContextBuilder.newBuilder("aws-s3") .endpoint("https://s3.amazonaws.com;) .credentials("...", "...") .overrides(overrides) .buildView(BlobStoreContext.class); BlobStore blobStore = blobStoreContext.getBlobStore(); blobStore.getBlob("roded-data", "blobname"); blobStore.getBlob("roded-data", "blobname"); blobStore.getBlob("roded-data", "blobname"); blobStoreContext.close(); System.exit(0); } {noformat} If run using a JUL logging.properties with the following logger set to FINEST: {noformat} sun.net.www.protocol.http.level=FINEST {noformat} The following log is produced: {noformat} 2019-10-10 18:15:19.668 FINE[org.jclouds.rest.internal.InvokeHttpMethod ] >> invoking GetBucketLocation 2019-10-10 18:15:19.733 FINE [org.jclouds.http.internal.JavaUrlHttpCommandExecutorService ] Sending request -1721710788: GET https://s3.amazonaws.com/roded-data?location HTTP/1.1 2019-10-10 18:15:19.893 FINEST [sun.net.www.protocol.http.HttpURLConnection ] Looking for HttpClient for URL https://s3.amazonaws.com/roded-data?location and proxy value of DIRECT 2019-10-10 18:15:19.893 FINEST [sun.net.www.protocol.http.HttpURLConnection ] Creating new HttpsClient with url:https://s3.amazonaws.com/roded-data?location and proxy:DIRECT with connect timeout:6 2019-10-10 18:15:20.837 FINE[sun.net.www.protocol.http.HttpURLConnection ] sun.net.www.MessageHeader@537b32ef8 pairs: {GET /roded-data?location HTTP/1.1: null}{x-amz-content-sha256: e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855}{X-Amz-Date: 20191010T151519Z}{Authorization: AWS4-HMAC-SHA256 Credential=AKIAJO5RLGWKFW5ASG3A/20191010/us-east-1/s3/aws4_request, SignedHeaders=host;x-amz-content-sha256;x-amz-date, Signature=896e11ddd9efac465b6ff2506d1688d454a50b3f73ac68d557ad036b1826e591}{User-Agent: jclouds/2019.224.2 java/1.8.0_222}{Host: s3.amazonaws.com}{Accept: text/html, image/gif, image/jpeg, *; q=.2, */*; q=.2}{Connection: keep-alive} 2019-10-10 18:15:21.169 FINE[sun.net.www.protocol.http.HttpURLConnection ] sun.net.www.MessageHeader@6f815e7f7 pairs: {null: HTTP/1.1 200 OK}{x-amz-id-2: 1VVlx4h/fBOFe3n/7IxvpWN0RoVcE2rSpnnxMjvAQ93lJ6tHJAS+3IlXAx++/ZMEblp7kjJT4eQ=}{x-amz-request-id: AE0779131201B495}{Date: Thu, 10 Oct 2019 15:15:21 GMT}{Content-Type: application/xml}{Transfer-Encoding: chunked}{Server: AmazonS3} 2019-10-10 18:15:21.185 FINE [org.jclouds.http.internal.JavaUrlHttpCommandExecutorService ] Receiving response -1721710788: HTTP/1.1 200 OK 2019-10-10 18:15:21.500 FINE[org.jclouds.rest.internal.InvokeHttpMethod ] >> invoking GetObject 2019-10-10 18:15:21.514 FINE [org.jclouds.http.internal.JavaUrlHttpCommandExecutorService ] Sending request -1332190413: GET https://roded-data.s3-eu-central-1.amazonaws.com/blobname HTTP/1.1 2019-10-10 18:15:21.517 FINEST [sun.net.www.protocol.http.HttpURLConnection ] Looking for HttpClient for URL https://roded-data.s3-eu-central-1.amazonaws.com/blobname and proxy value of DIRECT 2019-10-10 18:15:21.519 FINEST [sun.net.www.protocol.http.HttpURLConnection ] Creating new HttpsClient with url:https://roded-data.s3-eu-central-1.amazonaws.com/blobname and proxy:DIRECT with connect timeout:6 2019-10-10 18:15:22.319 FINE[sun.net.www.protocol.http.HttpURLConnection ] sun.net.www.MessageHeader@6a933be28 pairs: {GET /blobname HTTP/1.1: null}{x-amz-content-sha256: e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855}{X-Amz-Date: 20191010T151519Z}{Authorization: AWS4-HMAC-SHA256 Credential=AKIAJO5RLGWKFW5ASG3A/20191010/eu-central-1/s3/aws4_request, SignedHeaders=host;x-amz-content-sha256;x-amz-date, Signature=52fadcda579bb56e62b9b0489d7a0a90080103213021b0ea26d63f9e9620f4cc}{User-Agent: jclouds/2019.224.2 java/1.8.0_222}{Host: roded-data.s3-eu-central-1.amazonaws.com}{Accept: text/html, image/gif, image/jpeg, *; q=.2, */*; q=.2}{Connection: keep-alive} 2019-10-10 18:15:22.414 FINE