[
https://issues.apache.org/jira/browse/HADOOP-15669?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16578915#comment-16578915
]
Thomas Marquardt commented on HADOOP-15669:
-------------------------------------------
Thanks for the improvement to HTTPS. Can you share your performance numbers?
Overall looks great. Here's some feedback:
*AbfsHttpOperation.java:*
L186 - Seems we should use HttpsURLConnection.setDefaultSSLSocketFactory once
during AbfsClient initialization?
This would simplify the changes so that there are no changes to
AbfsHttpOperation or AbfsRestOperation,
and instead the AbfsClient constructor checks configuration and sets the
default factory.
*SSLSocketFactoryEx.java:*
L1 - You've exceeded the 80 character line limit a few times in this file,
which I think you should fix.
L25 - Move "java" imports above the "org" imports, and sort alphabetically.
Also, don't use * but do use
the full class name. I tried to find the guidelines for this, but could not.
Anyhow, you can follow
the patterns in the other source files.
L53 - .getLogger(SSLSocketFactoryEx.class);
L190 - Also set channelMode = SSLChannelMode.Default_JSSE;
L195 - Also set channelMode = SSLChannelMode.OpenSSL;
*pom.xml:*
L105-144 - I don't think we need to use shade, but if so, please explain why?
L247-251 - Generally we add dependencies to hadoop-project/pom.xml and everyone
uses the same version.
I recommend removing your changes to hadoop-azure/pom.xml, except for adding
the following,
which is similar to how codehaus is added on L241-245:
<dependency>
<groupId>org.wildfly.openssl</groupId>
<artifactId>wildfly-openssl</artifactId>
<scope>compile</scope>
</dependency>
All tests pass with fs.azure.scale.test.enabled=true set for WASB
(azure-auth-keys.xml) and ABFS (azure-bfs-auth-keys.xml):
Tests run: 262, Failures: 0, Errors: 0, Skipped: 11
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0
Tests run: 780, Failures: 0, Errors: 0, Skipped: 182
Tests run: 186, Failures: 0, Errors: 0, Skipped: 10
> ABFS: Improve HTTPS Performance
> -------------------------------
>
> Key: HADOOP-15669
> URL: https://issues.apache.org/jira/browse/HADOOP-15669
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/azure
> Reporter: Thomas Marquardt
> Assignee: Vishwajeet Dusane
> Priority: Major
> Attachments: ABFS - Improve HTTPS Performance Over Java Based
> Client.pdf, HADOOP-15669-HADOOP-15407-01.patch
>
>
> We see approximately 50% worse throughput for ABFS over HTTPs vs HTTP. Lets
> perform a detailed measurement and see what can be done to improve throughput.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
