[
https://issues.apache.org/jira/browse/HADOOP-16050?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16829315#comment-16829315
]
Sahil Takiar commented on HADOOP-16050:
---------------------------------------
Assigning to myself as I have seen similar overhead in Impala-S3 profiles. The
AWS SDK {{ClientConfiguration}} allows specifying a custom {{SSLSocketFactory}}
via {{awsConf.getApacheHttpClientConfig().setSslSocketFactory(...)}}, which
allows plugging in the Wildfly OpenSSL integration.
The approach HADOOP-15669 took was to:
(1) By default, the Wildfly OpenSSL plugin is used, but only if OpenSSL /
wildfly can be found locally, if not it falls back to the default JSEE
implementation
(2) Removes all GCM ciphers from the list of supported ciphers, but only when
running on Java 8
All of this is encapsulated inside a custom {{SSLSocketFactory}}. So
integration this code with S3A just requires some re-factoring, configuration
code, and unit tests.
The analysis done in HADOOP-15669 suggests that delegating to OpenSSL is even
faster than Java 9.
> Support setting cipher suites for s3a file system
> -------------------------------------------------
>
> Key: HADOOP-16050
> URL: https://issues.apache.org/jira/browse/HADOOP-16050
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/s3
> Affects Versions: 2.9.1
> Reporter: Justin Uang
> Assignee: Sahil Takiar
> Priority: Major
> Attachments: Screen Shot 2019-01-17 at 2.57.06 PM.png
>
>
> We have found that when running the S3AFileSystem, it picks GCM as the ssl
> cipher suite. Unfortunately this is well known to be slow on java 8:
> [https://stackoverflow.com/questions/25992131/slow-aes-gcm-encryption-and-decryption-with-java-8u20.]
>
> In practice we have seen that it can take well over 50% of our CPU time in
> spark workflows. We should add an option to set the list of cipher suites we
> would like to use. !Screen Shot 2019-01-17 at 2.57.06 PM.png!
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
